两部分潜变量模型的变分贝叶斯推断

doi:10.12341/jssms22592

系统科学与数学 ›› 2023, Vol. 43 ›› Issue (4): 1039-1068.DOI: 10.12341/jssms22592

• • 上一篇

两部分潜变量模型的变分贝叶斯推断

廖雪丽¹, 陈金叶², 张琪¹, 夏业茂¹

1. 南京林业大学理学院, 南京 210037;
2. 南京林业大学经济管理学院, 南京 210037

收稿日期:2022-08-29 修回日期:2022-11-17 发布日期:2023-05-18
通讯作者: 夏业茂, Email:ymxia@njfu.edu.cn
基金资助:
国家自然科学基金面上项目(11471161)资助课题.

PDF ( 10 ) 引用

廖雪丽, 陈金叶, 张琪, 夏业茂. 两部分潜变量模型的变分贝叶斯推断[J]. 系统科学与数学, 2023, 43(4): 1039-1068.

LIAO Xueli, CHEN Jinye, ZHANG Qi, XIA Yemao. Variational Bayesian Inference for Two-Part Latent Variable Models[J]. Journal of Systems Science and Mathematical Sciences, 2023, 43(4): 1039-1068.

Variational Bayesian Inference for Two-Part Latent Variable Models

LIAO Xueli¹, CHEN Jinye², ZHANG Qi¹, XIA Yemao¹

1. School of Science, Nanjing Forestry University, Nanjing 210037;
2. College of Economics and Management, Nanjing Forestry University, Nanjing 210037

Received:2022-08-29 Revised:2022-11-17 Published:2023-05-18

两部分潜变量模型是一种被广泛用于探索半连续数据中不可观测异质性的统计方法. 文章对两部分潜变量建立变分贝叶斯推断程序. 相比于马尔可夫链蒙特卡洛(MCMC)抽样方法, 变分贝叶斯方法具有计算速度快、可提供确定性解等优点. 利用Logistic 模型一个随机表示, 构造了一个适当的变分分布族来近似后验. 变分分布通过坐标上升变分算法获得; 给出了变分参数的更新计划, 建立了变量选择和模型评价贝叶斯程序. 经验结果展示了该方法的有效性和实用价值.

关键词： 两部分潜变量模型, 变分贝叶斯, 平均场族, 贝叶斯变量选择, 家庭金融调查

Two-part latent variable models are a widely appreciated statistical method in exploring unobserved heterogeneity underlying semi-continuous data. In this paper, a variational Bayesian inference procedure is developed for the analysis of twopart latent variable model (TPLVM). Compared with Markov Chains Monte Carlo (MCMC) sampling method, the variational Bayesian method has the advantages of fast computation and providing deterministic solution. By taking advantage of the stochastic representation of logistic model, we construct a proper variational distribution family to approximate posterior. The variational distribution is achieved via implementing coordinate ascent variational algorithm. We present an update scheme for learning variational parameters and establish a Bayesian procedure for the variable selection and model selection. The empirical results illustrate the effectiveness and practical merit of the proposed methodology.

Key words: Two-part latent variable models, variational Bayes, mean-field variational family, Bayesian variable selection, household finance survey

MR(2010)主题分类:

62F15
62F35

分享此文:

[1] Cragg J G. Some statistical models for limited dependent variables with application to the demand for durable goods. Econometrica, 1971, 39(5):829-844.
[2] Manning W G, Morris C N, Newhouse J P, et al. A two-part model of the demand for medical care:Preliminary results from the health insurance experiment, in health, economics, and health economics. Eds. by der Gaag J. van and Perlman M, Amsterdam:North-Holland, 1981, 103-104.
[3] Duan N, Manning W G, Morris C N, et al. A comparison of alternative models for the demand for medical care. Journal of Business and Economic Statistics, 1983, 1:115-126.
[4] Olsen M K, Schafer J L. A two-part random-effects model for semicontinuous longitudinal data. Journal of the American Statistical Association, 2001, 96(454):730-745.
[5] Neelon B, Zhu L, Neelon S E B. Bayesian two-part spatial models for semicontinuous data with application to emergency department expenditures. Biostatistics, 2015, 16(3):465-480.
[6] Liu L. Joint modeling longitudinal semi-continuous data and survival with application to longitudinal medical cost data. Statist. Med., 2009, 28(6):972-986.
[7] Han D X, Liu L, Su X G, et al. Variable selection for random effects two-part models. Statistical Methods in Medical Research, 2019, 28(9):2697-2709.
[8] Kim Y, Muthén B O. Two-part factor mixture modeling:Application to an aggressive behavior measurement instrument. Structural Equation Modeling:A Multidisciplinary Journal, 2009, 16(4):602-624.
[9] Xu S, Blozis S A, Vandewater E A. On fitting a multivariate two-part latent growth model. Structural Equation Modeling:A Multidisciplinary Journal, 2014, 21(1):131-148.
[10] Feng X N, Lu B, Song, X Y, et al. Financial literacy and household finances:A Bayesian two-part latent variable modeling approach. Journal of Empirical Finance, 2019, 51:119-137.
[11] 夏业茂,凌耀斌,熊双粲.两部分因子分析模型的贝叶斯推断.应用数学, 2018, 31(4):761-778. (Xia Y M, Ling Y B, Xiong S C. Bayesian inference for two-part factor analytic model. Mathematica Applicata, 2018, 31(4):761-778.)
[12] Xia Y M, Lu B, Tang N S. Inference on two-part latent variable analysis model with multivariate longitudinal data. Structural Equation Modeling:A Multidisciplinary Journal, 2019, 26(5):685- 709.
[13] Gou J W, Xia Y M, Jiang D P. Bayesian analysis of two-part nonlinear latent variable model:Semiparametric method. Statistical Modeling, 2022. DOI:10.1177/1471082X211059233.
[14] Geman S, Geman D. Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1984, 6:721-741.
[15] Gelfand A E, Smith A F M. Sampling-based approaches to calculating marginal densities. Journal of the American Statistical Association, 1990, 85(410):398-409.
[16] Geyer C J. Practical Markov chain Monte Carlo. Statistical Science, 1992, 7:473-511.
[17] Tierney L. Markov chains for exploring posterior distributions (with discussion). Annals of Statistics, 1994, 22(4):1701-1762.
[18] Peterson C, Anderson J. A mean field theory learning algorithm for neural networks. Complex Systems, 1987, 1:995-1019.
[19] Jaakkola T, Jordan M I. Computing upper and lower bounds on likelihoods in intractable networks. Uncertainty Artificial Intelligence, 1996, 340-348.
[20] Jaakkola T, Jordan M I. A variational approach to Bayesian logistic regression models and their extensions. Artificial Intelligence and Statistics, 1997, 1-12.
[21] Jordan M I, Ghahramani Z, Jaakkola T, et al. Introduction to variational methods for graphical models. Machine Learning, 1999, 37:183-233.
[22] Blei D M, Jordan M I. Variational inference for Dirichelt process mixtures. Journal of Bayesian Analysis, 2006, 1:121-144.
[23] Wainwright M J, Jordan M I. Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 2008, 1:1-305.
[24] Blei D M, Kucukelbir A, Mcauliffe J D. Variational inference:A review for statisticians. Journal of the American Statistical Association, 2017, 112(518):859-877.
[25] Wang Y X, Blei D M. Frequentist consistency of variational Bayes. Journal of the American Statistical Association, 2019, 114(527):1147-1161.
[26] Polson N G, Scott J G, Windle J. Bayesian inference for logistic models using Polya-Gamma latent variables. Journal of American statistician Association, 2013, 108(504):1339-1349
[27] Bollen K A. Structural Equations with Latent Variables. New York:John Wiley, 1989.
[28] Anderson T W. An Introduction to Multivariate Statistical Analysis. New York:John Wiley, 1984.
[29] Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Ser. B, 1996, 58:267-288.
[30] Park T, Casella G. The Bayesian lasso. Journal of the American Statistical Association, 2008, 103(482):681-686.
[31] Zou H. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 2006, 101(476):1418-1429.
[32] Tanner M A, Wong W H. The calculation of posterior distributions by data augmentation (with discussion). Journal of the American statistical Association, 1987, 82:528-550.
[33] Gelfand A E, Ghosh, S K. Model choice:A minimum posterior predictive loss approach. Biometrika, 1998, 85:1-11.
[34] Feng X N, Wu H T, Song X Y. Bayesian adaptive lasso for ordinal regression with latent variables. Sociological Methods & Research, 2017, 46(4):926-953.
[35] Ibrahim J G, Chen M H, Sinha D. Criterion based methods for Bayesian model assessment. Statistica Sinica, 2001, 11:419-443.
[36] Gelman A. Inference and monitoring convergence. Markov Chain Monte Carlo in Practice (Eds. by Gilks W R, Richardson S, Spiegelhalter D J), 131-144, London:Chapman and Hall, 1996.
[37] Holmes, C C, Held L. Bayesian auxiliary variable models for binary and multinomial regression. Bayesian Analysis, 2006, 1(1):145-168.
[38] Jaakkola T, Jordan M I. Bayesian parameter estimation via variational methods. Statistics and Computing, 2000, 10:25-37.
[39] Seshadri V. The Inverse Gaussian Distribution:Statistical Theory and Applications. New York:Springer-Verlag, 1999.
[40] Jørgensen B. Statistical properties of the inverse gaussian distribution. Lectures Notes in Statistics. Eds. by Brillinger D, Fienberg S, Gani J, et al. New York:Springer-Verlag, 1982.

[1]	黄淼淇, 夏志明. 均值模型中多变点问题的平均估计方法[J]. 系统科学与数学, 0, (): 11-0.
[2]	梁永玉, 田茂再. 基于分层贝叶斯时空Poisson模型的流行病建模研究[J]. 系统科学与数学, 2022, 42(2): 462-472.
[3]	夏业茂，陈高燕，刘应安. 基于多元${\bm t}$-分布的隐马尔可夫潜变量模型的稳健推断[J]. 系统科学与数学, 2016, 36(10): 1783-1803.
[4]	李海芬;汤银才. 对数稳定分布加速寿命试验的贝叶斯分析[J]. 系统科学与数学, 2011, 31(4): 448-457.
[5]	陈雪东;唐年胜. 带有不可忽略缺失数据的半参数再生散度模型的贝叶斯分析[J]. 系统科学与数学, 2010, 30(10): 1334-1350.
[6]	熊世峰;牟唯嫣. 二项分布基于logit变换的近似信仰推断[J]. 系统科学与数学, 2009, 29(8): 1071-1078.

阅读次数

全文

摘要