Least Squares Model Averaging for Two Non-Nested Linear Models

doi:10.1007/s11424-023-1172-6

This paper studies the least squares model averaging methods for two non-nested linear models. It is proved that the Mallows model averaging weight of the true model is root-$n$ consistent. Then the authors develop a penalized Mallows criterion which ensures that the weight of the true model equals 1 with probability tending to 1 and thus the averaging estimator is asymptotically normal. If neither candidate model is true, the penalized Mallows averaging estimator is asymptotically optimal. Simulation results show the selection consistency of the penalized Mallows method and the superiority of the model averaging approach compared with the model selection estimation.

Key words: Asymptotic optimality, consistency, least squares model averaging, non-nested models

[1] Zhang X and Zou G, Model averaging method and its application in forecast, Statistical Research, 2011, 28(6):97-102(in Chinese).
[2] Gao Y, Zhang X, Wang S, et al., Model averaging based on leave-subject-out cross-validation, Journal of Econometrics, 2016, 192:139-151.
[3] Gao Y, Luo M, and Zou G, Forecasting with model selection or model averaging:A case study for monthly container port throughput, Transportmetrica A:Transport Science, 2016, 12(4):366-384.
[4] Zhao S and Zhou J, Forecasting Chinese ports container throughput:A combining time series, Journal of Systems Science and Mathematical Sciences, 2018, 38(2):210-219(in Chinese).
[5] Gao Y, Zhou J, Wang H, et al., Forecasting Chinese ports container throughput based on the jackknife model averaging method, Journal of Systems Science and Mathematical Sciences, 2020, 40(4):729-737(in Chinese).
[6] Zhang X, Zheng Y, and Wang S, A demand forecasting method based on stochastic frontier analysis and model average:An application in air travel demand forecasting, Journal of Systems Science and Complexity, 2019, 32(2):167-185.
[7] Hoeting J A, Madigan D, Raftery A E, et al., Bayesian model averaging:A tutorial, Statistical Science, 1999, 14:382-417.
[8] Raftery A E and Zheng Y, Long-run performance of Bayesian model averaging, Journal of the American Statistical Association, 2003, 98:931-938.
[9] Buckland S T, Burnhamn K P, and Augustin N H, Model selection:An integral part of inference, Biometrics, 1997, 53:603-618.
[10] Yang Y, Mixing strategies for density estimation, Annals of Statistics, 1999, 28(1):76-87.
[11] Hjort N L and Claeskens G, Frequentist model average estimators, Journal of the American Statistical Association, 2003, 98:879-899.
[12] Yuan Z and Yang Y, Combining linear regression models:When and how, Journal of the American Statistical Association, 2005, 100:1202-1214.
[13] Hansen B E, Least squares model averaging, Econometrica, 2007, 75:1175-1189.
[14] Hansen B E, Averaging estimators for regressions with a possible structural break, Econometric Theory, 2009, 25:1498-1514.
[15] Hansen B E, Averaging estimators for autoregressions with a near unit root, Journal of Econometrics, 2010, 158:142-155.
[16] Liu Q and Okui R, Heteroskedasticity-robust Cp model averaging, Econometrics Journal, 2013, 16:463-472.
[17] Ando T and Li K, A model-averaging approach for high-dimensional regression, Journal of the American Statistical Association, 2014, 109:254-265.
[18] Ando T and Li K, A weight-relaxed model averaging approach for high-dimensional generalized linear models, Annals of Statistics, 2017, 45:2654-2679.
[19] Wan A T K, Zhang X, and Zou G, Least squares model averaging by Mallows criterion, Journal of Econometrics, 2010, 156:277-283.
[20] Zhang X, A new study on asymptotic optimality of least squares model averaging, Econometric Theory, 2021, 37:388-407.
[21] Liang H, Zou G, Wan A T K, et al., On optimal weight choice for Frequentist model average estimators, Journal of the American Statistical Association, 2011, 106(495):1053-1066.
[22] Hansen B E and Racine J S, Jackknife model averaging, Journal of Econometrics, 2012, 167:38-46.
[23] Zhang X, Wan A T K, and Zou G, Model averaging by jackknife criterion in models with dependent data, Journal of Econometrics, 2013, 174:82-94.
[24] Zhang X, Zou G, and Carroll R J, Model averaging based on Kullback-Leibler distance, Statistica Sinica, 2015, 25:1583-1598.
[25] Zhou J and Zhao S, Frequentist model averaging for high dimensional poisson models, Journal of Systems Science and Mathematical Sciences, 2018, 38(6):679-687(in Chinese).
[26] Zhao Z and Zou G, Average estimation of semiparametric models for high-dimensional longitudinal data, Journal of Systems Science and Complexity, 2020, 33(6):2013-2047.
[27] Chen X and Zhao Z, Model average for high-dimensional longitudinal data, Journal of Systems Science and Mathematical Sciences, 2020, 40(7):1297-1324(in Chinese).
[28] Zhang X, Consistency of model averaging estimators, Economics Letters, 2015, 130:120-123.
[29] Alhorn K, Dette H, and Schorning K, Optimal designs for model averaging in non-nested models, 2019, Working paper.
[30] Fang F and Liu M, Limit of the optimal weight in least squares model averaging with non-nested models, Economics Letters, 2020, 196:109586.
[31] Royston P and Thompson S G, Comparing non-nested regression models, Biometrics, 1995, 51(1):114-127.
[32] Zhang X, Zou G, Liang H, et al., Parsimonious model averaging with a diverging number of parameters, Journal of the American Statistical Association, 2020, 115(530):972-984.
[33] Akaike H, Information theory and an extension of the maximum likelihood principle, Proceedings of the Second International Symposium on Information Theory, Eds. by Petrov B N and Csaki F, Budapest, 1973, 267-281.
[34] Schwarz G, Estimating the dimension of a model, Annals of Statistics, 1978, 6(2):461-464.
[35] Mallows C L, Some comments on Cp, Technometrics, 1973, 15(4):661-675.
[36] Claeskens G and Hjort H L, Model Selection and Model Averaging, Cambridge University Press, New York, 2008.
[37] McAleer M, The significance of testing empirical non-nested models, Journal of Econometrics, 1995, 67(1):149-171.
[38] Davidson R and MacKinnon J G, Several tests for model specification in the presence of alternative hypotheses, Econometrica, 1981, 49:781-793.
[39] Watnik M R, Johnson W O, and Bedrick E J, Nonnested linear model selection revisited, Communications in Statistics-Theory and Methods, 2001, 30:1-20.
[40] Chen X, Fan Y, Wan A, et al., Post-J test inference in non-nested linear regression models, Science China Mathematics, 2015, 58(6):1203-1216.
[41] Bera A K and McAleer M, On exact and asymptotic tests of non-nested models, Statistics & Probability Letters, 1987, 5:19-22.
[42] Fan J and Li R, Variable selection via penalized likelihood and its oracle properties, Journal of the American Statistical Association, 2001, 96:1348-1360.
[43] Wahba G, Spline Models for Observational Data, Society for Industrial and Applied Mathematics, Philadelphia, 1990.
[44] Green P J and Silverman B W, Nonparametric Regression and Generalized Linear Models:A Roughness Penalty Approach, Volume 58 of Monographs on Statistics and Applied Probability, Chapman & Hall, London, 1994.
[45] Tibshirani R, Regression shrinkage and selection via lasso, Journal of the Royal Statistical Society, Series B, 1996, 58:267-288.
[46] Breiman L, Better subset regression using the nonnegative garrote, Technometrics, 1995, 37:373-384.

[1]	MA Xiaoyan, ZHOU Qin, ZI Xuemin. Multiple Change Points Detection in High-Dimensional Multivariate Regression [J]. Journal of Systems Science and Complexity, 2022, 35(6): 2278-2301.
[2]	YUAN Huifang, LIN Peng, JIANG Tao, XU Jinfeng. Model Averaging Multistep Prediction in an Infinite Order Autoregressive Process [J]. Journal of Systems Science and Complexity, 2022, 35(5): 1875-1901.
[3]	LUO Jing, QIN Hong. Asymptotic in the Ordered Networks with a Noisy Degree Sequence [J]. Journal of Systems Science and Complexity, 2022, 35(3): 1137-1153.
[4]	LIU Yue, ZOU Jiahui, ZHAO Shangwei, YANG Qinglong. Model Averaging Estimation for Varying-Coefficient Single-Index Models [J]. Journal of Systems Science and Complexity, 2022, 35(1): 264-282.
[5]	PENG Yunjie, ZHENG Xiaoqian, YU Wei, HE Kaixin, WANG Xuejun. Strong Law of Large Numbers for Weighted Sums of Random Variables and Its Applications in EV Regression Models [J]. Journal of Systems Science and Complexity, 2022, 35(1): 342-360.
[6]	JIANG Yanguang,HUANG Yi,XUE Wenchao,FANG Haitao. On Designing Consistent Extended Kalman Filter [J]. Journal of Systems Science and Complexity, 2017, 30(4): 751-764.
[7]	FAN Yan,GAI Yujie,ZHU Lixing. Asymtotics of Dantzig Selector for a General Single-Index Model [J]. Journal of Systems Science and Complexity, 2016, 29(4): 1123-1144.
[8]	LI Danping,RONG Ximin,ZHAO Hui. Time-Consistent Investment Strategy for DC Pension Plan with Stochastic Salary Under CEV Model [J]. Journal of Systems Science and Complexity, 2016, 29(2): 428-454.
[9]	HUANG Rui, CUI Hengjian. Consistency of Chi-Squared Test with Varying Number of Classes [J]. Journal of Systems Science and Complexity, 2015, 28(2): 439-450.
[10]	Huixiu ZHAO, Jinguan LIN. THE LARGE SAMPLE PROPERTIES OF THE SOLUTIONS OF GENERAL ESTIMATING EQUATIONS [J]. Journal of Systems Science and Complexity, 2012, 25(2): 315-328.
[11]	Qunying WU. FURTHER STUDY STRONG CONSISTENCY OF$M$ ESTIMATOR IN LINEAR MODEL FOR $\widetilde{\rho}$-MIXING RANDOM SAMPLES [J]. Journal of Systems Science and Complexity, 2011, 24(5): 969-980.
[12]	Tianfa XIE;Guoying LI. LARGE SAMPLE AND ROBUST PROPERTIES OF \mbox{\boldmath$\widetilde{L}^2$}-MEDIAN [J]. Journal of Systems Science and Complexity, 2010, 23(6): 1133-1142.
[13]	Kedian MU ;Zhi JIN;Didar ZOWGHI. A PRIORITY-BASED NEGOTIATIONS APPROACH FOR HANDLING INCONSISTENCIES IN MULTI-PERSPECTIVE SOFTWARE REQUIREMENTS [J]. Journal of Systems Science and Complexity, 2008, 21(4): 574-596.
[14]	Dongyang SHI;Xiaobin HAO. ACCURACY ANALYSIS FOR QUASI-CAREY ELEMENT [J]. Journal of Systems Science and Complexity, 2008, 21(3): 456-462.
[15]	Qijiang SONG;Hanfu CHEN. IDENTIFICATION FOR WIENER SYSTEMS WITH INTERNAL NOISE [J]. Journal of Systems Science and Complexity, 2008, 21(3): 378-393.

Viewed

Full text

Abstract