A universal approximate cross-validation criterion for regular risk functions

Commenges, Daniel, Proust-Lima, Cecile, Samieri, Cecilia and Liquet, Benoit (2015) A universal approximate cross-validation criterion for regular risk functions. International Journal of Biostatistics, 11 1: 51-67. doi:10.1515/ijb-2015-0004

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads
UQ360703_OA.pdf Full text (open access) application/pdf 486.09KB 0

Author Commenges, Daniel
Proust-Lima, Cecile
Samieri, Cecilia
Liquet, Benoit
Title A universal approximate cross-validation criterion for regular risk functions
Journal name International Journal of Biostatistics   Check publisher's open access policy
ISSN 1557-4679
Publication date 2015-05-01
Year available 2015
Sub-type Article (original research)
DOI 10.1515/ijb-2015-0004
Open Access Status File (Publisher version)
Volume 11
Issue 1
Start page 51
End page 67
Total pages 17
Place of publication Berlin, Germany
Publisher Walter de Gruyter
Collection year 2016
Language eng
Abstract Selection of estimators is an essential task in modeling. A general framework is that the estimators of a distribution are obtained by minimizing a function (the estimating function) and assessed using another function (the assessment function). A classical case is that both functions estimate an information risk (specifically cross-entropy); this corresponds to using maximum likelihood estimators and assessing them by Akaike information criterion (AIC). In more general cases, the assessment risk can be estimated by leave-one-out cross-validation. Since leave-one-out cross-validation is computationally very demanding, we propose in this paper a universal approximate cross-validation criterion under regularity conditions (UACVR). This criterion can be adapted to different types of estimators, including penalized likelihood and maximum a posteriori estimators, and also to different assessment risk functions, including information risk functions and continuous rank probability score (CRPS). UACVR reduces to Takeuchi information criterion (TIC) when cross-entropy is the risk for both estimation and assessment. We provide the asymptotic distributions of UACVR and of a difference of UACVR values for two estimators. We validate UACVR using simulations and provide an illustration on real data both in the psychometric context where estimators of the distributions of ordered categorical data derived from threshold models and models based on continuous approximations are compared.
Keyword AIC
Estimator choice
Kullback-Leibler risk
Model selection
Ordered categorical observations
Psychometric tests
Q-Index Code C1
Q-Index Status Confirmed Code
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: School of Mathematics and Physics
Official 2016 Collection
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 0 times in Thomson Reuters Web of Science Article
Scopus Citation Count Cited 0 times in Scopus Article
Google Scholar Search Google Scholar
Created: Sun, 24 May 2015, 00:15:48 EST by System User on behalf of School of Mathematics & Physics