ROC curve equivalence using the Kolmogorov-Smirnov test

Bradley, Andrew P. (2013) ROC curve equivalence using the Kolmogorov-Smirnov test. Pattern Recognition Letters, 34 5: 470-475. doi:10.1016/j.patrec.2012.12.021

Author Bradley, Andrew P.
Title ROC curve equivalence using the Kolmogorov-Smirnov test
Journal name Pattern Recognition Letters   Check publisher's open access policy
ISSN 0167-8655
Publication date 2013-04
Sub-type Article (original research)
DOI 10.1016/j.patrec.2012.12.021
Volume 34
Issue 5
Start page 470
End page 475
Total pages 6
Place of publication Amsterdam, Netherlands
Publisher Elsevier
Collection year 2014
Language eng
Abstract This paper describes a simple, non-parametric and generic test of the equivalence of receiver operating characteristic (ROC) curves based on a modified Kolmogorov-Smirnov (KS) test. The test is described in relation to the commonly used techniques such as the area under the ROC curve (AUC) and the Neyman-Pearson method. We first review how the KS test is used to test the null hypotheses that the class labels predicted by a classifier are no better than random. We then propose an interval mapping technique that allows us to use two KS tests to test the null hypothesis that two classifiers have ROC curves that are equivalent. We demonstrate that this test discriminates different ROC curves both when one curve dominates another and when the curves cross and so are not discriminated by AUC. The interval mapping technique is then used to demonstrate that, although AUC has its limitations, it can be a model-independent and coherent measure of classifier performance.
Keyword ROC curves
Q-Index Code C1
Q-Index Status Confirmed Code
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: Official 2014 Collection
School of Information Technology and Electrical Engineering Publications
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 6 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 6 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Sun, 28 Apr 2013, 00:23:52 EST by System User on behalf of School of Information Technol and Elec Engineering