Precision-recall operating characteristic (P-ROC) curves in imprecise environments

Landgrebe, Thomas C. W., Paclik, Pavel, Duin, Robert P. W. and Bradley, Andrew P. (2006). Precision-recall operating characteristic (P-ROC) curves in imprecise environments. In: Y. Tang, P. Wang, G. Lorette and D. S. Yeung, 18th International Conference on Pattern Recognition (ICPR'06). The 18th International Conference on Pattern Recognition (ICPR 2006), Hong Kong, (123-127). 20-24 August 2006. doi:10.1109/ICPR.2006.941

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads

Author Landgrebe, Thomas C. W.
Paclik, Pavel
Duin, Robert P. W.
Bradley, Andrew P.
Title of paper Precision-recall operating characteristic (P-ROC) curves in imprecise environments
Conference name The 18th International Conference on Pattern Recognition (ICPR 2006)
Conference location Hong Kong
Conference dates 20-24 August 2006
Proceedings title 18th International Conference on Pattern Recognition (ICPR'06)   Check publisher's open access policy
Journal name 18th International Conference on Pattern Recognition, Vol 4, Proceedings   Check publisher's open access policy
Place of Publication Piscataway, NJ, U.S.A.
Publisher IEEE - Institute of Electrical Electronics Engineers Inc.
Publication Year 2006
Sub-type Fully published paper
DOI 10.1109/ICPR.2006.941
ISBN 07695252-0
ISSN 1051-4651
Editor Y. Tang
P. Wang
G. Lorette
D. S. Yeung
Volume 4
Start page 123
End page 127
Total pages 5
Collection year 2006
Language eng
Abstract/Summary Traditionally, machine learning algorithms have been evaluated in applications where assumptions can be reliably made about class priors and/or misclassification costs. In this paper, we consider the case of imprecise environments, where little may be known about these factors and they may well vary significantly when the system is applied. Specifically, the use of precision-recall analysis is investigated and compared to the more well known performance measures such as error-rate and the receiver operating characteristic (ROC). We argue that while ROC analysis is invariant to variations in class priors, this invariance in fact hides an important factor of the evaluation in imprecise environments. Therefore, we develop a generalised precision-recall analysis methodology in which variation due to prior class probabilities is incorporated into a multi-way analysis of variance (ANOVA). The increased sensitivity and reliability of this approach is demonstrated in a remote sensing application.
Subjects E1
280207 Pattern Recognition
730111 Hearing, vision, speech and their disorders
Keyword ANOVA
Imprecise environments
Precision-recall analysis
Receiver operating characteristic
Variance analysis
Q-Index Code E1
Q-Index Status Provisional Code
Institutional Status UQ

 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 14 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 49 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Fri, 24 Aug 2007, 08:32:39 EST