Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset

Hobson, Peter, Lovell, Brian C., Percannella, Gennaro, Vento, Mario and Wiliem, Arnold (2015) Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset. Artificial Intelligence in Medicine, 65 3: 239-250. doi:10.1016/j.artmed.2015.08.001

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads
UQ375306_OA.pdf application/pdf 1.32MB 0

Author Hobson, Peter
Lovell, Brian C.
Percannella, Gennaro
Vento, Mario
Wiliem, Arnold
Title Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset
Journal name Artificial Intelligence in Medicine   Check publisher's open access policy
ISSN 0933-3657
1873-2860
Publication date 2015-11-01
Year available 2015
Sub-type Article (original research)
DOI 10.1016/j.artmed.2015.08.001
Open Access Status File (Author Post-print)
Volume 65
Issue 3
Start page 239
End page 250
Total pages 12
Place of publication Amsterdam, Netherlands
Publisher Elsevier
Language eng
Subject 2701 Medicine (miscellaneous)
1702 Artificial Intelligence
Abstract Objective This paper presents benchmarking results of human epithelial type 2 (HEp-2) interphase cell image classification methods on a very large dataset. The indirect immunofluorescence method applied on HEp-2 cells has been the gold standard to identify connective tissue diseases such as systemic lupus erythematosus and Sjögren's syndrome. However, the method suffers from numerous issues such as being subjective, time consuming and labor intensive. This has been the main motivation for the development of various computer-aided diagnosis systems whose main task is to automatically classify a given cell image into one of the predefined classes. Methods and material The benchmarking was performed in the form of an international competition held in conjunction with the International Conference of Image Processing in 2013: fourteen teams, composed of practitioners and researchers in this area, took part in the initiative. The system developed by each team was trained and tested on a very large HEp-2 cell dataset comprising over 68,000 images of HEp-2 cell. The dataset contains cells with six different staining patterns and two levels of fluorescence intensity. For each method we provide a brief description highlighting the design choices and an in-depth analysis on the benchmarking results. Results The staining pattern recognition accuracy attained by the methods varies between 47.91% and slightly above 83.65%. However, the difference between the top performing method and the seventh ranked method is only 5%. In the paper, we also study the performance achieved by fusing the best methods, finding that a recognition rate of 85.60% is reached when the top seven methods are employed. Conclusions We found that highest performance is obtained when using a strong classifier (typically a kernelised support vector machine) in conjunction with features extracted from local statistics. Furthermore, the misclassification profiles of the different methods highlight that some staining patterns are intrinsically more difficult to recognize. We also noted that performance is strongly affected by the fluorescence intensity level. Thus, low accuracy is to be expected when analyzing low contrasted images.
Formatted abstract
Objective: This paper presents benchmarking results of human epithelial type 2 (HEp-2) interphase cell image classification methods on a very large dataset. The indirect immunofluorescence method applied on HEp-2 cells has been the gold standard to identify connective tissue diseases such as systemic lupus erythematosus and Sjögren's syndrome. However, the method suffers from numerous issues such as being subjective, time consuming and labor intensive. This has been the main motivation for the development of various computer-aided diagnosis systems whose main task is to automatically classify a given cell image into one of the predefined classes.

Methods and material: The benchmarking was performed in the form of an international competition held in conjunction with the International Conference of Image Processing in 2013: fourteen teams, composed of practitioners and researchers in this area, took part in the initiative. The system developed by each team was trained and tested on a very large HEp-2 cell dataset comprising over 68,000 images of HEp-2 cell. The dataset contains cells with six different staining patterns and two levels of fluorescence intensity. For each method we provide a brief description highlighting the design choices and an in-depth analysis on the benchmarking results.

Results: The staining pattern recognition accuracy attained by the methods varies between 47.91% and slightly above 83.65%. However, the difference between the top performing method and the seventh ranked method is only 5%. In the paper, we also study the performance achieved by fusing the best methods, finding that a recognition rate of 85.60% is reached when the top seven methods are employed.

Conclusions: We found that highest performance is obtained when using a strong classifier (typically a kernelised support vector machine) in conjunction with features extracted from local statistics. Furthermore, the misclassification profiles of the different methods highlight that some staining patterns are intrinsically more difficult to recognize. We also noted that performance is strongly affected by the fluorescence intensity level. Thus, low accuracy is to be expected when analyzing low contrasted images.
Keyword Large-scale benchmarking
Computer aided diagnosis systems
Indirect Immunofluorescence
HEp-2 cell classification
Q-Index Code C1
Q-Index Status Provisional Code
Grant ID LP130100230
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: Official 2016 Collection
School of Information Technology and Electrical Engineering Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 18 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 23 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Thu, 10 Dec 2015, 21:25:20 EST by Arnold Wiliem on behalf of School of Information Technol and Elec Engineering