Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset

Hobson, Peter, Lovell, Brian C., Percannella, Gennaro, Vento, Mario and Wiliem, Arnold (2015) Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset. Artificial Intelligence in Medicine, 65 3: 239-250. doi:10.1016/j.artmed.2015.08.001

Author Hobson, Peter
Lovell, Brian C.
Percannella, Gennaro
Vento, Mario
Wiliem, Arnold
Title Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset
Journal name Artificial Intelligence in Medicine   Check publisher's open access policy
ISSN 0933-3657
Publication date 2015-11
Sub-type Article (original research)
DOI 10.1016/j.artmed.2015.08.001
Open Access Status Not Open Access
Volume 65
Issue 3
Start page 239
End page 250
Total pages 12
Place of publication Amsterdam, Netherlands
Publisher Elsevier
Collection year 2016
Language eng
Formatted abstract
Objective: This paper presents benchmarking results of human epithelial type 2 (HEp-2) interphase cell image classification methods on a very large dataset. The indirect immunofluorescence method applied on HEp-2 cells has been the gold standard to identify connective tissue diseases such as systemic lupus erythematosus and Sjögren's syndrome. However, the method suffers from numerous issues such as being subjective, time consuming and labor intensive. This has been the main motivation for the development of various computer-aided diagnosis systems whose main task is to automatically classify a given cell image into one of the predefined classes.

Methods and material: The benchmarking was performed in the form of an international competition held in conjunction with the International Conference of Image Processing in 2013: fourteen teams, composed of practitioners and researchers in this area, took part in the initiative. The system developed by each team was trained and tested on a very large HEp-2 cell dataset comprising over 68,000 images of HEp-2 cell. The dataset contains cells with six different staining patterns and two levels of fluorescence intensity. For each method we provide a brief description highlighting the design choices and an in-depth analysis on the benchmarking results.

Results: The staining pattern recognition accuracy attained by the methods varies between 47.91% and slightly above 83.65%. However, the difference between the top performing method and the seventh ranked method is only 5%. In the paper, we also study the performance achieved by fusing the best methods, finding that a recognition rate of 85.60% is reached when the top seven methods are employed.

Conclusions: We found that highest performance is obtained when using a strong classifier (typically a kernelised support vector machine) in conjunction with features extracted from local statistics. Furthermore, the misclassification profiles of the different methods highlight that some staining patterns are intrinsically more difficult to recognize. We also noted that performance is strongly affected by the fluorescence intensity level. Thus, low accuracy is to be expected when analyzing low contrasted images.
Keyword Large-scale benchmarking
Computer aided diagnosis systems
Indirect Immunofluorescence
HEp-2 cell classification
Q-Index Code C1
Q-Index Status Provisional Code
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: Official 2016 Collection
School of Information Technology and Electrical Engineering Publications
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 1 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 3 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Thu, 10 Dec 2015, 11:25:20 EST by Arnold Wiliem on behalf of School of Information Technol and Elec Engineering