Augmenting image descriptions using structured prediction output

Han, Y., Wei, X., Cao, X., Yang, Y. and Zhou, X. (2014) Augmenting image descriptions using structured prediction output. IEEE Transactions on Multimedia, 16 6: 1665-1676. doi:10.1109/TMM.2014.2321530

Author Han, Y.
Wei, X.
Cao, X.
Yang, Y.
Zhou, X.
Title Augmenting image descriptions using structured prediction output
Journal name IEEE Transactions on Multimedia   Check publisher's open access policy
ISSN 1520-9210
Publication date 2014-10-01
Year available 2014
Sub-type Article (original research)
DOI 10.1109/TMM.2014.2321530
Volume 16
Issue 6
Start page 1665
End page 1676
Total pages 12
Place of publication Piscataway, NJ, United States
Publisher Institute of Electrical and Electronics Engineers
Language eng
Abstract The need for richer descriptions of images arises in a wide spectrum of applications ranging from image understanding to image retrieval. While the Automatic Image Annotation (AIA) has been extensively studied, image descriptions with the output labels lack sufficient information. This paper proposes to augment image descriptions using structured prediction output. We define a hierarchical tree-structured semantic unit to describe images, from which we can obtain not only the class and subclass one image belongs to, but also the attributes one image has. After defining a new feature map function of structured SVM, we decompose the loss function into every node of the hierarchical tree-structured semantic unit and then predict the tree-structured semantic unit for testing images. In the experiments, we evaluate the performance of the proposed method on two open benchmark datasets and compare with the state-of-the-art methods. Experimental results show the better prediction performance of the proposed method and demonstrate the strength of augmenting image descriptions.
Keyword Image annotation
Image descriptions
Structured learning
Tree-structured semantic unit
Q-Index Code C1
Q-Index Status Confirmed Code
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: Official 2015 Collection
School of Information Technology and Electrical Engineering Publications
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 7 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 6 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Tue, 21 Oct 2014, 10:23:21 EST by System User on behalf of School of Information Technol and Elec Engineering