Multi-feature fusion via hierarchical regression for multimedia analysis

Yang, Yi, Song, Jingkuan, Huang, Zi, Ma, Zhigang, Sebe, Nicu and Hauptmann, Alexander G. (2013) Multi-feature fusion via hierarchical regression for multimedia analysis. IEEE Transactions On Multimedia, 15 3: 572-581. doi:10.1109/TMM.2012.2234731

Attached Files (Some files may be inaccessible until you login with your UQ eSpace credentials)
Name Description MIMEType Size Downloads

Author Yang, Yi
Song, Jingkuan
Huang, Zi
Ma, Zhigang
Sebe, Nicu
Hauptmann, Alexander G.
Title Multi-feature fusion via hierarchical regression for multimedia analysis
Journal name IEEE Transactions On Multimedia   Check publisher's open access policy
ISSN 1520-9210
1941-0077
Publication date 2013-04-01
Year available 2012
Sub-type Article (original research)
DOI 10.1109/TMM.2012.2234731
Open Access Status DOI
Volume 15
Issue 3
Start page 572
End page 581
Total pages 10
Place of publication Piscataway, NJ, United States
Publisher Institute of Electrical and Electronics Engineers
Language eng
Subject 1711 Signal Processing
2214 Media Technology
1706 Computer Science Applications
2208 Electrical and Electronic Engineering
Abstract Multimedia data are usually represented by multiple features. In this paper, we propose a new algorithm, namely Multi-feature Learning via Hierarchical Regression for multimedia semantics understanding, where two issues are considered. First, labeling large amount of training data is labor-intensive. It is meaningful to effectively leverage unlabeled data to facilitate multimedia semantics understanding. Second, given that multimedia data can be represented by multiple features, it is advantageous to develop an algorithm which combines evidence obtained from different features to infer reliable multimedia semantic concept classifiers. We design a hierarchical regression model to exploit the information derived from each type of feature, which is then collaboratively fused to obtain a multimedia semantic concept classifier. Both label information and data distribution of different features representing multimedia data are considered. The algorithm can be applied to a wide range of multimedia applications and experiments are conducted on video data for video concept annotation and action recognition. Using Trecvid and CareMedia video datasets, the experimental results show that it is beneficial to combine multiple features. The performance of the proposed algorithm is remarkable when only a small amount of labeled training data are available.
Formatted abstract
Multimedia data are usually represented by multiple features. In this paper, we propose a new algorithm, namely Multi-feature Learning via Hierarchical Regression for multimedia semantics understanding, where two issues are considered. First, labeling large amount of training data is labor-intensive. It is meaningful to effectively leverage unlabeled data to facilitate multimedia semantics understanding. Second, given that multimedia data can be represented by multiple features, it is advantageous to develop an algorithm which combines evidence obtained from different features to infer reliable multimedia semantic concept classifiers. We design a hierarchical regression model to exploit the information derived from each type of feature, which is then collaboratively fused to obtain a multimedia semantic concept classifier. Both label information and data distribution of different features representing multimedia data are considered. The algorithm can be applied to a wide range of multimedia applications and experiments are conducted on video data for video concept annotation and action recognition. Using Trecvid and CareMedia video datasets, the experimental results show that it is beneficial to combine multiple features. The performance of the proposed algorithm is remarkable when only a small amount of labeled training data are available.
Keyword Action recognition
Multiple feature fusion
Semi-supervised learning
Video concept annotation
Q-Index Code C1
Q-Index Status Confirmed Code
Grant ID IIS-0812465
IIS-0917072
1RC1MH090021-01
FP7-248984 GLOCAL
Institutional Status UQ
Additional Notes Published online ahead of print: 20 December 2012.

Document type: Journal Article
Sub-type: Article (original research)
Collections: Official 2014 Collection
School of Information Technology and Electrical Engineering Publications
 
Versions
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 65 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 76 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Sun, 28 Apr 2013, 10:05:36 EST by System User on behalf of Scholarly Communication and Digitisation Service