Manifold learning based cross-media retrieval: A solution to media object complementary nature

Zhuang, Yueting, Yang, Yi, Wu, Fei and Pan, Yunhe (2007) Manifold learning based cross-media retrieval: A solution to media object complementary nature. Journal of VLSI Signal Processing, 46 2-3: 153-164. doi:10.1007/s11265-006-0020-y

Author Zhuang, Yueting
Yang, Yi
Wu, Fei
Pan, Yunhe
Title Manifold learning based cross-media retrieval: A solution to media object complementary nature
Language of Title eng
Journal name Journal of VLSI Signal Processing   Check publisher's open access policy
Language of Journal Name eng
ISSN 0922-5773
Publication date 2007-03
Sub-type Article (original research)
DOI 10.1007/s11265-006-0020-y
Volume 46
Issue 2-3
Start page 153
End page 164
Total pages 12
Place of publication Secaucus, NJ, United States
Publisher Springer New York LLC
Language eng
Abstract Media objects of different modalities always exist jointly and they are naturally complementary of each other, either in the view of semantics or in the view of modality. In this paper, we propose a manifold learning based cross-media retrieval approach that gives solutions to the two intrinsically basic but crucial questions of media objects semantics understanding and cross-media retrieval. First, considering the semantic complementary, how can we represent the concurrent media objects and fuse the complementary information they carry to understand the integrated semantics precisely. Second, considering the modality complementary, how can we accomplish the modality bridge to establish the cross-index and facilitate the cross-media retrieval? To solve the two problems, we first construct a Multimedia Document (MMD) Semi-Semantic Graph (MMDSSG) and then adopt Multidimensional Scaling to create an MMD Semantic Space (MMDSS). Both long-term and short-term feedbacks are proposed to boost the system performance. The first one is used to refine the MMDSSG and the second one is adopted to introduce new items that are not in the training set into the MMDSS. Since all of the MMDs and their component media objects of different modalities lie in the MMDSS and they are indexed uniformly by their coordinates in the MMDSS regardless of their modalities, the semantic subspace is actually a bridge of media objects which are of different modalities and the crossmedia retrieval can be easily achieved. Experiment results are encouraging and indicate that the proposed approach is effective.
Keyword Cross-media retrieval
Semantic complementary
Modality complementary
Manifold learning
Q-Index Code C1
Q-Index Status Provisional Code
Institutional Status Non-UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: ERA 2012 Admin Only
School of Information Technology and Electrical Engineering Publications
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 8 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 11 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Tue, 06 Mar 2012, 15:42:40 EST by Mr Mathew Carter on behalf of School of Information Technol and Elec Engineering