Weakly supervised photo cropping

Zhang, Luming, Song, Mingli, Yang, Yi, Zhao, Qi, Zhao, Chen and Sebe, Nicu (2014) Weakly supervised photo cropping. IEEE Transactions on Multimedia, 16 1: 94-107. doi:10.1109/TMM.2013.2286817

Author Zhang, Luming
Song, Mingli
Yang, Yi
Zhao, Qi
Zhao, Chen
Sebe, Nicu
Title Weakly supervised photo cropping
Journal name IEEE Transactions on Multimedia   Check publisher's open access policy
ISSN 1520-9210
Publication date 2014-01-01
Year available 2013
Sub-type Article (original research)
DOI 10.1109/TMM.2013.2286817
Open Access Status Not yet assessed
Volume 16
Issue 1
Start page 94
End page 107
Total pages 14
Place of publication Piscataway, NJ, United States
Publisher Institute of Electrical and Electronics Engineers
Language eng
Abstract Photo cropping is widely used in the printing industry, photography, and cinematography. Conventional photo cropping methods suffer from three drawbacks: 1) the semantics used to describe photo aesthetics are determined by the experience of model designers and specific data sets, 2) image global configurations, an essential cue to capture photos aesthetics, are not well preserved in the cropped photo, and 3) multi-channel visual features from an image region contribute differently to human aesthetics, but state-of-the-art photo cropping methods cannot automatically weight them. Owing to the recent progress in image retrieval community, image-level semantics, i.e., photo labels obtained without much human supervision, can be efficiently and effectively acquired. Thus, we propose weakly supervised photo cropping, where a manifold embedding algorithm is developed to incorporate image-level semantics and image global configurations with graphlets, or, small-sized connected subgraph. After manifold embedding, a Bayesian Network (BN) is proposed. It incorporates the testing photo into the framework derived from the multi-channel post-embedding graphlets of the training data, the importance of which is determined automatically. Based on the BN, photo cropping can be casted as searching the candidate cropped photo that maximally preserves graphlets from the training photos, and the optimal cropping parameter is inferred by Gibbs sampling. Subjective evaluations demonstrate that: 1) our approach outperforms several representative photo cropping methods, including our previous cropping model that is guided by semantics-free graphlets, and 2) the visualized graphlets explicitly capture photo semantics and global spatial configurations.
Keyword Bayesian network
Image-level semantics
Photo cropping
Weakly supervised
Q-Index Code C1
Q-Index Status Confirmed Code
Institutional Status UQ
Additional Notes Date of publication: 22 October 2013.

Document type: Journal Article
Sub-type: Article (original research)
Collections: Official 2014 Collection
School of Information Technology and Electrical Engineering Publications
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 45 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 50 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Tue, 14 Jan 2014, 10:23:41 EST by System User on behalf of School of Information Technol and Elec Engineering