Image aesthetics assessment using Deep Chatterjee's machine

Zhangyang Wang, Ding Liu, Shiyu Chang, Florin Dolcos, Diane M Beck, Thomas S Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Image aesthetics assessment has been challenging due to its subjective nature. Inspired by the Chatterjee's visual neuroscience model, we design Deep Chatterjee's Machine (DCM) tailored for this task. DCM first learns attributes through the parallel supervised pathways, on a variety of selected feature dimensions. A high-level synthesis network is trained to associate and transform those attributes into the overall aesthetics rating. We then extend DCM to predicting the distribution of human ratings, since aesthetics ratings are often subjective. We also highlight our first-of-its-kind study of label-preserving transformations in the context of aesthetics assessment, which leads to an effective data augmentation approach. Experimental results on the AVA dataset show that DCM gains significant performance improvement, compared to other state-of-the-art models.

Original languageEnglish (US)
Title of host publication2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages941-948
Number of pages8
ISBN (Electronic)9781509061815
DOIs
StatePublished - Jun 30 2017
Event2017 International Joint Conference on Neural Networks, IJCNN 2017 - Anchorage, United States
Duration: May 14 2017May 19 2017

Publication series

NameProceedings of the International Joint Conference on Neural Networks
Volume2017-May

Other

Other2017 International Joint Conference on Neural Networks, IJCNN 2017
CountryUnited States
CityAnchorage
Period5/14/175/19/17

Fingerprint

Labels
High level synthesis

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this

Wang, Z., Liu, D., Chang, S., Dolcos, F., Beck, D. M., & Huang, T. S. (2017). Image aesthetics assessment using Deep Chatterjee's machine. In 2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings (pp. 941-948). [7965953] (Proceedings of the International Joint Conference on Neural Networks; Vol. 2017-May). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IJCNN.2017.7965953

Image aesthetics assessment using Deep Chatterjee's machine. / Wang, Zhangyang; Liu, Ding; Chang, Shiyu; Dolcos, Florin; Beck, Diane M; Huang, Thomas S.

2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. p. 941-948 7965953 (Proceedings of the International Joint Conference on Neural Networks; Vol. 2017-May).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, Z, Liu, D, Chang, S, Dolcos, F, Beck, DM & Huang, TS 2017, Image aesthetics assessment using Deep Chatterjee's machine. in 2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings., 7965953, Proceedings of the International Joint Conference on Neural Networks, vol. 2017-May, Institute of Electrical and Electronics Engineers Inc., pp. 941-948, 2017 International Joint Conference on Neural Networks, IJCNN 2017, Anchorage, United States, 5/14/17. https://doi.org/10.1109/IJCNN.2017.7965953
Wang Z, Liu D, Chang S, Dolcos F, Beck DM, Huang TS. Image aesthetics assessment using Deep Chatterjee's machine. In 2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2017. p. 941-948. 7965953. (Proceedings of the International Joint Conference on Neural Networks). https://doi.org/10.1109/IJCNN.2017.7965953
Wang, Zhangyang ; Liu, Ding ; Chang, Shiyu ; Dolcos, Florin ; Beck, Diane M ; Huang, Thomas S. / Image aesthetics assessment using Deep Chatterjee's machine. 2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 941-948 (Proceedings of the International Joint Conference on Neural Networks).
@inproceedings{77bc7921fcca4f0e895a8c3b0cec02e0,
title = "Image aesthetics assessment using Deep Chatterjee's machine",
abstract = "Image aesthetics assessment has been challenging due to its subjective nature. Inspired by the Chatterjee's visual neuroscience model, we design Deep Chatterjee's Machine (DCM) tailored for this task. DCM first learns attributes through the parallel supervised pathways, on a variety of selected feature dimensions. A high-level synthesis network is trained to associate and transform those attributes into the overall aesthetics rating. We then extend DCM to predicting the distribution of human ratings, since aesthetics ratings are often subjective. We also highlight our first-of-its-kind study of label-preserving transformations in the context of aesthetics assessment, which leads to an effective data augmentation approach. Experimental results on the AVA dataset show that DCM gains significant performance improvement, compared to other state-of-the-art models.",
author = "Zhangyang Wang and Ding Liu and Shiyu Chang and Florin Dolcos and Beck, {Diane M} and Huang, {Thomas S}",
year = "2017",
month = "6",
day = "30",
doi = "10.1109/IJCNN.2017.7965953",
language = "English (US)",
series = "Proceedings of the International Joint Conference on Neural Networks",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "941--948",
booktitle = "2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings",
address = "United States",

}

TY - GEN

T1 - Image aesthetics assessment using Deep Chatterjee's machine

AU - Wang, Zhangyang

AU - Liu, Ding

AU - Chang, Shiyu

AU - Dolcos, Florin

AU - Beck, Diane M

AU - Huang, Thomas S

PY - 2017/6/30

Y1 - 2017/6/30

N2 - Image aesthetics assessment has been challenging due to its subjective nature. Inspired by the Chatterjee's visual neuroscience model, we design Deep Chatterjee's Machine (DCM) tailored for this task. DCM first learns attributes through the parallel supervised pathways, on a variety of selected feature dimensions. A high-level synthesis network is trained to associate and transform those attributes into the overall aesthetics rating. We then extend DCM to predicting the distribution of human ratings, since aesthetics ratings are often subjective. We also highlight our first-of-its-kind study of label-preserving transformations in the context of aesthetics assessment, which leads to an effective data augmentation approach. Experimental results on the AVA dataset show that DCM gains significant performance improvement, compared to other state-of-the-art models.

AB - Image aesthetics assessment has been challenging due to its subjective nature. Inspired by the Chatterjee's visual neuroscience model, we design Deep Chatterjee's Machine (DCM) tailored for this task. DCM first learns attributes through the parallel supervised pathways, on a variety of selected feature dimensions. A high-level synthesis network is trained to associate and transform those attributes into the overall aesthetics rating. We then extend DCM to predicting the distribution of human ratings, since aesthetics ratings are often subjective. We also highlight our first-of-its-kind study of label-preserving transformations in the context of aesthetics assessment, which leads to an effective data augmentation approach. Experimental results on the AVA dataset show that DCM gains significant performance improvement, compared to other state-of-the-art models.

UR - http://www.scopus.com/inward/record.url?scp=85031042933&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85031042933&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.2017.7965953

DO - 10.1109/IJCNN.2017.7965953

M3 - Conference contribution

AN - SCOPUS:85031042933

T3 - Proceedings of the International Joint Conference on Neural Networks

SP - 941

EP - 948

BT - 2017 International Joint Conference on Neural Networks, IJCNN 2017 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

ER -