CCMR: A Classic-enriched Connotation-aware Music Retrieval System on Social Media with Visual Inputs

The increasing popularity of digital music and the growing ubiquity of network connection have promoted the expansion of online music sharing platforms (e.g., YouTube, Spotify). In this paper, we focus on a challenging problem of connotation-aware music retrieval with visual inputs. The goal of the problem is to explore the implicit feeling or emotion expressed beyond the explicit contents in music and image and retrieves music pieces relevant to the connotation implicitly conveyed in the visual inputs. Two critical challenges exist in solving the connotation-aware music retrieval problem: (1) it is challenging to accurately identify the implicit connotation from both images and music pieces; (2) it is non-trivial to establish the correct connotative association across different data modalities. To address the above challenges, we present a novel classic-enriched connotation-aware music retrieval (CCMR) system to effectively identify connotation-aware music for visual inputs. We evaluate the proposed CCMR system on a real-world dataset. Results show that CCMR outperforms state-of-the-art baselines in retrieving music pieces that are highly relevant to the connotation of the visual inputs.