Ten years of MIREX: Reflections, challenges and opportunities

J. Stephen Downie, Xiao Hu, Jin Ha Lee, Kahyun Choi, Sally Jo Cunningham, Yun Hao

Research output: Contribution to conferencePaperpeer-review


The Music Information Retrieval Evaluation eXchange (MIREX) has been run annually since 2005, with the October 2014 plenary marking its tenth iteration. By 2013, MIREX has evaluated approximately 2000 individual music information retrieval (MIR) algorithms for a wide range of tasks over 37 different test collections. MIREX has involved researchers from over 29 different countries with a median of 109 individual participants per year. This paper summarizes the history of MIREX from its earliest planning meeting in 2001 to the present. It reflects upon the administrative, financial, and technological challenges MIREX has faced and describes how those challenges have been surmounted. We propose new funding models, a distributed evaluation framework, and more holistic user experience evaluation tasks—some evolutionary, some revolutionary—for the continued success of MIREX. We hope that this paper will inspire MIR community members to contribute their ideas so MIREX can have many more successful years to come.

Original languageEnglish (US)
Number of pages6
StatePublished - 2014
Event15th International Society for Music Information Retrieval Conference, ISMIR 2014 - Taipei, Taiwan, Province of China
Duration: Oct 27 2014Oct 31 2014


Conference15th International Society for Music Information Retrieval Conference, ISMIR 2014
Country/TerritoryTaiwan, Province of China

ASJC Scopus subject areas

  • Music
  • Information Systems


Dive into the research topics of 'Ten years of MIREX: Reflections, challenges and opportunities'. Together they form a unique fingerprint.

Cite this