The music information retrieval evaluation exchange (2005-2007): A window into music information retrieval research

Research output: Contribution to journalReview article

Abstract

The Music Information Retrieval Evaluation eXchange (MIREX) is the community-based framework for the formal evaluation of Music Information Retrieval (MIR) systems and algorithms. By looking at the background, structure, challenges, and contributions of MIREX this paper provides some insights into the world of MIR research. Because MIREX tasks are defined by the community they reflect the interests, techniques, and research paradigms of the community as a whole. Both MIREX and MIR have a strong bias toward audio-based approaches as most MIR researchers have strengths in signal processing. Spectral-based approaches to MIR tasks have led to advancements in the MIR field but they now appear to be reaching their limits of effectiveness. This limitation is called the "glass ceiling" problem and the MIREX results data support its existence. The post-hoc analyses of MIREX results data indicate that there are groups of systems that perform equally well within various MIR tasks. There are many challenges facing MIREX and MIR research most of which have their root causes in the intellectual property issues surrounding music. The current inability of researchers to test their approaches against the MIREX test collections outside the annual MIREX cycle is hindering the rapid development of improved MIR systems.

Original languageEnglish (US)
Pages (from-to)247-255
Number of pages9
JournalAcoustical Science and Technology
Volume29
Issue number4
DOIs
StatePublished - Jul 11 2008

Fingerprint

information retrieval
music
evaluation
intellectual property

Keywords

  • Evaluation
  • MIREX
  • Music information retrieval

ASJC Scopus subject areas

  • Acoustics and Ultrasonics

Cite this

@article{3f7ecf3c2b644b82beeb2cd391e69e1d,
title = "The music information retrieval evaluation exchange (2005-2007): A window into music information retrieval research",
abstract = "The Music Information Retrieval Evaluation eXchange (MIREX) is the community-based framework for the formal evaluation of Music Information Retrieval (MIR) systems and algorithms. By looking at the background, structure, challenges, and contributions of MIREX this paper provides some insights into the world of MIR research. Because MIREX tasks are defined by the community they reflect the interests, techniques, and research paradigms of the community as a whole. Both MIREX and MIR have a strong bias toward audio-based approaches as most MIR researchers have strengths in signal processing. Spectral-based approaches to MIR tasks have led to advancements in the MIR field but they now appear to be reaching their limits of effectiveness. This limitation is called the {"}glass ceiling{"} problem and the MIREX results data support its existence. The post-hoc analyses of MIREX results data indicate that there are groups of systems that perform equally well within various MIR tasks. There are many challenges facing MIREX and MIR research most of which have their root causes in the intellectual property issues surrounding music. The current inability of researchers to test their approaches against the MIREX test collections outside the annual MIREX cycle is hindering the rapid development of improved MIR systems.",
keywords = "Evaluation, MIREX, Music information retrieval",
author = "Downie, {J. Stephen}",
year = "2008",
month = "7",
day = "11",
doi = "10.1250/ast.29.247",
language = "English (US)",
volume = "29",
pages = "247--255",
journal = "Acoustical Science and Technology",
issn = "1346-3969",
publisher = "Acoustical Society of Japan",
number = "4",

}

TY - JOUR

T1 - The music information retrieval evaluation exchange (2005-2007)

T2 - A window into music information retrieval research

AU - Downie, J. Stephen

PY - 2008/7/11

Y1 - 2008/7/11

N2 - The Music Information Retrieval Evaluation eXchange (MIREX) is the community-based framework for the formal evaluation of Music Information Retrieval (MIR) systems and algorithms. By looking at the background, structure, challenges, and contributions of MIREX this paper provides some insights into the world of MIR research. Because MIREX tasks are defined by the community they reflect the interests, techniques, and research paradigms of the community as a whole. Both MIREX and MIR have a strong bias toward audio-based approaches as most MIR researchers have strengths in signal processing. Spectral-based approaches to MIR tasks have led to advancements in the MIR field but they now appear to be reaching their limits of effectiveness. This limitation is called the "glass ceiling" problem and the MIREX results data support its existence. The post-hoc analyses of MIREX results data indicate that there are groups of systems that perform equally well within various MIR tasks. There are many challenges facing MIREX and MIR research most of which have their root causes in the intellectual property issues surrounding music. The current inability of researchers to test their approaches against the MIREX test collections outside the annual MIREX cycle is hindering the rapid development of improved MIR systems.

AB - The Music Information Retrieval Evaluation eXchange (MIREX) is the community-based framework for the formal evaluation of Music Information Retrieval (MIR) systems and algorithms. By looking at the background, structure, challenges, and contributions of MIREX this paper provides some insights into the world of MIR research. Because MIREX tasks are defined by the community they reflect the interests, techniques, and research paradigms of the community as a whole. Both MIREX and MIR have a strong bias toward audio-based approaches as most MIR researchers have strengths in signal processing. Spectral-based approaches to MIR tasks have led to advancements in the MIR field but they now appear to be reaching their limits of effectiveness. This limitation is called the "glass ceiling" problem and the MIREX results data support its existence. The post-hoc analyses of MIREX results data indicate that there are groups of systems that perform equally well within various MIR tasks. There are many challenges facing MIREX and MIR research most of which have their root causes in the intellectual property issues surrounding music. The current inability of researchers to test their approaches against the MIREX test collections outside the annual MIREX cycle is hindering the rapid development of improved MIR systems.

KW - Evaluation

KW - MIREX

KW - Music information retrieval

UR - http://www.scopus.com/inward/record.url?scp=46649102100&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=46649102100&partnerID=8YFLogxK

U2 - 10.1250/ast.29.247

DO - 10.1250/ast.29.247

M3 - Review article

AN - SCOPUS:46649102100

VL - 29

SP - 247

EP - 255

JO - Acoustical Science and Technology

JF - Acoustical Science and Technology

SN - 1346-3969

IS - 4

ER -