Capturing the workflows of music information retrieval for repeatability and reuse

Kevin R. Page, Ben Fields, David De Roure, Tim Crawford, J. Stephen Downie

Research output: Contribution to journalArticle

Abstract

Many solutions for the reuse and re-purposing of Music Information Retrieval (MIR) methods, and the tools implementing those methods, have been introduced over recent years. Proposals for achieving interoperability between systems have ranged from shared software libraries and interfaces, through common frameworks and portals, to standardised file formats and metadata. Here we assess these solutions for their suitability to be reused and combined as repurposable components within assemblies (or workflows) that can be used in novel and possibly more ambitious ways. Reuse and repeatability also have great implications for the process of MIR research: the encapsulation of any algorithm and its operation - including inputs, parameters, and outputs - is fundamental to the repeatability and reproducibility of an experiment. This is desirable both for the open and reliable evaluation of algorithms and for the advancement of MIR by building more effectively upon prior research. At present there is no clear best practice widely adopted by the field. Based upon our analysis of contemporary systems and their adoption we reflect as to whether this should be considered a failure. Are there limits to interoperability unique to MIR, and how might they be overcome Beyond workflows how much research context can, and should, be captured We frame our assessment within the emerging notion of Research Objects for reproducible research in other domains, and describe how their adoption could serve as a route to reuse in MIR.

Original languageEnglish (US)
Pages (from-to)435-459
Number of pages25
JournalJournal of Intelligent Information Systems
Volume41
Issue number3
DOIs
StatePublished - Dec 1 2013

Fingerprint

Information retrieval
Interoperability
Metadata
Encapsulation
Experiments

Keywords

  • Music Information Retrieval (MIR)
  • Reproducible research
  • Research Objects
  • Workflows

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Capturing the workflows of music information retrieval for repeatability and reuse. / Page, Kevin R.; Fields, Ben; Roure, David De; Crawford, Tim; Downie, J. Stephen.

In: Journal of Intelligent Information Systems, Vol. 41, No. 3, 01.12.2013, p. 435-459.

Research output: Contribution to journalArticle

Page, Kevin R. ; Fields, Ben ; Roure, David De ; Crawford, Tim ; Downie, J. Stephen. / Capturing the workflows of music information retrieval for repeatability and reuse. In: Journal of Intelligent Information Systems. 2013 ; Vol. 41, No. 3. pp. 435-459.
@article{260cf8d131f84919892e6fed104f82e0,
title = "Capturing the workflows of music information retrieval for repeatability and reuse",
abstract = "Many solutions for the reuse and re-purposing of Music Information Retrieval (MIR) methods, and the tools implementing those methods, have been introduced over recent years. Proposals for achieving interoperability between systems have ranged from shared software libraries and interfaces, through common frameworks and portals, to standardised file formats and metadata. Here we assess these solutions for their suitability to be reused and combined as repurposable components within assemblies (or workflows) that can be used in novel and possibly more ambitious ways. Reuse and repeatability also have great implications for the process of MIR research: the encapsulation of any algorithm and its operation - including inputs, parameters, and outputs - is fundamental to the repeatability and reproducibility of an experiment. This is desirable both for the open and reliable evaluation of algorithms and for the advancement of MIR by building more effectively upon prior research. At present there is no clear best practice widely adopted by the field. Based upon our analysis of contemporary systems and their adoption we reflect as to whether this should be considered a failure. Are there limits to interoperability unique to MIR, and how might they be overcome Beyond workflows how much research context can, and should, be captured We frame our assessment within the emerging notion of Research Objects for reproducible research in other domains, and describe how their adoption could serve as a route to reuse in MIR.",
keywords = "Music Information Retrieval (MIR), Reproducible research, Research Objects, Workflows",
author = "Page, {Kevin R.} and Ben Fields and Roure, {David De} and Tim Crawford and Downie, {J. Stephen}",
year = "2013",
month = "12",
day = "1",
doi = "10.1007/s10844-013-0260-9",
language = "English (US)",
volume = "41",
pages = "435--459",
journal = "Journal of Intelligent Information Systems",
issn = "0925-9902",
publisher = "Springer Netherlands",
number = "3",

}

TY - JOUR

T1 - Capturing the workflows of music information retrieval for repeatability and reuse

AU - Page, Kevin R.

AU - Fields, Ben

AU - Roure, David De

AU - Crawford, Tim

AU - Downie, J. Stephen

PY - 2013/12/1

Y1 - 2013/12/1

N2 - Many solutions for the reuse and re-purposing of Music Information Retrieval (MIR) methods, and the tools implementing those methods, have been introduced over recent years. Proposals for achieving interoperability between systems have ranged from shared software libraries and interfaces, through common frameworks and portals, to standardised file formats and metadata. Here we assess these solutions for their suitability to be reused and combined as repurposable components within assemblies (or workflows) that can be used in novel and possibly more ambitious ways. Reuse and repeatability also have great implications for the process of MIR research: the encapsulation of any algorithm and its operation - including inputs, parameters, and outputs - is fundamental to the repeatability and reproducibility of an experiment. This is desirable both for the open and reliable evaluation of algorithms and for the advancement of MIR by building more effectively upon prior research. At present there is no clear best practice widely adopted by the field. Based upon our analysis of contemporary systems and their adoption we reflect as to whether this should be considered a failure. Are there limits to interoperability unique to MIR, and how might they be overcome Beyond workflows how much research context can, and should, be captured We frame our assessment within the emerging notion of Research Objects for reproducible research in other domains, and describe how their adoption could serve as a route to reuse in MIR.

AB - Many solutions for the reuse and re-purposing of Music Information Retrieval (MIR) methods, and the tools implementing those methods, have been introduced over recent years. Proposals for achieving interoperability between systems have ranged from shared software libraries and interfaces, through common frameworks and portals, to standardised file formats and metadata. Here we assess these solutions for their suitability to be reused and combined as repurposable components within assemblies (or workflows) that can be used in novel and possibly more ambitious ways. Reuse and repeatability also have great implications for the process of MIR research: the encapsulation of any algorithm and its operation - including inputs, parameters, and outputs - is fundamental to the repeatability and reproducibility of an experiment. This is desirable both for the open and reliable evaluation of algorithms and for the advancement of MIR by building more effectively upon prior research. At present there is no clear best practice widely adopted by the field. Based upon our analysis of contemporary systems and their adoption we reflect as to whether this should be considered a failure. Are there limits to interoperability unique to MIR, and how might they be overcome Beyond workflows how much research context can, and should, be captured We frame our assessment within the emerging notion of Research Objects for reproducible research in other domains, and describe how their adoption could serve as a route to reuse in MIR.

KW - Music Information Retrieval (MIR)

KW - Reproducible research

KW - Research Objects

KW - Workflows

UR - http://www.scopus.com/inward/record.url?scp=84888343387&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84888343387&partnerID=8YFLogxK

U2 - 10.1007/s10844-013-0260-9

DO - 10.1007/s10844-013-0260-9

M3 - Article

AN - SCOPUS:84888343387

VL - 41

SP - 435

EP - 459

JO - Journal of Intelligent Information Systems

JF - Journal of Intelligent Information Systems

SN - 0925-9902

IS - 3

ER -