IMIRSEL: A secure music retrieval testing environment

The Music Information Retrieval (MIR) and Music Digital Library (MDL) research communities have long noted the need for formal evaluation mechanisms. Issues concerning the unavailability of freely-available music materials have greatly hindered the creation of standardized test collections with which these communities could scientifically assess the strengths and weaknesses of their various music retrieval techniques. The International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL) is being developed at the University of Illinois at Urbana-Champaign (UIUC) specifically to overcome this hindrance to the scientific evaluation of MIR/MDL systems. Together with its subsidiary Human Use of Music Information Retrieval Systems (HUMIRS) project, IMIRSEL will allow MIR/MDL researchers access to the standardized large-scale collection of copyright-sensitive music materials and standardized test queries being housed at UIUC's National Center for Supercomputing Applications (NCSA). Virtual Research Labs (VRL), based upon NCSA's Data-to-Knowledge (D2K) tool set, are being developed through which MIR/MDL researchers will interact with the music materials under a "trusted code" security model.