The music information retrieval evaluation exchange "Do-it- yourself" web service

The Do-It-Yourself (DIY) web service of the Music Information Retrieval Evaluation eXchange (MIREX) represents a means by which researchers can remotely submit, execute, and evaluate their Music Information Retrieval (MIR) algorithms against standardized datasets that are not otherwise freely distributable. Since its inception in 2005 at the International Music Information Retrieval Systems Evaluation Laboratory (IMIRSEL), MIREX has, to date, required heavy interaction by IMIRSEL team members in the execution, debugging, and validation of submitted code. The goal of the MIREX DIY web service is to put such responsibilities squarely into the hands of submitters, and also enable the evaluations of algorithms yearround, as opposed to annual exchanges.