Now That We've Found the "Hidden Web," What Can We Do with It?

Timothy W. Cole, Joanne Kaczmarek, Paul F. Marty, Christopher J. Prom, Beth Sandore, Sarah Shreeves

Research output: Chapter in Book/Report/Conference proceedingConference contribution


The Open Archives Initiative (OAI) Protocol for Metadata Harvesting (PMH) is designed to facilitate discovery of the "hidden web" of scholarly information, such as that contained in databases, finding aids, and XML documents. OAI-PMH supports standardized exchange of metadata describing items in disparate collections, of such as those held by museums and libraries. This paper describes recent work done by the University of Illinois Library, recipient of one of seven OAI-related grants from the Andrew W. Mellon Foundation. An overview is given of the process used to export metadata records describing holdings of the Spurlock Museum at the University of Illinois. These metadata records were initially created to help track artifacts as they were procured, stored, and displayed and now are used also to support end-user searching via the Spurlock Museum Website. Spurlock metadata records were mapped to Dublin Core (DC) and then harvested into the Illinois project's metadata repository. The details of the processes used to transform the Spurlock records into OAI compliant metadata and the lessons learned during this process are illustrative of the work necessary to make museum collections available using OAI-PMH. The OAI-based search and discovery services being developed by the University of Illinois are discussed. Issues, such as the need for normalization of metadata, importance of presenting search results in context, and difficulties caused by institution-to-institution variations in metadata authoring practices are covered.
Original languageEnglish (US)
Title of host publicationMuseums and the Web 2002: Selected Papers from an International Conference (6th, Boston, MA, April 17-20, 2002)
PublisherArchives & Museum Informatics
Number of pages15
StatePublished - Apr 2002


  • Open Archives Initiative (OAI)
  • metadata harvesting
  • Dublin Core
  • cultural heritage
  • interoperability
  • heterogeneous collections


Dive into the research topics of 'Now That We've Found the "Hidden Web," What Can We Do with It?'. Together they form a unique fingerprint.

Cite this