Semantic indexing for a complete subject discipline

Yi Ming Chung, Qin He, Kevin Powell, Bruce Schatz

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

As part of the Illinois Digital Library Initiative (DLI) project we developed `scalable semantics' technologies. These statistical techniques enabled us to index large collections for deeper search than word matching. Through the auspices of the DARPA Information Management program, we are developing an integrated analysis environment, the Interspace Prototype, that uses `semantic indexing' as the foundation for supporting concept navigation. These semantic indexes record the contextual correlation of noun phrases, and are computed generically, independent of subject domain. Using this technology, we were able to compute semantic indexes for a subject discipline. In particular, in the summer of 1998, we computed concept spaces for 9.3 M MEDLINE bibliographic records from the National Library of Medicine (NLM) which extensively covered the biomedical literature for the period from 1966 to 1997. In this experiment, we first partitioned the collection into smaller collections (repositories) by subject, extracted noun phrases from titles and abstracts, then performed semantic indexing on these sub-collections by creating a concept space for each repository. The computation required 2 days on a 128-node SGI/CRAY Origin 2000 at the National Center for Supercomputer Applications (NCSA). This experiment demonstrated the feasibility of scalable semantics techniques for large collections. With the rapid increase in computing power, we believe this indexing technology will shortly be feasible on personal computers.

Original languageEnglish (US)
Title of host publicationProceedings of the ACM International Conference on Digital Libraries
PublisherACM
Pages39-48
Number of pages10
ISBN (Print)1581131453, 9781581131451
DOIs
StatePublished - 1999
EventProceedings of the 1999 4th ACM International Conference on Digital Libraries (DL'99) - Berkeley, CA, USA
Duration: Aug 11 1999Aug 14 1999

Publication series

NameProceedings of the ACM International Conference on Digital Libraries

Other

OtherProceedings of the 1999 4th ACM International Conference on Digital Libraries (DL'99)
CityBerkeley, CA, USA
Period8/11/998/14/99

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Fingerprint Dive into the research topics of 'Semantic indexing for a complete subject discipline'. Together they form a unique fingerprint.

Cite this