A Prototype Gutenberg-HathiTrust Sentence-level Parallel Corpus

Dataset

Description

This is a sentence-level parallel corpus in support of research on OCR quality. The source data comes from: (1) Project Gutenberg for human-proofread "clean" sentences; and, (2) HathiTrust Digital Library for the paired sentences with OCR errors. In total, this corpus contains 167,079 sentence pairs from 189 sampled books in four domains (i.e., agriculture, fiction, social science, world war history) published from 1793 to 1984. There are 36,337 sentences that have two OCR views paired with each clean version. In addition to sentence texts, this corpus also provides the location (i.e., sentence and chapter index) of each sentence in its belonging Gutenberg volume.
Date made availableJun 20 2022
PublisherUniversity of Illinois Urbana-Champaign

Keywords

  • HathiTrust Digital Library
  • digital humanities
  • sentence-level parallel corpus
  • optical character recognition
  • digital libraries
  • Project Gutenberg
  • OCR errors

Cite this