Quality Control of 19th Century Weather Data

Nancy Westcott

Research output: Book/Report/Conference proceedingTechnical report

Abstract

The Climate Database Modernization Program's (CDMP) Forts and Volunteer Observer Database Project has resulted in a dramatic increase in the number of U.S. daily cooperative network observations available prior to 1893. Currently, data from 395 stations have been captured from the original scanned images. The stations are primarily located east of the Mississippi River, but coverage extends to all 48 contiguous U.S. states and Alaska. A rigorous quality control process is used to ensure that the keyed data matches the original form. This process involves careful collection of the metadata from the form, double-keying of the data, and a series of automated quality control tests. Values flagged by these tests are typically verified manually and corrections are applied as needed, although in some cases errors are automatically corrected. An analysis of the quality control process for 40 stations shows that on average, about 31 percent of the flags verify the information, 52 percent can be corrected, and 17 percent are deemed uncorrectable. The correctable errors typically result from unclear forms, mis-keyed data, and errors in the metadata for the image. Due to changes in observation practices since the nineteenth century, care must be taken in using the data for analysis. Despite these caveats, the nineteenth century weather dataset is being used in an increasing number of climate studies.
Original languageEnglish (US)
StatePublished - 2011

Publication series

NameISWS Contract Report 2011-02
No.CR-2011-02

Keywords

  • ISWS

Fingerprint Dive into the research topics of 'Quality Control of 19th Century Weather Data'. Together they form a unique fingerprint.

Cite this