Online searching: What measure satisfaction?

Librarians are assuming an increasingly prominent role in providing electronic information retrieval services. In light of this increased involvement, research that clarifies the relationship between information needs and satisfaction with these services takes on special significance. This article describes a research project which used quantitative measures to identify and examine potential indicators of satisfaction. In particular, the established concepts of relevance and precision were incorporated into the research design, but with a new perspective; relevance judgments were made by users after they had reviewed the search results. Precision measures were then calculated based upon these relevance judgments. Findings in the study revealed a low overall association between precision and satisfaction, regardless of whether the users' expectations were for exhaustive (high-recall) or specific (high-precision) results. Moreover, a point of diminishing returns was observed in satisfaction with more than 20 references. Such findings suggest that the established concepts for measuring retrieval effectiveness be reexamined in light of significant user involvement in the online search process.