Evaluation of interactive knowledge-based systems: Overview and design for empirical testing

An overview of levels and approaches in the evaluation of knowledge-based systems is presented. There is a need for empirical studies using objective criteria in advance of completing the technical evaluation of such systems. A methodology for this type of evaluation developed for a particular knowledge-based indexing system is presented. It is suggested that the proposed study may serve as a model for the design of any evaluation in which the results of existing intellectual procedures are compared with results achieved when these procedures are aided by use of an appropriate expert system.