Toward automated definition acquisition from operations law

Definition acquisition is a necessary step in building an artificial cognitive assistant that helps military personnel to gain fast and precise understanding of the various terms and procedures defined in applicable legal documents. We approach the task of identifying definitional sentences from operations law documents by formalizing this task as a sentence-classification task and solving it by using machine-learning methods. This paper reports on a series of empirical experiments in that we evaluate and compare the performance of learning algorithms in terms of label-prediction accuracy. Using supervised techniques results in an F1 score of 95.93 and a 96.72 recall rate. However, for real-world applications, it would be too costly and unrealistic to ask personnel involved in military operations to label substantial amounts of data in order to build a new classifier for different types or genres of text data. Therefore, we propose and implement a semisupervised (SS) solution that trades off prediction accuracy to label efficiency. Our SS approach achieves a 90.47 F1 score and 93.44 recall rate by using only eight sentences labeled by a human expert.