Overview of Test Assembly Methods in Multistage Testing

In multistage testing (MST), examinees receive different sets of items from preassembled tests that are matched to provisional estimates of their ability levels. Though it has many potential benefits, MST generates new challenges for test assembly because of the large number of possible paths through the test: a well-designed MST must (1) have distinct information curves between modules in each stage to sufficiently differentiate pathways through the test, (2) have sufficiently parallel forms for all pathways in parallel panels, and (3) meet all nonstatistical design constraints (such as content balancing and enemy items) across many different possible pathways. This becomes highly demanding, especially when the item bank is limited. Although automated test assembly (ATA) algorithms can reduce much of the burden on test developers, these algorithms must be adapted to the increased complexity of MST design. This chapter first discusses how the approach to MST assembly differs from assembling linear tests, followed by an overview of current ATA methods for MST. Then, we will present a new paradigm for MST assembly called assembly-on-the-fly, which borrows well-established item selection algorithms in computerized adaptive testing (CAT) to construct individualized modules for each examinee dynamically (also see Han and Guo, this volume, for related methods). Finally, we mention several possible directions for future development in MST assembly.