Large-Scale Parallel Cognitive Diagnostic Test Assembly Using A Dual-Stage Differential Evolution-Based Approach

Abstract

Parallel testing, which uses different test forms to assess examinees, is a necessary and important technique in both educational and psychometric assessments. A key but challenging problem for successful parallel testing lies in generating a high-quality parallel test set. Most existing parallel test assembly methods were developed for classic test theory and item response theory. In the context of cognitive diagnosis models, which is a new instrument featuring the ability to assess the examinee’s status on fine-grained attributes, the investigation of parallel test assembly is limited, particularly for large parallel scale. This study aims to provide an efficient dual-stage solution for the large-scale parallel cognitive diagnostic test (CDT) assembly problem. In the first stage, the assembly of individual CDTs is treated as a multimodal optimization problem and a niching differential evolution algorithm is developed to find an elite set of CDTs with near-optimal diagnostic performance. By redesigning evolutionary operators, the efficient search mechanism in differential evolution is transferred to the binary context and suits the purpose of optimizing item assignment to a CDT. In the second stage, a graph representation is defined to capture the set of elite CDTs and their overlapping relationships. A deterministic algorithm is applied to the graph to find specific nodal maximum cliques and provide two types of parallel test sets that satisfy different examiner preferences. Simulation studies under a variety of conditions and real-data demonstration show that the proposed method outperforms the existing approaches on large-scale instances while remaining competitive on small-scale cases. IEEE