A Counterfactual–Deterministic Adaptive Testing Framework: Mathematical Formulation and Simulation Evidence
Rusen Meylani, Dicle University (Turkey)
Abstract
Abstract
Computerized adaptive testing aims to shorten exams by selecting questions based on a student’s previous answers. In most existing systems, each new question is chosen to improve the precision of a single ability score. This approach assumes that all students differ only in overall ability and that they respond to questions in essentially the same way.
This study proposes a different perspective. Instead of assuming one common response pattern, we consider the possibility that students may approach questions in different ways. For example, some may respond cautiously, others may guess more often, and others may make systematic types of errors. These patterns are treated as competing explanations of how answers are produced—not as fixed personality types, but as plausible response behaviors.
The proposed adaptive method therefore selects questions not only to refine an ability estimate, but also to determine which response pattern best explains a student’s answers. As the test progresses, the system continuously updates two elements: (1) the student’s estimated ability level and (2) the likelihood that each response pattern explains their behavior. Questions are chosen because they are especially useful for distinguishing between these competing explanations, rather than simply increasing score precision.
The test ends when one of two conditions is met: either the student’s ability has been estimated with sufficient accuracy, or the system becomes confident about which response pattern best fits the student’s answers. Stopping decisions are therefore based both on measurement precision and on stability of interpretation.
To evaluate the approach, simulation studies were conducted using item pools of different sizes. Results show that the proposed method substantially reduces test length while maintaining strong agreement with full-length test scores. Correlations between adaptive and fixed-form scores exceeded 0.86 across conditions, and score differences were small relative to the reporting scale. Importantly, shorter tests did not advantage or disadvantage students at particular ability levels.
Overall, this framework redefines adaptive testing as a process of comparing competing explanations of student responses rather than only refining a single score. The method preserves score comparability with traditional fixed tests while offering a transparent and computationally efficient way to shorten assessments. It provides a practical alternative for educational contexts where fairness, reproducibility, and score stability are essential.
Keywords: adaptive testing, student assessment, response patterns, test length reduction, score comparability
The Future of Education




























