The Future of Education

Collaborative problem solving (CPS) is a multidimensional skill that is essential for 21st-century education. Although CPS has gained significant use as a means of improving the quality of education in K-12, higher education, and informal learning settings, it has not been adequately assessed in chemistry. This study aims to develop and evaluate a standardized CPS assessment tool for chemistry (CPS-C) using a human-to-agent (H-A) method. In addition, the study aimed to investigate the influence of student variables, including interest, motivation, cognitive ability, and other factors, on CPS performance. The CPS-C tool consists of 60 multiple-choice items based on the three collaborative skills and four problem-solving skills in the PISA 2015 framework [1]. A preliminary qualitative study was conducted with 52 grade 11 students, followed by a main study with 292 students. Their raw scores were scaled using the MIRT model in R (version 4.2.1) [2], the Generalized Partial Credit Model (GPCM) [3] was used to rate item scores, and multiple linear regression analysis was used to examine the effects of variables on CPS-C performance. The results showed that the CPS-C instrument showed acceptable internal consistency, with EAP/PV reliability coefficients between 0.61 and 0.71 for each dimension and good item fit, all the items showed that the expected value of the infit was between 0.8 and 1.1. In addition, cognitive ability (B=0.722, p<0.001, VIF=1.112) and prior knowledge (B=1.636, p<0.001, VIF=1.159) had a significant impact on CPS performance. These findings suggest that the CPS-C instrument can be a valuable tool for assessing CPS skills in chemistry and identifying factors that predict CPS competence.