New Perspectives in Science Education

The purpose of this study is to evaluate the psychometric quality of chemistry test items of the Ethiopian University Entrance Examination (2020). Analysis was carried out through traditional and modern test theories including Classical Test Theory, Factor Analysis, Generalizability theory, and Item Response Theory. Sample selection was census-based in which all the scores of 45,081 examinees who took the test were taken from the archival data obtained from the National Educational Assessment and Examinations Agency. The test consisted of 80 multiple-choice items. The findings showed that test reliability was found to be 0.876. 10 items were found with marginal discrimination index, and 12 items were poor discriminators. Significant mean differences were also found in the four test forms and the mean of test form 04 was the highest. 17 principal factors were explored through the exploratory factor analysis. Consequently, only 58 items were identified with satisfactory factor loadings. The 3pl IRT model analysis showed that 6 items were found with higher guessing estimate coefficients. From the findings, it was recommended that items with low discrimination and very high difficulty need to be revised before they are stored in the item pool for future use. The performance mean differences among test forms call test developers to use equivalent test forms rather than using forms with the same and scrambled test items

Keywords: Item analysis, psychometrics, achievement test, test analysis, reliability, validity

REFERENCES

Aryadoust, V., & Raquel, M. (Eds.). (2019). Quantitative Data Analysis for Language Assessment Volume I

2. Bichi, A. A. (2013). Item Analysis using a Derived Science Achievement Test Data. 4(5), 8.

3. Crocker, L. M., & Algina, J. (2008). Introduction to classical and modern test theory. Cengage Learning.

4. Eleje, L. I., & Esomonu, N. P. M. (2018). Test of Achievement in Quantitative Economics for Secondary     

   Schools: Construction and Validation Using Item Response Theory.

5. Finch, W. H. (n.d.). Applied Psychometrics Using SPSS and AMOS. 288.

6. Gagnon, R. J. (2019). Measurement Theory and Applications for the Social Sciences. Measurement: Interdisciplinary Research and Perspectives, 17(4), 209–210. https://doi.org/10.1080/15366367.2019.1610343

7. Reynolds, C. R., Altmann, R. A., & Allen, D. N. (2021a). Mastering Modern Psychological Testing: Theory and Methods. Springer International Publishing. https://doi.org/10.1007/978-3-030-59455-8

8.  Shultz, K. S., Whitney, D. J., & Zickar, M. J. (2020). Measurement Theory in Action: Case Studies and Exercises (3rd ed.). Routledge. https://doi.org/10.4324/9781003127536