Development of Computer-Based Chemical Five-Tier Diagnostic Test Instruments: A Generalized Partial Credit Model


  • Achmad Rante Suparman Universitas Papua
  • Eli Rohaeti Universitas Negeri Yogyakarta
  • Sri Wening Universitas Negeri Yogyakarta



Five-tier, chemical diagnostic, computer-based test, Generalized Partial Credit Model


This study focuses on developing a five-tier chemical diagnostic test based on a computer-based test with 11 assessment categories with an assessment score from 0 to 10. A total of 20 items produced were validated by education experts, material experts, measurement experts, and media experts and obtained an average index of the Aiken test > 0.70. The validation results were tested on 580 respondents and analyzed using the Generalized Partial Credit Model (GPCM) Item Response Theory (IRT) type. The results of the analysis show that all of the items meet the requirements to be said to be valid for the model; the evidence of the value this: RMSEA < 0.08, CFI > 0.87, SRMR < 0.10, GFI > 0.90, NFI > 0.90, NNFI > 0.90, IFI > 0.90, TLI > 0.90, and RFI > 0.90, and all items were obtained has a p.S_X2 value greater than 0.05 which indicates that all items developed are fit and by the GPCM model. The construct reliability (CR) value is 0.99, which suggests the construct is reliable. The most challenging item is item 9, and the most accessible item is item 4


Aiken, L. R. (1980) 'Content validity and reliability of single items or questionnaires', Educational and Psychological Measurement, Vol. 40, No. 4, pp. 955-959.

Anam, R. S., Widodo, A., Sopandi, W. and Wu, H. K. (2019) 'Developing a five-tier diagnostic test to identify students’ misconceptions in science: an example of the heat transfer concepts', Elementary Education Online, Vol. 18, No. 3, pp. 1014-1029.

Barke, H. D., Hazari, A. and Sileshi Yitbarek (2009) Misconception in Chemistry, Berlin: Springer.

Bayuni, T. C., Sopandi, W. and Sujana, A. (2018) 'Identification misconception of primary school teacher education students in changes of matters using a five-tier diagnostic test', Journal of Physics: Conference Series, Bandung, Vol. 1013, p. 012086

Chi, S., Wang, Z., Luo, M., Yang, Y. and Huang, M. (2018) 'Student progression on chemical symbol representation abilities at different grade levels (Grades 10-12) across gender', Chemistry Education Research and Practice, Vol. 19, No. 4, pp. 1055-1064.

Cornick, J. E. (2015) 'Factor structure of the exercise self-efficacy scale', Measurement in Physical Education and Exercise Science, Vol. 19, No. 4, pp. 208-218.

Crocker, L. and Algina, J. (2008) Introduction to Classical and Modern Test Theory, 2nd edition, Mason: Cengage Learning.

Dagnall, N., Denovan, A., Parker, A., Drinkwater, K. and Stephen Walsh, R. (2018) 'Confirmatory factor analysis of the inventory of personality organization-reality testing subscale', Frontiers in Psychology, Vol. 9, 1116, pp. 1-12.

Debelak, R., Stobl, C. and Zeigenfuse, M. D. (2022) An Introduction to the Rasch Model with Examples in R, New York: CRC Press.

Demars, C. (2010) Item Response Theory, New York: Oxford University Press.

Derkach, T. M. (2021) 'The origin of misconceptions in inorganic chemistry and their correction by computer modelling', Journal of Physics: Conference Series, Kryvyi Rih, Vol. 1840, pp. 1-13.

Dewanti, S. S., Hadi, S. and Nu’man, M. (2021) 'The application of item response theory in analysis of characteristics of mathematical literacy test items', İlköğretim Online, Vol. 20, No. 1, pp. 1226-1237.

Dewi, F. C., Parlan, P. and Suryadharma, I. B. (2020) 'Development of four-tier diagnostic test for identifying misconception in chemical equilibrium', AIP Conference Proceedings, Vol. 2215, pp. 1-6.

Drasgow, F. and Mattern, K. (2006) 'New tests and new items: Opportunities and issues', in Bartram, D. and Hambleton, R. K. (ed.) Computer-Based Testing and the Internet, Chichester: John Wiley & Sons Ltd.

Ghozali, I. and Fuad (2008) Structural equation modeling: teori, konsep, dan aplikasi dengan Program Lisrel 8.80, Semarang: Badan Penerbit Universitas Diponegoro.

Groen, M. M. van and Eggen, T. J. H. M. (2020) 'Educational Test Approaches: The Suitability of Computer-Based Test Types for Assessment and Evaluation in Formative and Summative Contexts', Journal of Applied Testing Technology, Vol. 21, No. 1, pp. 12-24.

Hambleton, R. K. (2006) 'Psychometric Models, Test Designs and Item Types for the Next Generation of Educational and Psychological Tests', in Bartram, D. and Hambleton, R. K. (ed.) Computer Based Testing and the Internet, Chichester: John Wiley & Sons Ltd.

Hambleton, R. K. and Swaminathan, H. (1985) Item Response Theory: Principles and Applications, New York: Springer Science+Business Media.

Hambleton, R. K., Swaminathan, H. and Rogers, H. J. (1991) Fundamentals of Item Response Theory Library, London: Sage Publications.

Harza, A. E. K. P., Wiji, W. and Mulyani, S. (2021) 'Potency to overcome misconceptions by using multiple representations on the concept of chemical equilibrium', Journal of Physics: Conference Series, Jawa Barat, Vol. 1806, pp. 1-6.

Igbaria, M., Zinatelli, N., Cragg, P. and Cavaye, A. L. M. (1997) 'Personal computing acceptance factors in small firms: A structural equation model', MIS Quarterly, Vol. 21, No. 3, pp. 279-305.

Istiyono, E., Dwandaru, W. S. B., Erfianti, L. and Astuti, W. (2020) 'Applying CBT in physics learning to measure students’ higher order thinking skills', Journal of Physics: Conference Series, Yogyakarta, Vol. 1440, p. 012061.

Istiyono, Edi, Dwandaru, W. S. B., Fenditasari, K., Ayub, M. R. S. S. N. and Saepuzaman, D. (2023) 'The development of a four-tier diagnostic test based on modern Test theory in physics education', European Journal of Educational Research, Vol. 12, No. 1, pp. 371-385.

Jusniar, J., Effendy, E., Budiasih, E. and Sutrisno, S. (2021) 'Eliminating misconceptions on reaction rate to enhance conceptual understanding of chemical equilibrium using EMBE-R strategy', International Journal of Instruction, Vol. 14, No. 1, pp. 85-104.

Kang, T. and Chen, T. T. (2011) 'Performance of the generalized S-X2 item fit index for the graded response model', Asia Pacific Education Review, Vol. 12, No. 1, pp. 89-96.

Kay, C. C., Yiin, H. K., Chu, C. K. and Hong, K. Y. (2010) 'Misconceptions in the teaching of chemistry in secondary schools in Singapore & Malaysia', Proceedings of the Sunway Academic Conference, Petaling Jaya, No. 1996, pp. 1-10.

Kimmons, R., Clark, B. and Lim, M. (2017) 'Understanding web activity patterns among teachers, students and teacher candidates', Journal of Computer Assisted Learning, Vol. 33, No. 6, pp. 588-596.

Kiray, S. A. and Simsek, S. (2020) 'Determination and evaluation of the science teacher candidates’ misconceptions about density by using four-tier diagnostic test', International Journal of Science and Mathematics Education, Vol. 19, No. 5, pp. 935-955.

Kwahk, K. Y. and Lee, J. N. (2008) 'The role of readiness for change in ERP implementation: Theoretical bases and empirical validation', Information and Management, Vol. 45, No. 7, pp. 474-481.

Lamichhane, R., Reck, C. and Maltese, A. V. (2018) 'Undergraduate chemistry students’ misconceptions about reaction coordinate diagrams', Chemistry Education Research and Practice, Vol. 19, No. 3, pp. 834-845.

Linden, W. J. van der (2018) Handbook of Item Response Theory Volume Three: Applications, Boca Raton: CRC Press.

Linden, W. J. and Hambleton, R. K. (1997) Handbook of Modern Item Response Theory, New York: Springer.

Llanos, J., Fernández-Marchante, C. M., García-Vargas, J. M., Lacasa, E., De La Osa, A. R., Sanchez-Silva, M. L., De Lucas-Concuegra, A., Garcia, M. T. and Borreguero, A. M. (2021) 'Game-based learning and just-in-time teaching to address misconceptions and improve safety and learning in laboratory activities', Journal of Chemical Education, Vol. 98, No. 10, pp. 3118-3130.

Lowyck, J. (2014) Handbook of Research on Educational Communications and Technology New York: Springer.

Marsh, H. W., Balla, J. R. and Mcdonald, R. P. (1988) 'Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size', Psychological Bulletin, Vol. 103, No. 3, pp. 391-410.

Martin, F. and Betrus, A. K. K. (2019) Digital media for learning: Theories, processes, and solutions digital media for learning: Theories, processes, and solutions, Cham: Springer Nature Switzerland.

Merriënboer, J. J. G. van and Bruin, A. B. H. de (2014) 'Research Paradigms and Perspectives on Learning', in Spector, J. M., Merrill, M. D., Elen, J. and Bishop M. J. (ed.) Handbook of Research on Educational Communications and Technology, New York: Springer.

Mills, C. n. and Breithaupt, K. J. (2016) Educational Measurement From Foundations to Future, New York: The Guilford Press.

Millsap, R. E. and Kwok, O. M. (2004) 'Evaluating the impact of partial factorial invariance on selection in two populations', Psychological Methods, Vol. 9, No. 1, pp. 93-115.

Muraki, E. (1992) 'A generalized partial credit model: Application of an EM algorithm', Applied Psychological Measurement, Vol. 16, No. 2, pp. 159-176.

Murniningsih, Muna, K. and Irawati, R. K. (2020) 'Analysis of misconceptions by four tier tests in electrochemistry, case study on students of the chemistry education study program UIN Antasari Banjarmasin', Journal of Physics: Conference Series, Vol. 1440, pp. 1-6.

Mutlu, A. and Sesen, B. A. (2015) 'Development of a two-tier diagnostic test to assess undergraduates’ understanding of some chemistry concepts', Procedia - Social and Behavioral Sciences, Vol. 174, pp. 629-635.

Nicoll, G. (2001) 'A report of undergraduates’ bonding misconceptions', International Journal of Science Education, Vol. 23, No. 7, pp. 707-730.

Oriondo, L. L. and Antonio, E. M. D. (1984) Evaluating Educational Outcomes (Tests, Measurement and Evaluation), Manila: Rex Book Store.

Özmen, H. (2004) 'Some student misconceptions in chemistry: A literature review of chemical bonding', Journal of Science Education and Technology, Vol. 13, No. 2, pp. 147-159.

Pokorný, M. (2023) 'Experience with online learning of mathematics in primary education', International Journal of Emerging Technologies in Learning (IJET), Vol. 18, No. 02, pp. 203-213.

Potvin, P. (2023) 'Response of science learners to contradicting information: A review of research', Studies in Science Education, Vol. 59, No. 1, pp. 67-108.

Prodjosantoso, A. K., Hertina, A. M. and Irwanto (2019) 'The misconception diagnosis on ionic and covalent bonds concepts with three tier diagnostic test', International Journal of Instruction, Vol. 12, No. 1, pp. 1477-1488.

Putra, A. S. U., Hamidah, I. and Nahadi (2020) 'The development of five-tier diagnostic test to identify misconceptions and causes of students’ misconceptions in waves and optics materials', Journal of Physics: Conference Series, Vol. 1521, No. 2.

Rupp, A. A. and Zumbo, B. D. (2006) 'Understanding parameter invariance in unidimensional IRT models', Educational and Psychological Measurement, Vol. 66, No. 1, pp. 63-84.

Rusmini, Suyono, Jatmiko, B. and Yonata, B. (2021) 'The diagnosis of misconception on the concept of acid-base theory in prospective teacher students used a three-tier test', Journal of Physics: Conference Series, Vol. 1899, No. 1, pp. 1-7.

Suparman, A. R., Rohaeti, E. and Wening, S. (2022) 'Development of attitude assessment instruments towards socioscientific issues in chemistry learning', European Journal of Educational Research, Vol. 11, No. 4, pp. 1947-1958.

Suparman, A. R., Rohaeti, E. and Wening, S. (2023) 'Effect of computer based test on motivation: A meta-analysis', European Journal of Educational Research, Vol. 12, No. 4, pp. 1583-1599.

Tie, B., Chen, G. and He, J. (2022) 'Validation of the inflexible eating questionnaire in a large sample of Chinese adolescents: psychometric properties and gender-related differential item functioning', Eating and Weight Disorders, Vol. 27, No. 3, pp. 1029-1037.

Tien, L. T. and Osman, K. (2017) Overcoming Students’ Misconceptions in Science, Singapore: Springer Nature.

Treagust, D., Chittleborough, G. and Mamiala, T. (2003) 'The role of submicroscopic and symbolic representations in chemical explanations', International Journal of Science Education, Vol. 25, No. 11, pp. 1353-1368.

Üce, M. and Ceyhan, İ. (2019) 'Misconception in chemistry education and practices to eliminate them: Literature analysis', Journal of Education and Training Studies, Vol. 7, No. 3, pp. 202.

Vladusic, R., Bucat, R. B. and Ozic, M. (2022) 'Understanding covalent bonding - a scan across the Croatian education system', Chemistry Education Research and Practice, Vol. 23, No. 4.

Wang, F., Franco-Penya, H.-H., Kelleher, J. D., Pugh, J. and Ross, R. (2017) 'An analysis of the application of simplified silhouette to the evaluation of k-means clustering validity', International Conference Machine Learning and Data Mining in Pattern Recognition, Vol. 13, pp. 291-305.

Wang, Z., Yan, W., Zeng, C., Tian, Y. and Dong, S. (2023) 'A unified interpretable intelligent learning diagnosis framework for learning performance prediction in intelligent tutoring systems', International Journal of Intelligent Systems, Vol. 2023, pp. 1-20.

Widarti, H. R., Permanasari, A., Mulyani, S., Rokhim, D. A. and Habiddin (2021) 'Multiple representation-based learning through cognitive dissonance strategy to reduce student’s misconceptions in volumetric analysis', TEM Journal, Vol. 10, No. 3, pp. 1263-1273.

Additional Files



How to Cite

Suparman, A. R., Rohaeti, E. and Wening, S. (2024) ’Development of Computer-Based Chemical Five-Tier Diagnostic Test Instruments: A Generalized Partial Credit Model’, Journal on Efficiency and Responsibility in Education and Science, vol. 17, no. 1, pp. 92–106.



Research Paper