A NEW APPROACH TO THE ACHIEVEMENT TEST ITEMS EVALUATION: THE CORRECTNESS COEFFICIENT

Authors

  • Tomáš Foltýnek Mendel University in Brno, Faculty of Business and Economics, Department of Informatics

Keywords:

evaluation methodologies, intelligent tutoring systems, media in education

Abstract

This paper deals with the problem set of an achievement test item scoring. The scoring process is generalized with the help of correctness coefficient – the new concept set up by the author. The paper describes complexly formalization of the scoring process, contextualizes the contemporary used methods to the general context and brings new methods as well. The scoring methods of sorting items and guessing penalty are described in detail. Observations described in this paper can help examiners with more accurate assessment of achievement test results. In the first part, the theoretical basics of the test item scoring are given. We are going to find out that whole scoring process depends on the teaching objectives, test item types, curriculum taxonomy and achievement test objectives. Then the theory of the test item types is described. After this theoretical introduction the concepts of the total achievement test score and correctness coefficient are set up. Let’s emphasize that using of the correctness coefficient is the new contribution of the author. Than the correctness coefficient is used to express the measure of examinee’s answer accuracy within the different test item types. Using the correctness coefficient for evaluation of closed multiple-choice items, injective and general relational items, narrow open items, joining items and sorting items are deeply examined and described. Various scoring method for these item types are discussed, especially for the sorting and joining items. Afterwards the theory of penalty guessing is expressed with the help of the correctness co efficient, which strengthens the ability and universality of theory being described. The main goal of this research paper is to provide the complex theoretical overview of the test item scoring problems, which can be useful for pedagogues, examiners and testing application (or e-learning system) developers to provide more accurate and clear evaluation process of the achievement test.

 

References

Bloom, B. S. (ed.) (1956) Taxonomy of Educational Objectives. Vol. 1: Cognitive Domain. New York: McKay, 1956.

Booth, R.; Clayton, B.; Hartcher, R.; Hungar, S.; Hyde, P.; Wilson, P. (2003) The development of quality online assessment in vocational education and training. Leabrrook: Australian National Training Authority.

Brusilovsky, P. (1999) ‘Adaptive and intelligent technologies for web-based education’. In C. Rollinger, & C. Peylo (Eds.). Intelligent systems and teleteaching [Special issue]. Künstliche Intelligenz, vol. 4, pp. 19–25. 

Davis, B.G. (1993) Tools for Teaching. San Francisco: Jossey-Bass.

Foltýnek, T. (2005) The electronic test results as a feedback for teachers. In Trends in e-learning – Belcom’05 proceedings of papers. Prague: CVUT.

Foltýnek, T.; Motyčka, A. (2006) On the achievement test items scoring. In Efficiency and responsibility in education – proceedings of papers 2006. Prague: ČZU.

Hobgood, B.; Thibault, M.; Walbert, D. (2004) Kinetic connections: Bloom’s taxonomy in action, [online], Available: http://www.learnnc.org/articles/bloom0405-1 [31 Aug 2006].

Linrace, J.M.; Wright, B.D. (1995) How to Assign Item Weights. In Rasch Measurement Transaction, Vol. 8 no. 4.

Mužić, V. (1993) How to Outwit a Test. Zagreb: Školske novine.

Payne, D.A. (1968) The specification and measurement of learning outcomes. Waltham: Blaisdell Publishning Company.

Segall, N.; Doolen, T.L.; Porter J.D. (2005) ‘A usability comparison of PDA-based quizzes and paper-and-pencil quizzes’, Computers & Education, vol. 45, no. 4, pp. 417 – 432. 

Stalker, M.J. (1968) ‘The Penalty for Not Guessing’, Journal of Educational Measurement, Vol. 5, No. 2 , pp. 141-144. 

Additional Files

Published

2009-06-30

How to Cite

Foltýnek, T. (2009) ’A NEW APPROACH TO THE ACHIEVEMENT TEST ITEMS EVALUATION: THE CORRECTNESS COEFFICIENT’, Journal on Efficiency and Responsibility in Education and Science, vol. 2, no. 1, pp. 28-40 Retrieved from https://www.eriesjournal.com/index.php/eries/article/view/13.

Issue

Section

Research Paper