• František Dařena Mendel University in Brno


information systems, evaluation, multi-criteria analysis, Analytical Hierarchy Process


Importance of information systems in supporting business activities and managerial decision making is growing. Decisions related to selecting a suitable information system, including the technological background, human resources, procedures and information belong to one of the most difficult and most responsible ones. As in the case of other types of investments, assets and resources invested into information system should return in a reasonable time. There has been a lot of work done in the research and application of IS evaluation techniques to different kinds of information systems. Such evaluations involve a wide variety of technical and technological considerations made by technical experts, on the other hand impacts on management of the organization or financial impacts can be addressed. The objective of the paper is to reveal the preferences of graduate students related to their information systems evaluation and to propose a general framework for such evaluations. During the experimental period two surveys were carried out within the information systems course – at the beginning when the students were completely uninformed and at the end when the students had the knowledge of individual aspects of information systems, their role within organizations and process of information systems evaluation. The former survey used a simple scoring method whereas the latter relied on formal usage of the Analytical Hierarchy Process. The results show the differences in opinions of the students between these two surveys. Presented criteria hierarchy as well as the importance of individual evaluation criteria can be used for demonstration of attitudes of graduate students of management study programs and as a general framework for information systems evaluation.


Basak, I., Saaty, T. L. (1993) ‘Group decision making using the analytic hierarchy process’, Mathematical and Computer Modelling, vol. 17, pp. 101-109.

Boonstra, A. (2006) ‘Interpreting an ERP-implementation project from a stakeholder perspective’, International Journal of Project Management, vol. 24, no. 1, pp. 38-52.

Boonstra, A., de Vries, J. (2008) ‘Managing stakeholders around inter-organizational systems: A diagnostic approach’, The Journal of Strategic Information Systems, vol. 17, no. 3, pp. 190-201.

Borovits, I., Giladi, R. (1993) ‘Evaluating cost/utilization of organizations' information systems and users’, Information & Management, vol. 25, no. 5, pp. 273-280.

Chatzoglou, P.D., Diamantidis, A.D. (2009) ‘IT/IS implementation risks and their impact on firm performance’, International Journal of Information Management, vol. 29, no. 2, pp. 119-128.

Chen, D., Doumeingts, G., Vernadat, F. (2008) ‘Architectures for enterprise integration and interoperability: Past, present and future’, Computers in Industry, vol. 59, no. 7, pp. 647-659.

Díez, E., McIntosh, B.S. (2009) ‘A review of the factors which influence the use and usefulness of information systems’, Environmental Modelling & Software, vol. 24, no. 5, pp. 588-602.

Gremy, F., Fessler, J.M., Bonnin, M. (1999) ‘Information systems evaluation and subjectivity’, International Journal of Medical Informatics, vol. 56, no. 1-3, pp. 13-23.

Heo, J, Han, I. (2003) ‘Performance measure of information systems (IS) in evolving computing environments: an empirical investigation’, Information & Management, vol. 40, no. 4, pp. 243-256.

Homburg, V. (2008) Understanding E-Government: Information systems in public administration, Routledge, New York.

Irani, Z. (2002) ‘Information systems evaluation: navigating through the problem domain’, Information & Management, vol. 40, no. 1, pp. 11-24.

Irani, Z. Ghoneim, A., Love, P.E.D. (2006) ‘Evaluating cost taxonomies for information systems management’, European Journal of Operational Research, vol. 173, no. 3, pp. 1103-1122.

Lai, V.S., Wong, B.K., Cheung, W. (2002) ‘Group decision making in a multiple criteria environment: A case using the AHP in software selection’, European Journal of Operational Research, vol. 137, no. 1, pp. 134-144.

Lipovetsky, S. (1996) ‘The synthetic hierarchy method: An optimizing approach to obtaining priorities in the AHP’, European Journal of Operational Research, vol. 93, no. 3, pp. 550-564.

Ozdemir, M.S., Saaty, T.L. (2006) ‘The unknown in decision making: What to do about it’, European Journal of Operational Research, vol. 174, pp. 349–359.

Renkema, T.J.W., Berghout, E.W. (1997) ‘Methodologies for information systems investment evaluation at the proposal stage: a comparative review’, Information and Software Technology, vol. 39, no. 1, pp. 1-13.

Saaty, T.L. (1980) The Analytic Hierarchy Process, McGraw Hill, New York.

Saaty, T.L., Shih, H.S. (2009) ‘Structures in decision making: On the subjective geometry of hierarchies and networks’, European Journal of Operational Research, vol. 199, pp. 867–872.

Saaty, T.L., Vargas, L.G. (2007) ‘Dispersion of group judgments’, Mathematical and Computer Modelling, vol. 46, pp. 918–925.

Salmeron, J.L.,  Herrero, I. (2005) ‘An AHP-based methodology to rank critical success factors of executive information systems’, Computer Standards & Interfaces, vol. 28, pp. 1–12.

Thouin, M.F., Hoffman, J.J., Ford, E.W. (2009) ‘IT outsourcing and firm-level performance: A transaction cost perspective’, Information & Management, vol. 46, no. 8, pp. 463-469.

Vaidya, O.S., Kumar, S. (2006) ‘Analytic hierarchy process: An overview of applications’, European Journal of Operational Research, vol. 169, pp. 1–29.

Wang, Y.D., Forgionne, G. (2006) ‘A decision-theoretic approach to the evaluation of information retrieval systems’, Information Processing & Management, vol. 42, no. 4, pp. 863-874.

Wei, C.C., Chien, C.F., Wang, M.J.J. (2005) ‘An AHP-based approach to ERP system selection’, International Journal of Production Economics, vol. 96, pp. 47–62.

Wiseman, C. (1985) Strategy and Computers: Information Systems as Competitive Weapons, Dow Jones-Irwin, Homewood. 

Yusof, M.M., Papazafeiropoulou, A., Paul, R.J., Stergioulas, L.K. (2008) ‘Investigating evaluation frameworks for health information systems’, International Journal of Medical Informatics, vol. 77, no. 6, pp. 377-385.

Yusof, M.M., Paul, R.J., Stergioulas, L. (2005) ‘Health Information Systems Evaluation: A Focus on Clinical Decision Supports System’, Connecting Medical Informatics and Bio-Informatics: Proceedings of MIE2005 – The XIXth International Congress of the European Federation for Medical Informatics, IOS Press, pp. 855-860. 

ANSI/IEEE 1471-2000, Recommended Practice for Architecture Description of Software-Intensive Systems.

Additional Files



How to Cite

Dařena, F. (2011) ’INFORMATION SYSTEMS EVALUATION CRITERIA BASED ON ATTITUDES OF GRADUATE STUDENTS’, Journal on Efficiency and Responsibility in Education and Science, vol. 4, no. 1, pp. 46–56 Retrieved from



Research Paper