INFORMATION SYSTEMS EVALUATION CRITERIA BASED ON ATTITUDES OF GRADUATE STUDENTS
Keywords:
information systems, evaluation, multi-criteria analysis, Analytical Hierarchy ProcessAbstract
Importance of information systems in supporting business activities and managerial decision making is growing. Decisions related to selecting a suitable information system, including the technological background, human resources, procedures and information belong to one of the most difficult and most responsible ones. As in the case of other types of investments, assets and resources invested into information system should return in a reasonable time. There has been a lot of work done in the research and application of IS evaluation techniques to different kinds of information systems. Such evaluations involve a wide variety of technical and technological considerations made by technical experts, on the other hand impacts on management of the organization or financial impacts can be addressed. The objective of the paper is to reveal the preferences of graduate students related to their information systems evaluation and to propose a general framework for such evaluations. During the experimental period two surveys were carried out within the information systems course – at the beginning when the students were completely uninformed and at the end when the students had the knowledge of individual aspects of information systems, their role within organizations and process of information systems evaluation. The former survey used a simple scoring method whereas the latter relied on formal usage of the Analytical Hierarchy Process. The results show the differences in opinions of the students between these two surveys. Presented criteria hierarchy as well as the importance of individual evaluation criteria can be used for demonstration of attitudes of graduate students of management study programs and as a general framework for information systems evaluation.
References
Basak, I., Saaty, T. L. (1993) ‘Group decision making using the analytic hierarchy process’, Mathematical and Computer Modelling, vol. 17, pp. 101-109.
Boonstra, A. (2006) ‘Interpreting an ERP-implementation project from a stakeholder perspective’, International Journal of Project Management, vol. 24, no. 1, pp. 38-52.
Boonstra, A., de Vries, J. (2008) ‘Managing stakeholders around inter-organizational systems: A diagnostic approach’, The Journal of Strategic Information Systems, vol. 17, no. 3, pp. 190-201.
Borovits, I., Giladi, R. (1993) ‘Evaluating cost/utilization of organizations' information systems and users’, Information & Management, vol. 25, no. 5, pp. 273-280.
Chatzoglou, P.D., Diamantidis, A.D. (2009) ‘IT/IS implementation risks and their impact on firm performance’, International Journal of Information Management, vol. 29, no. 2, pp. 119-128.
Chen, D., Doumeingts, G., Vernadat, F. (2008) ‘Architectures for enterprise integration and interoperability: Past, present and future’, Computers in Industry, vol. 59, no. 7, pp. 647-659.
Díez, E., McIntosh, B.S. (2009) ‘A review of the factors which influence the use and usefulness of information systems’, Environmental Modelling & Software, vol. 24, no. 5, pp. 588-602.
Gremy, F., Fessler, J.M., Bonnin, M. (1999) ‘Information systems evaluation and subjectivity’, International Journal of Medical Informatics, vol. 56, no. 1-3, pp. 13-23.
Heo, J, Han, I. (2003) ‘Performance measure of information systems (IS) in evolving computing environments: an empirical investigation’, Information & Management, vol. 40, no. 4, pp. 243-256.
Homburg, V. (2008) Understanding E-Government: Information systems in public administration, Routledge, New York.
Irani, Z. (2002) ‘Information systems evaluation: navigating through the problem domain’, Information & Management, vol. 40, no. 1, pp. 11-24.
Irani, Z. Ghoneim, A., Love, P.E.D. (2006) ‘Evaluating cost taxonomies for information systems management’, European Journal of Operational Research, vol. 173, no. 3, pp. 1103-1122.
Lai, V.S., Wong, B.K., Cheung, W. (2002) ‘Group decision making in a multiple criteria environment: A case using the AHP in software selection’, European Journal of Operational Research, vol. 137, no. 1, pp. 134-144.
Lipovetsky, S. (1996) ‘The synthetic hierarchy method: An optimizing approach to obtaining priorities in the AHP’, European Journal of Operational Research, vol. 93, no. 3, pp. 550-564.
Ozdemir, M.S., Saaty, T.L. (2006) ‘The unknown in decision making: What to do about it’, European Journal of Operational Research, vol. 174, pp. 349–359.
Renkema, T.J.W., Berghout, E.W. (1997) ‘Methodologies for information systems investment evaluation at the proposal stage: a comparative review’, Information and Software Technology, vol. 39, no. 1, pp. 1-13.
Saaty, T.L. (1980) The Analytic Hierarchy Process, McGraw Hill, New York.
Saaty, T.L., Shih, H.S. (2009) ‘Structures in decision making: On the subjective geometry of hierarchies and networks’, European Journal of Operational Research, vol. 199, pp. 867–872.
Saaty, T.L., Vargas, L.G. (2007) ‘Dispersion of group judgments’, Mathematical and Computer Modelling, vol. 46, pp. 918–925.
Salmeron, J.L., Herrero, I. (2005) ‘An AHP-based methodology to rank critical success factors of executive information systems’, Computer Standards & Interfaces, vol. 28, pp. 1–12.
Thouin, M.F., Hoffman, J.J., Ford, E.W. (2009) ‘IT outsourcing and firm-level performance: A transaction cost perspective’, Information & Management, vol. 46, no. 8, pp. 463-469.
Vaidya, O.S., Kumar, S. (2006) ‘Analytic hierarchy process: An overview of applications’, European Journal of Operational Research, vol. 169, pp. 1–29.
Wang, Y.D., Forgionne, G. (2006) ‘A decision-theoretic approach to the evaluation of information retrieval systems’, Information Processing & Management, vol. 42, no. 4, pp. 863-874.
Wei, C.C., Chien, C.F., Wang, M.J.J. (2005) ‘An AHP-based approach to ERP system selection’, International Journal of Production Economics, vol. 96, pp. 47–62.
Wiseman, C. (1985) Strategy and Computers: Information Systems as Competitive Weapons, Dow Jones-Irwin, Homewood.
Yusof, M.M., Papazafeiropoulou, A., Paul, R.J., Stergioulas, L.K. (2008) ‘Investigating evaluation frameworks for health information systems’, International Journal of Medical Informatics, vol. 77, no. 6, pp. 377-385.
Yusof, M.M., Paul, R.J., Stergioulas, L. (2005) ‘Health Information Systems Evaluation: A Focus on Clinical Decision Supports System’, Connecting Medical Informatics and Bio-Informatics: Proceedings of MIE2005 – The XIXth International Congress of the European Federation for Medical Informatics, IOS Press, pp. 855-860.
ANSI/IEEE 1471-2000, Recommended Practice for Architecture Description of Software-Intensive Systems.
Additional Files
Published
How to Cite
Issue
Section
License
Authors declare with this manuscript intended for publication to ERIES Journal that:
- all co-authors agree with the publication of the manuscript even after amendments arising from peer review;
- all co-authors agree with the posting of the full text of this work on the web page of ERIES Journal and to the inclusion of references in databases accessible on the internet;
- no results of other researchers were used in the submitted manuscript without their consent, proper citation, or acknowledgement of their cooperation or material provided;
- the results (or any part of them) used in the manuscript have not been sent for publication to any other journal nor have they already been published (or if so, that the relevant works are cited in this manuscript);
- submission of the manuscript for publication was completed in accordance with the publishing regulations pertaining to place of work;
- experiments performed comply with current laws and written consent of the Scientific Ethics Committee / National Animal Care Authority (as is mentioned in the manuscript submitted);
- grant holders confirm that they have been informed of the submitted manuscript and they agree to its publication.
Authors retain copyright and grant ERIES Journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the published work with an acknowledgement of its initial publication in ERIES Journal. Moreover, authors are able to post the published work in an institutional repository with an acknowledgement of its initial publication in ERIES Journal. In addition, authors are permitted and encouraged to post the published work online (e.g. institutional repositories or on their website) as it can lead to productive exchanges, as well as earlier and greater citation of published work.
