A comparative analysis of end‐user training methods
Tóm tắt
The aim of this study is to evaluate effectiveness of several end‐user training methods during the deployment of a new information system in an Israeli financial institution.
Four training methods, namely, frontal (lectures), simulator (e‐learning), mentor (personal guidance), and hybrid (mix of the three aforementioned methods) were all compared in terms of the number of referrals to an online helpdesk following training sessions.
A pair‐wise analysis to pinpoint the source of the differences between the various training methods indicates that all training methods are significantly different except for frontal and simulator and therefore an organization considering these two methods can select them indifferently. A combination of mentor and simulator was found to be significantly inferior to the frontal and simulator hybrid method.
The main limitation of the study is that evaluation was based on data associated with a specific organization, and thus external validity to other industries or countries is somewhat limited. Among other limitations are cross‐organizational differences in terms of: difficulty of calls, individual experience in the IS/IT area, and prior exposure to similar technologies.
Although this research does not recommend a specific training method, our findings suggest that deploying a new software system is a matter that requires early assessment by the organization and that the best results are obtained by employing the hybrid training approach.
The use of the number of referrals to an online helpdesk is a novel metric for measuring training effectiveness. The study also investigates the decline of the number of referrals over time.
Từ khóa
Tài liệu tham khảo
Ahituv, N., Neumann, S. and Riley, H.M. (1994), Principles of Information Systems for Management, 4th ed., Business and Educational Technologies, Dubuque, IA.
Ahituv, N., Neumann, S. and Zviran, Z. (2002), “A system development methodology for ERP systems”, The Journal of Computer Information Systems, Vol. 42 No. 3, pp. 56‐67.
Ausubel, D.P. (1968), Educational Psychology: A Cognitive View, Holt, Rinehart & Winston, New York, NY.
Ballantine, J., Bonner, M., Levy, M., Martin, A., Munro, I. and Powell, P.L. (1988), “Developing a 3‐D model of information systems success”, in Garrity, E.J. and Sanders, G.L. (Eds), Information Systems Success Measurement, Idea Group Information Technology Management Series, IGI Publishing, Hershey, PA, pp. 46‐59.
Bandura, A. and Locke, E.A. (2003), “Negative self‐efficacy and goal effects revisited”, Journal of Applied Psychology, Vol. 88 No. 1, pp. 87‐99.
Bostrom, R.P., Olfman, L. and Sein, M.K. (1988), “End user computing a framework to investigate the training/learning process”, in Carey, J.M. (Ed.), Human Factors in Management Information Systems, Ablex Publishing Corportion, Norwood, NJ, pp. 221‐50.
Bostrom, R.P., Olfman, L. and Sein, M.K. (1990), “The importance of learning style in end‐user training”, MIS Quarterly, Vol. 14 No. 1, pp. 101‐19.
Butler Cox Foundation (1986), CSC Index, Bloomsbury Square, Butler Cox Foundation, London.
Casey, M.S. and Doverspike, D. (2005), “Training needs analysis and evaluation for new technologies through the use of problem‐based inquiry”, Performance Improvement Quarterly, Vol. 18 No. 1, pp. 110‐24.
Cheney, P.H., Mann, R.I. and Amoroso, D.L. (1986), “Organizational factors affecting the success of end‐user computing”, Journal of MIS, Vol. 3 No. 1, pp. 65‐80.
Chivers, G. (2006), “The training of university lifelong learning professionals as researchers”, Journal of European Industrial Training, Vol. 30 No. 5, pp. 330‐48.
Coe, L.R. (1996), “Five small secrets to systems success”, Information Resources Management Journal, Vol. 9 No. 4, pp. 29‐38.
Davis, S.A. and Bostrom, R.P. (1993), “Training end users: an experimental investigation of the roles of the computer interface and training methods”, MIS Quarterly, Vol. 17 No. 1, p. 61.
DeLone, W.H. (1988), “Determinants of success for computer usage in small business”, MIS Quarterly, Vol. 12 No. 1, pp. 51‐61.
Devaraj, S. and Kohli, R. (2003), “Performance impacts of information technology: is actual usage the missing link?”, Management Science, Vol. 49 No. 3, pp. 273‐89.
Ein‐Dor, P. and Segev, E. (1991), “Intensity of end user computing”, SIGMIS Database, Vol. 22 Nos 1/2, pp. 30‐7.
Facteau, J.D., Dobbins, G.H., Russell, J.E.A., Ladd, R.T. and Kudisch, J.D. (1995), “The influence of general perceptions of the training environment on pre‐training motivation and perceived training transfer”, Journal of Management, Vol. 21, pp. 1‐25.
Gackowski, Z. (2003), “Case/real‐life problem‐based learning with information system projects”, Journal of Information Technology Education, Vol. 2, pp. 358‐65.
Galloway, R.L. and Whyte, G.A. (1989), “The information systems function as a service operation”, International Journal of Operations & Production Management, Vol. 9 No. 4.
Glaser, R. and Bassock, M. (1989), “Learning theory and the study on instruction”, Annual Review of Psychology, Vol. 40, pp. 631‐66.
Grover, V. and Teng, T.C. (1994), “Facilitating the implementation of customer based inter‐organizational systems: an empirical analysis of innovation and support factors”, Information Systems Journal, Vol. 4, pp. 61‐89.
Gupta, S. and Bostrom, R.P. (2006), “End‐user training methods: what we know, need to know”, Proceedings of the 2006 ACM SIGMIS CPR Conference on Computer Personnel Research: Forty Four Years of Computer Personnel Research: Achievements, Challenges & the Future (Claremont, CA, USA, April 13‐15, 2006), SIGMIS CPR '06. ACM, New York, NY, pp. 172‐82.
Gwebu, K.L. and Wang, J. (2007), “The role of organizational, environmental and human factors in e‐learning diffusion”, International Journal of Web‐Based Learning and Teaching Technologies, Vol. 2 No. 2, pp. 59‐78.
Harackiewicz, J.M., Durik, A.M. and Barron, K.E. (2004), “Multiple goals, optimal motivation, and the development of interest”, in Forgas, J.P., Williams, K.D. and Laham, S.M. (Eds), Social Motivation: Conscious and Unconscious Processes, Cambridge University Press, Cambridge.
Harackiewicz, J.M., Barron, K.E., Tauer, J.M. and Elliot, A.J. (2002), “Predicting success in college: a longitudinal study of achievement goals and ability measures as predictors of interest and performance from freshman year through graduation”, Journal of Educational Psychology, Vol. 94 No. 3, pp. 562‐75.
Hsu, K. and Turoff, M. (2007), “Focusing computer training to user needs: the impacts of online and offline training on learning”, International Journal of Innovation and Learning, Vol. 4 No. 5, pp. 530‐46.
Karten, N. (1987), “Managing end user computing when the only constant is change”, Journal of Systems Management, Vol. 38 No. 10, p. 26.
Kirkpatrick, D.L. (1998), Evaluating Training Programs: The Four Levels, 2nd ed., Berrett‐Koehler, San Francisco, CA.
Kirkpatrick, J. (2007), “The hidden power of Kirkpatrick's four levels”, T+D, Vol. 61 No. 8, pp. 34‐7.
Klein, K.J. and Ralls, R.S. (1995), “The organizational dynamic of computerized technology implementation: a review of the empirical literature”, Advances in Global High Technology Management, Vol. 5, pp. 31‐79.
Kraiger, K., Ford, J.K. and Salas, E. (1993), “Application of cognitive, skill‐based, and affective theories of learning outcomes to new methods of training evaluation”, Journal of Applied Psychology Monograph, Vol. 78 No. 2, pp. 311‐32.
Laoledchai, Y., Pek Wee Land, L. and Low, G. (2008), “Improving the effectiveness of end‐user training outcomes”, paper presented at the 19th Australasian Conference on Information Systems, Christchurch, 3‐5 December.
Lyytinen, K. (1988), “Expectation failure concept and systems analysts' view of information system failures: results of an exploratory study”, North‐Holland Information & Management, Vol. 14, pp. 45‐56.
Mahapatra, R. and Lai, V.S. (2005), “Evaluating end‐user training programs”, Communication of the ACM, Vol. 48 No. 1, pp. 67‐70.
Miller, J. and Doyle, B.A. (1987), “Measuring effectiveness of computer‐based information systems in the financial services sector”, MIS Quarterly, Vol. 11 No. 1, pp. 107‐24.
Nelson, R.R. and Cheney, P.H. (1987), “Training end users: an exploratory study”, MIS Quarterly, Vol. 11 No. 4, p. 547.
Phillips, J.J. and Phillips, P.P. (2000), “The return on investment process: issues and trends”, Training Journal, Vol. 71 No. 1, pp. 8‐12.
Rivard, S. and Huff, S.L. (1988), “Factors of success for end‐user computing”, Communications of the ACM, Vol. 31 No. 5, pp. 552‐61.
Santhanam, R. (2002), “Improving training outcomes using pre‐training scripts: a theory of planned behavior approach”, Information and Organization, Vol. 12 No. 3, pp. 135‐52.
Schneberger, S., Amoroso, D.L. and Durfee, A. (2007), “Factors that influence the performance of computer‐based assessments: an extension of the technology acceptance model”, The Journal of Computer Information Systems, Vol. 48 No. 2, pp. 74‐90.
Sein, M.K., Bostrom, R.P. and Olfman, L. (1987), “Training end users to compute: cognitive, motivational social issues”, INFOR, Vol. 25 No. 3, pp. 236‐55.
Simon, S.J. (2000), “The relationship of learning style and training method to end‐user computer satisfaction and computer use: a structural equation model”, Information Technology, Learning, and Performance Journal, Vol. 18 No. 1, pp. 41‐59.
Simon, S.J., Grover, V., Teng, J.T.C. and Whitcomb, K. (1996), “The relationship of information system training methods and cognitive ability to end‐user satisfaction, comprehension, and skill transfer: a longitudinal field study”, Information Systems Research, Vol. 7 No. 4, pp. 466‐90.
Sugrue, B. (2003), State of the Industry 2003: Astd's Annual Review of US and International Trends in Workplace Learning and Performance (Astd State of the Industry Report), American Society for Training, Alexandria, VA.
Thompson, R., Compeau, D. and Higgins, C. (2006), “Intentions to use information technologies: an integrative model”, Journal of Organizational and End User Computing, Vol. 18 No. 3, p. 25.
Tornatzky, L.G. and Fleischer, M. (1990), The Processes of Technological Innovation, Heath & Co., Lexington, MA.
Training‐Magazine (2004), State of Industry Report, Nielsen Business Media, New York, NY, pp. 20‐36.
Westman, M., Hobfoll, S.E., Chen, S., Davidson, O.B. and Laski, S. (2005), “Organizational stress through the lens of conservation of resources (COR) theory”, in Perrewé, P. and Ganster, D.C. (Eds), Research in Occupational Stress and Well‐being, Vol. 4, JAI Press, Greenwich, CT, pp. 171‐224.
Zmud, R.W. and Lind, M.R. (1985), “Linking mechanism supporting end‐user computing”, Proceedings of the Twenty‐First Annual Conference on Computer Personnel Research, pp. 74‐80 (Minneapolis, MN, USA, May 02‐03, J.C. Wetherbe, Ed. SIGCPR '85, ACM, New York, NY).
Campbell, J.P. (1971), “Personal training & development”, Annual Review of Psychology, Annual Reviews, Palo Alto, CA.
Davis, F.D. and Yi, M.Y. (2004), “Improving computer skill training: behavior modeling, symbolic mental rehearsal, and the role of knowledge structures”, Journal of Applied Psychology, Vol. 89 No. 3, pp. 509‐23.
Dolezalek, H. (2004), “Training Magazine's 23rd Annual Comprehensive Analysis of Employer‐sponsored Training in the United States”, Training, Vol. 41 No. 10, p. 20.
Jurison, J. (2000), “Perceived value and technology adoption across four end‐user groups”, Journal of End User Computing, Vol. 12 No. 4, pp. 21‐8.
Mao, J. and Brown, B.R. (2005), “The effectiveness of online task support vs instructor‐led training”, Journal of Organizational and End User Computing, Vol. 17 No. 3, pp. 27‐46.
Pitman, B. (1994), “Stop wasting training dollars: train for outcomes, not outputs”, Journal of Systems Management, Vol. 45 No. 6, p. 25.
Shayo, C. and Olfman, L. (2000), “The role of training in preparing end‐users to learn related software”, Journal of End User Computing, Vol. 12 No. 1, pp. 3‐13.
Worthen, B.R. and Sanders, J.R. (1987), Education Evaluation, Longman, New York, NY.