Benchmarking software development productivity of CMMI level 5 projects

Information Technology and Management - Tập 16 - Trang 235-251 - 2015
Dinesh R. Pai1, Girish H. Subramanian1, Parag C. Pendharkar1
1School of Business Administration, Pennsylvania State University at Harrisburg, Middletown, USA

Tóm tắt

In this paper, data envelopment analysis variable returns to scale (DEA VRS) model is applied to data collected on 79 software development projects from a leading CMMI level 5 organization. We divide overall software effort into software development effort, software quality conformance effort (EoC), and software maintenance non-conformance (EoNC) effort due to poor software quality at delivery time. Partitioning effort into software development and software quality metrics provides us a comprehensive model to measure productivity of software projects and to identify best practice projects. Some of positive productivity drivers from the DEA best practice efficient projects point to good customer rapport and application familiarity. Inefficient projects had problems such as customer requirements volatility, and the use of unfamiliar technology. The DEA results identify 12 “best practice” projects that can be emulated for software process improvement. Additionally, our results point to approximately 50 % potential for productivity improvement in software projects to get to the level of “best practice” projects. This study shows that including EoC and EoNC as inputs has a positive impact on the best practice frontier.

Tài liệu tham khảo

Hill PR (2010) Practical software project estimation: a toolkit for estimating software development effort & duration. McGraw-Hill, San Francisco

Jalote P (2004) CMM in practice: processes for executing software projects at Infosys. Pearson, Delhi

Flitman A (2003) Towards meaningful benchmarking of software development team productivity. Benchmarking Int J 10(4):382–399

Paradi JC, Reese DN, Rosen D (1997) Applications of DEA to measure the efficiency of software production at two large Canadian banks. Ann Oper Res 73:91–115

Parthasarathy S, Anbazhagan N (2008) Evaluating ERP projects using DEA and regression analysis. Int J Bus Inf Syst 3(2):140–157

Stensrud E, Myrtveit I (2003) Identifying high performance ERP projects. IEEE Trans Comput 29(5):398–416

Yang Z, Paradi JC (2004) DEA evaluation of a Y2K software retrofit program. IEEE Trans Eng Manag 51(3):279–287

Card DN, El Emam K, Scalzo B (2001) Measurement of object-oriented software development projects. Software Productivity Consortium NFP, Herndon

Basili VR, Briand LC, Melo WL (1996) How reuse influences productivity in object-oriented systems. Commun ACM 39(10):104–116

Blackburn JD, Scudder GD, Van Wassenhove LN (1996) Improving speed and productivity of software development: a global survey of software developers. IEEE Trans Softw Eng 22(12):875–885

Briand LC, El-Emam K, Bomarius F COBRA (1998) A hybrid method for software cost estimation, benchmarking and risk assessment. In: 20th international conference on software engineering, pp 390–399

Jeffery R, Ruhe M, Wieczorek I (2001) Using public domain metrics to estimate software development effort. In: Proceedings of METRICS’01, pp 16–27

Boehm B (1981) Software engineering economics. Prentice-Hall, Englewood Cliffs

Boehm B, Clark B, Horowitz E, Westland C, Madachy R, Selby R (1996) The Cocomo 2.0 software cost estimation model—a status report. Am Program. 9:2–17

Banker RD, Kemerer CF (1989) Scale economies in new software development. IEEE Trans Softw Eng 15(10):1199–1205

Pendharkar PC, Rodger JA, Subramanian GH (2008) An empirical study of the Cobb–Douglas production function properties of software development effort. Inf Softw Technol 50:1181–1188

Kneuper R (2008) Improving software and systems development processes using capability maturity model integration (CMMI-DEV). Rocky Nook, Santa Barbara

Goldenson DR, Gibson DL (2003) Demonstrating the impact and benefits of CMMI: an update and preliminary results. CMU/SEI-2003-SR-009. Carnegie Mellon Software Engineering Institute, Pittsburgh

Charnes A, Cooper WW, Lewin AY, Seiford LM (1994) Data envelopment analysis: theory, methodology, and applications. Kluwer, Boston

Charnes A, Cooper WW, Golany B, Seiford L, Stutz J (1985) Foundations of data envelopment analysis for Pareto-Koopmans efficient empirical production functions. J Econom 30:91–107

Emrouznedjad A, Parker BR, Tavares G (2008) Evaluation of research in efficiency and productivity: a survey and analysis of the first 30 years of scholarly literature in DEA. Soc Econom Plan Sci 44:151–157

Seiford LM (1996) Data envelopment analysis: the evaluation of the state of the art (1978–1995). J Prod Anal 9:99–137

Seiford LM, Thrall RM (1990) Recent developments in DEA: the mathematical programming approach to frontier analysis. J Econom 46:7–38

Tavares G (2002) A bibliography of data envelopment analysis (1978–2001). RUTCOR, Rutgers University, Newark

Charnes A, Cooper WW, Rhodes E (1978) Measuring the efficiency of decision making units. Eur J Oper Res 2:429–444

Banker RD, Charnes A, Cooper WW (1984) Some models for estimating technical and scale inefficiencies in data envelopment analysis. Manag Sci 30(9):1078–1092

Banker RD, Datar SM, Kemerer CF (1991) A model to evaluate variables impacting the productivity of software maintenance projects. Manag Sci 37(1):1–18

Elam J (1991) Evaluating the efficiency of IS organizations using data envelope analysis. In: Proceedings of the International Function Point Users Group

Banker RD, Chang H, Kemerer CF (1994) Evidence on economies of scale in software development. Inf Softw Technol 36(5):275–282

Mahmood MA, Pettingell KJ, Shaskevich AI (1996) Measuring productivity of software projects: a data envelope analysis approach. Decis Sci 27(1):57–80

Myrtveit I, Stensrud E (1999) Benchmarking COTS projects using data envelopment analysis. In: Proceedings of METRICS’99, Washington, D.C., pp 269–278

Zhang S, Wang Y, Tong J, Zhou J, Ruan L (2006) Evaluation of project quality: a DEA-based approach. Lect Notes Comput Sci 3966:88–96

Asmild M, Paradi JC, Kulkarni A (2006) Using data envelopment analysis in software development productivity measurement. Softw Process Improv Pract 11:561–572

Wray B, Mathieu R (2008) Evaluating the performance of open source software projects using data envelopment analysis. Inf Manag Comput Secur 16(5):449–462

Maxwell K, Forselius P (2000) Benchmarking software development productivity. Softw IEEE 17(1):80–88

Pendharkar PC, Rodger J (2009) The Relationship between software development team size and software development cost. Commun ACM 52(1):141–144

McCall JA, Richards PK, Walters GF (1977) Factors in software quality. Volumes 1, 2, 3, US Rome Air Development Center Reports NTIS AD/A-049 014, 015, 055

Boehm BW, Brown JR, Kaspar JR, Lipow M, MacCleod GJ, Merrit MJ (1978) Characteristics of software quality. North-Holland, Amsterdam

Fenton NE (1994) Software measurement: a necessary scientific basis. IEEE Trans Softw Eng 20(3):199–206

Kemayel L, Mili A, Ouederni I (1991) Controllable factors for programmer productivity: a statistical study. J Syst Softw 16(2):151–163

Zhu J (2003) Quantitative models for performance evaluation and benchmarking: data envelopment analysis with spreadsheets. Kluwer, Boston

Bowlin WF (1998) Measuring performance: an introduction to data envelopment analysis (DEA). J Cost Anal 7:3–27

Chen Y, Zhu J (2004) Measuring information technology’s indirect impact on firm performance. Inf Technol Manag 5(1):9–22

Soteriou AC, Zenios SA (1998) Data envelopment analysis: an introduction and an application to bank branch performance assessment. In: Marcoulides G (ed) Modern methods for business research. Lawrence Erlbaum Associates, London

Drake L, Howcroft B (1994) Relative efficiency in the branch network of a UK bank: an empirical study. Omega 22(1):83–90

Cooper WW, Seiford LM, Zhu J (2004) Handbook on data envelopment analysis. Springer, New York

Sexton TR, Silkman RH, Hogan AJ (eds) (1986) Data envelopment analysis: Critique and extensions, new directions for program evaluation. Jossey-Bass, San Francisco

Andersen P, Petersen NC (1993) A procedure for ranking efficient units in data envelopment analysis. Manag Sci 39:1261–1264

Seiford LM, Zhu J (1999) Infeasibility of super efficiency data envelopment analysis models. INFOR 37(2):174–187

Torgersen AM, Førsund FR, Kittelsen SA (1996) Slack adjusted efficiency measures and ranking of efficient units. J Prod Anal 7:379–398

Boh WF, Slaughter SA, Espinosa JA (2007) Learning from experience in software development: a multilevel analysis. Manag Sci 53(8):1315–1331

Keil M, Cule PE, Lyytinen K, Schmidt RC (1998) A framework for identifying software project risks. Commun ACM 41(11):76–83

Mantle MW, Lichty R (2012) Managing the unmanageable: rules, tools, and insights for managing software people and teams. Addison-Wesley, Boston

Kittelsen S (1999) Monte Carlo simulations of DEA efficiency measures and hypothesis tests. Department of Economics, University of Oslo, Oslo

Pendharkar PC, Rodger JA (2007) An empirical study of the impact of team size on software development effort. Inf Technol Manag 8:253–262

Thông tin
Thông tin xuất bản

Information Technology and Management

Tập 16

235-251

Thông tin tác giả