Patch rejection in Firefox: negative reviews, backouts, and issue reopening
Tóm tắt
Writing patches to fix bugs or implement new features is an important software development task, as it contributes to raise the quality of a software system. Not all patches are accepted in the first attempt, though. Patches can be rejected because of problems found during code review, automated testing, or manual testing. A high rejection rate, specially later in the lifecycle, may indicate problems with the software development process. Our objective is to better understand the relationship among different forms of patch rejection and to characterize their frequency within a project. This paper describes one step towards this objective, by presenting an analysis of a large open source project, Firefox. In order to characterize patch rejection, we relied on issues and source code commits from over four years of the project’s history. We computed monthly metrics on the occurrence of three indicators of patch rejection—negative code reviews, commit backouts, and bug reopening—and measured the time it takes both to submit a patch and to reject inappropriate patches. In Firefox, 20 % of the issues contain rejected patches. Negative reviews, backouts, and issue reopening are relatively independent events; in particular, about 70 % of issue reopenings are premature; 75 % of all inappropriate changes are rejected within four days. Patch rejection is a frequent event, occurring multiple times a day. Given the relative independence of rejection types, existing studies that focus on one single rejection type fail to detect many rejections. Although inappropriate changes cause rework, they have little effect on the quality of released versions of Firefox.
Tài liệu tham khảo
Almossawi, A (2012) Investigating the Architectural Drivers of Defects in Open-source Software Systems: an Empirical Study of Defects and Reopened Defects in GNOME. Master thesis, Massachusetts Institute of Technology, 2012.
An, L, Khomh F, Adams B (2014) Supplementary bug fixes vs. re-opened bugs In: Source Code Analysis and Manipulation (SCAM), IEEE 14th International Working Conference On, 205–214.. IEEE, Washington, DC, USA.
Bird, C, Gourley A, Devanbu P (2007) Detecting patch submission and acceptance in oss projects In: Proceedings of the Fourth International Workshop on Mining Software Repositories. MSR 2007, 26.. IEEE Computer Society, Washington, DC, USA. doi:10.1109/MSR.2007.6.
Bugzilla (2015) Bug Fields. https://bugzilla.mozilla.org/page.cgi?id=fields.html Accessed 13/06/2015.
Chapin, N, Hale JE, Kham KM, Ramil JF, Tan WG (2001) Types of software evolution and software maintenance. J Softw Maint 13(1): 3–30.
Cockburn, A, Williams L (2003) Agile software development: It’s about feedback and change. Computer 36(6): 0039–43.
Herzig, K, Just S, Zeller A (2013) It’s not a bug, it’s a feature: How misclassification impacts bug prediction In: Proceedings of the 2013 International Conference on Software Engineering. ICSE ’13, 392–401.. IEEE Press, Piscataway, NJ, USA.
Jeong, G, Kim S, Zimmermann T, Yi K (2009) Improving code review by predicting reviewers and acceptance of patches. Technical Report ROSAEC MEMO 2009-006.
Jongyindee, A, Ohira M, Ihara A, Matsumoto KI (2011) Good or bad committers? a case study of committers’ cautiousness and the consequences on the bug fixing process in the Eclipse project In: Proc. of the 2011 Joint Conf. of the 21st International Workshop on Softw. Measurement and the 6th International Conf. on Softw. Process and Product Measurement, 116–125.. IEEE Computer Society, Washington, DC, USA. doi:10.1109/IWSM-MENSURA.2011.24.
Khomh, F, Dhaliwal T, Zou Y, Adams B (2012) Do faster releases improve software quality? an empirical case study of mozilla firefox. In: Lanza M, Pent MD, Xi T (eds)9th IEEE Working Conference on Mining Software Repositories, MSR 2012, 179–188.. IEEE, Washington, DC, USA.
Lehman, MM, Ramil JF, Wernick PD, Perry DE, Turski WM (1997) Metrics and laws of software evolution-the nineties view In: Software Metrics Symposium, 1997. Proceedings., Fourth International, 20–32.. IEEE, Washington, DC, USA.
Mantyla, MV, Khomh F, Adams B, Engstrom E, Petersen K (2013) On rapid releases and software testing In: Software Maintenance (ICSM), 29th IEEE International Conference On, 20–29.. IEEE Computer Society, Washington, DC, USA. doi:10.1109/ICSM.2013.13.
Mozilla (2011) Mozilla Release Processes. http://mozilla.github.io/process-releases/ Accessed 13/06/2015.
Mozilla (2014) Mozilla wiki: Releases. https://wiki.mozilla.org/Releases Accessed 13/06/2015.
Nurolahzade, M, Nasehi SM, Khandkar SH, Rawal S (2009) The role of patch review in software evolution: An analysis of the mozilla firefox In: Proceedings of the Joint International and Annual ERCIM Workshops on Principles of Software Evolution (IWPSE) and Software Evolution (Evol) Workshops. IWPSE-Evol ’09, 9–18.. ACM, New York, NY, USA. doi:10.1145/1595808.1595813.
Park, J, Kim M, Ray B, Bae DH (2012) An empirical study of supplementary bug fixes In: 9th IEEE Working Conference on Mining Software Repositories, 40–49.. IEEE Computer Society, Washington, DC, USA. doi:10.1109/MSR.2012.6224298.
Plewnia, C, Dyck A, Lichter H (2014) On the influence of release engineering on software reputation In: 2nd International Workshop on Release Engineering. http://releng.polymtl.ca/RELENG2014/html/proceedings/releng2014_submission__3.pdf.
Rigby, PC, Storey MA (2011) Understanding broadcast based peer review on open source software projects In: Proceedings of the 33rd International Conference on Software Engineering, 541–550.. ACM, New York, NY, USA.
Rigby, PC, German DM, Cowen L, Storey MA (2014) Peer review on open source software projects: Parameters, statistical models, and theory. ACM Trans Softw Eng Methodol34.
Shihab, E, Ihara A, Kamei Y, Ibrahim WM, Ohira M, Adams B, Hassan AE, Matsumoto KI (2010) Predicting re-opened bugs: A case study on the eclipse project In: Proceedings of the 2010 17th Working Conference on Reverse Engineering. WCRE ’10, 249–258.. IEEE Computer Society, Washington, DC, USA. doi:10.1109/WCRE.2010.36.
Shihab, E (2012) An exploration of challenges limiting pragmatic software defect prediction. PhD thesis, School of Computing, Queen’s University, Kingston, Ontario, Canada.
Souza, R, Chavez C, Bittencourt RA (2014) Do rapid releases affect bug reopening? A case study of Firefox In: Software Engineering (SBES), 2014 Brazilian Symposium On, 31–40.. IEEE Computer Society, Washington, DC, USA. doi:10.1109/SBES.2014.10.
Souza, R, Chavez C, Bittencourt R (2015) Rapid releases and patch backouts: A software analytics approach. Softw IEEE 32(2): 89–96. doi:10.1109/MS.2015.30.
Tao, Y, Han D, Kim S (2014) Writing acceptable patches: An empirical study of open source project patches In: 30th IEEE International Conference on Software Maintenance and Evolution, Victoria, BC, Canada, September 29 - October 3, 2014, 271–280.. IEEE Computer Society, Washington, DC, USA. doi:10.1109/ICSME.2014.49.
Zimmermann, T, Nagappan N, Guo PJ, Murphy B (2012) Characterizing and predicting which bugs get reopened In: Proceedings of the 2012 International Conference on Software Engineering. ICSE 2012, 1074–1083.. IEEE Press, Piscataway, NJ, USA.