Cách nghiên cứu kỹ thuật phần mềm phù hợp với khoa học thiết kế: một bài đánh giá

Empirical Software Engineering - Tập 25 - Trang 2630-2660 - 2020
Emelie Engström1, Margaret-Anne Storey2, Per Runeson1, Martin Höst1, Maria Teresa Baldassarre3
1Lund University, Lund, Sweden
2University of Victoria, Victoria, Canada
3University of Bari, Bari, Italy

Tóm tắt

Đánh giá và truyền đạt nghiên cứu kỹ thuật phần mềm có thể gặp nhiều thách thức. Khoa học thiết kế được công nhận là một mô hình nghiên cứu phù hợp cho nghiên cứu ứng dụng, nhưng hiếm khi được sử dụng một cách rõ ràng để trình bày các đóng góp nghiên cứu đã lên kế hoạch hoặc đã đạt được trong lĩnh vực kỹ thuật phần mềm. Việc áp dụng lăng kính khoa học thiết kế vào nghiên cứu kỹ thuật phần mềm có thể cải thiện việc đánh giá và truyền đạt các đóng góp nghiên cứu. Mục tiêu của nghiên cứu này là 1) hiểu xem liệu lăng kính khoa học thiết kế có giúp tóm tắt và đánh giá các đóng góp nghiên cứu kỹ thuật phần mềm hay không, và 2) đặc trưng hóa các loại đóng góp khoa học thiết kế khác nhau trong tài liệu kỹ thuật phần mềm. Trong các nghiên cứu trước đây, chúng tôi đã phát triển một mẫu tóm tắt hình ảnh, tóm tắt các cấu trúc cốt lõi của mô hình khoa học thiết kế. Trong nghiên cứu này, chúng tôi sử dụng mẫu này trong một bài đánh giá của một tập hợp 38 bài báo kỹ thuật phần mềm đạt giải thưởng để trích xuất, phân tích và đặc trưng hóa các đóng góp khoa học thiết kế của chúng. Chúng tôi đã xác định năm cụm bài báo, phân loại chúng theo các loại đóng góp khoa học thiết kế khác nhau. Lăng kính khoa học thiết kế giúp nhấn mạnh đóng góp lý thuyết của kết quả nghiên cứu—về mặt các quy tắc công nghệ—và phản ánh về tính liên quan thực tiễn, tính mới lạ và độ chặt chẽ của các quy tắc đề xuất bởi nghiên cứu.

Từ khóa

#kỹ thuật phần mềm #khoa học thiết kế #đánh giá nghiên cứu #đóng góp lý thuyết #quy tắc công nghệ

Tài liệu tham khảo

Beecham S, O’Leary P, Baker S, Richardson I, Noll J (2014) Making software engineering research relevant. Computer 47(4):80–83. https://doi.org/10.1109/MC.2014.92 Budgen D, Kitchenham BA, Charters SM, Turner M, Brereton P, Linkman SG (2008) Presenting software engineering results using structured abstracts: a randomised experiment. Empir Softw Eng 13 (4):435–468. https://doi.org/10.1007/s10664-008-9075-7 Cartaxo B, Pinto G, Vieira E, Soares S (2016) Evidence briefings: Towards a medium to transfer knowledge from systematic reviews to practitioners. In: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM ’16, pp 57:1–57:10 Carver JC, Juristo N, Baldassarre MT, Vegas S (2014) Replications of software engineering experiments. Empir Softw Eng 19 (2):267–276. https://doi.org/10.1007/s10664-013-9290-8 Deng Q, Ji S (2018) A review of design science research in information systems: Concept, process, outcome, and evaluation. Pacific Asia journal of the association for information systems, vol 10 Deng Q, Wang Y, Ji S (2017) Design science research in information systems: A systematic literature review 2001-2015. In: CONF-IRM 2017 Proceedings Dybå T, Sjøberg D, Cruzes DS (2012) What works for whom, where, when, and why? On the role of context in empirical software engineering. In: Proceedings of the 2012 ACM-IEEE international symposium on empirical software engineering and measurement, pp 19–28. https://doi.org/10.1145/2372251.2372256 Gregor S, Hevner AR (2013) Positioning and presenting design science research for maximum impact. MIS Q 37(2):337–356 Grigoleit F, Vetro A, Diebold P, Fernandez DM, Bohm W (2015) In quest for proper mediums for technology transfer in software engineering. In: 2015 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp 1–4. https://doi.org/10.1109/ESEM.2015.7321203 Hannay JE, Sjöberg DIK, Dybå T (2007) A systematic review of theory use in software engineering experiments. IEEE Trans Softw Eng 33(2):87–107 Hevner AR, March ST, Park J, Ram S (2004) Design science in information systems research. MIS Q 28(1):75–105 Indulska M, Recker JC (2008) Design science in IS research : a literature analysis. In: Gregor S, Ho S (eds) 4th Biennial ANU workshop on information systems foundations. ANU E Press, Canberra Johannesson P, Perjons E (2014) An introduction to design science. Springer Publishing Company, Incorporated Juristo N, Gómez OS (2010) Replication of software engineering experiments. In: Empirical software engineering and verification. Springer, pp 60–88 Petersen K, Feldt R, Mujtaba S, Mattsson M (2008) Systematic mapping studies in software engineering. In: Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering, EASE’08. BCS Learning & Development Ltd., Swindon, pp 68–77 Rainer A, Hall T, Baddoo N (2003) Persuading developers to “buy into” software process improvement: a local opinion and empirical evidence. In: International Symposium on Empirical Software Engineering, ISESE, pp 326–335 Sedlmair M, Meyer M, Munzner T (2012) Design study methodology: Reflections from the trenches and the stacks. IEEE Trans Vis Comput Graph 18(12):2431–2440. https://doi.org/10.1109/TVCG.2012.213 Sein MK, Henfridsson O, Purao S, Rossi M, Lindgren R (2011) Action design research. MIS Q 35(1):37–56 Shneiderman B (2016) The new ABCs of research: achieving breakthrough collaborations, 1st edn. Oxford University Press, Inc., New York Shull FJ, Carver JC, Vegas S, Juristo N (2008) The role of replications in empirical software engineering. Empir Softw Eng 13(2):211–218. https://doi.org/10.1007/s10664-008-9060-1 Sjøberg DI, Dybå T, Anda BC, Hannay JE (2008) Building theories in software engineering. In: Guide to advanced empirical software engineering. Springer, pp 312–336 Stol KJ, Fitzgerald B (2013) Uncovering theories in software engineering. In: 2013 2nd SEMAT Workshop on a General Theory of Software Engineering (GTSE), pp 5–14. https://doi.org/10.1109/GTSE.2013.6613863 Stol KJ, Fitzgerald B (2015) Theory-oriented software engineering. Towards general theories of software engineering, vol 101, pp 79–98, DOI https://doi.org/10.1016/j.scico.2014.11.010 Storey MA, Engström E, Höst M, Runeson P, Bjarnason E (2017) Using a visual abstract as a lens for communicating and promoting design science research in software engineering. In: Empirical Software Engineering and Measurement (ESEM), pp 181–186. https://doi.org/10.1109/ESEM.2017.28 van Aken JE (2004) Management research based on the paradigm of the design sciences: the quest for field-tested and grounded technological rules: paradigm of the design sciences. J Manag Stud 41(2):219–246. https://doi.org/10.1111/j.1467-6486.2004.00430.x van Aken JE (2005) Management research as a design science: articulating the research products of mode 2 knowledge production in management. Br J Manag 16 (1):19–36. https://doi.org/10.1111/j.1467-8551.2005.00437.x Wieringa R (2009) Design science as nested problem solving. In: Proceedings of the 4th International Conference on Design Science Research in Information Systems and Technology, DESRIST ’09. ACM, New York, pp 8:1–8:12, DOI https://doi.org/10.1145/1555619.1555630, (to appear in print) Wieringa R, Daneva M, Condori-Fernandez N (2011) The structure of design theories, and an analysis of their use in software engineering experiments. In: 2011 International symposium on empirical software engineering and measurement, pp 295–304 Wieringa R, Moralı A (2012) Technical action research as a validation method in information systems design science. In: Peffers K, Rothenberger M, Kuechler B (eds) Design science research in information systems. Advances in theory and practice. Springer, Berlin, pp 220–238 Wieringa RJ (2014) Design science methodology for information systems and software engineering. Springer, Berlin Wohlin C, Aurum A (2015) Towards a decision-making structure for selecting a research design in empirical software engineering. Empir Softw Eng 20(6):1427–1455. https://doi.org/10.1007/s10664-014-9319-7 Alimadadi S, Sequeira S, Mesbah A, Pattabiraman K (2014) Understanding javascript event-based interactions. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014. ACM, New York, pp 367–377, DOI https://doi.org/10.1145/2568225.2568268, (to appear in print) Avgerinos T, Rebert A, Cha SK, Brumley D (2014) Enhancing symbolic execution with veritesting. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014. ACM, New York, pp 1083–1094, DOI https://doi.org/10.1145/2568225.2568293, (to appear in print) Bell J, Kaiser G (2014) Unit test virtualization with vmvm. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014. ACM, New York, pp 550–561, DOI https://doi.org/10.1145/2568225.2568248, (to appear in print) Bersani MM, Bianculli D, Ghezzi C, Krstić S, Pietro PS (2016) Efficient large-scale trace checking using mapreduce. In: Proceedings of the 38th International Conference on Software Engineering, ICSE ’16. ACM, New York, pp 888–898, DOI https://doi.org/10.1145/2884781.2884832, (to appear in print) Cacho N, César T, Filipe T, Soares E, Cassio A, Souza R, Garcia I, Barbosa EA, Garcia A (2014) Trading robustness for maintainability: An empirical study of evolving c# programs. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014. ACM, New York, pp 584–595, DOI https://doi.org/10.1145/2568225.2568308, (to appear in print) Christakis M, Müller P, Wüstholz V (2016) Guiding dynamic symbolic execution toward unverified program executions. In: Proceedings of the 38th International Conference on Software Engineering, ICSE ’16. ACM, New York, pp 144–155, DOI https://doi.org/10.1145/2884781.2884843, (to appear in print) Fan L, Su T, Chen S, Meng G, Liu Y, Xu L, Pu G, Su Z (2018) Large-scale analysis of framework-specific exceptions in android apps. In: Proceedings of the 40th International Conference on Software Engineering, ICSE ’18. ACM, New York, pp 408–419, DOI https://doi.org/10.1145/3180155.3180222, (to appear in print) Floyd B, Santander T, Weimer W (2017) Decoding the representation of code in the brain: An fmri study of code review and expertise. In: Proceedings of the 39th international conference on software engineering. IEEE Press, pp 175–186 Gousios G, Storey MA, Bacchelli A (2016) Work practices and challenges in pull-based development: The contributor’s perspective. In: Proceedings of the 38th International Conference on Software Engineering, ICSE ’16. ACM, New York, pp 285–296, DOI https://doi.org/10.1145/2884781.2884826, (to appear in print) Hasan S, King Z, Hafiz M, Sayagh M, Adams B, Hindle A (2016) Energy profiles of java collections classes. In: Proceedings of the 38th International Conference on Software Engineering, ICSE ’16. ACM, New York, pp 225–236, DOI https://doi.org/10.1145/2884781.2884869, (to appear in print) Hoda R, Noble J (2017) Becoming agile: a grounded theory of agile transitions in practice. In: Proceedings of the 39th International Conference on Software Engineering. IEEE Press, pp 141–151 Inozemtseva L, Holmes R (2014) Coverage is not strongly correlated with test suite effectiveness. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014. ACM, New York, pp 435–445, DOI https://doi.org/10.1145/2568225.2568271, (to appear in print) Landman D, Serebrenik A, Vinju JJ (2017) Challenges for static analysis of java reflection: Literature review and empirical study. In: Proceedings of the 39th International Conference on Software Engineering, ICSE ’17. IEEE Press, Piscataway, pp 507–518, DOI https://doi.org/10.1109/ICSE.2017.53, (to appear in print) Lavallée M, Robillard PN (2015) Why good developers write bad code: An observational case study of the impacts of organizational factors on software quality. In: Proceedings of the 37th International Conference on Software Engineering - Volume 1, ICSE ’15. IEEE Press, Piscataway, pp 677–687 Liu Y, Xu C, Cheung SC (2014) Characterizing and detecting performance bugs for smartphone applications. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014. ACM, New York, pp 1013–1024, DOI https://doi.org/10.1145/2568225.2568229, (to appear in print) Loncaric C, Ernst MD, Torlak E (2018) Generalized data structure synthesis. In: Chaudron M, Crnkovic I, Chechik M, Harman M (eds) Proceedings of the 40th international conference on software engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018. ACM, pp 958–968. https://doi.org/10.1145/3180155.3180211 Madsen M, Tip F, Andreasen E, Sen K, Møller A (2016) Feedback-directed instrumentation for deployed javascript applications. In: Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016, pp 899–910. https://doi.org/10.1145/2884781.2884846 Menendez D, Nagarakatte S (2016) Termination-checking for llvm peephole optimizations. In: Proceedings of the 38th International Conference on Software Engineering, ICSE ’16. ACM, New York, pp 191–202, DOI https://doi.org/10.1145/2884781.2884809, (to appear in print) Milicevic A, Near JP, Kang E, Jackson D (2015) Alloy*: A general-purpose higher-order relational constraint solver. In: Proceedings of the 37th international conference on software engineering - Volume 1, ICSE ’15. IEEE Press, Piscataway, pp 609–619 Murphy-Hill E, Zimmermann T, Nagappan N (2014) Cowboys, ankle sprains, and keepers of quality: How is video game development different from software development?. In: Proceedings of the 36th International Conference on Software Engineering, ICSE 2014. ACM, New York, pp 1–11, DOI https://doi.org/10.1145/2568225.2568226, (to appear in print) Nistor A, Chang PC, Radoi C, Lu S (2015) Caramel: Detecting and fixing performance problems that have non-intrusive fixes. In: Proceedings of the 37th international conference on software engineering - Volume 1, ICSE ’15. IEEE Press, Piscataway, pp 902–912 Okur S, Hartveld DL, Dig D, Deursen AV (2014) A study and toolkit for asynchronous programming in c#. In: Proceedings of the 36th international conference on software engineering, ICSE 2014. ACM, New York, pp 1117–1127, DOI https://doi.org/10.1145/2568225.2568309, (to appear in print) Rath M, Rendall J, Guo JLC, Cleland-Huang J, Mäder P. (2018) Traceability in the wild: Automatically augmenting incomplete trace links. In: Proceedings of the 40th international conference on software engineering, ICSE ’18. ACM, New York, pp 834–845, DOI https://doi.org/10.1145/3180155.3180207, (to appear in print) Ren Z, Jiang H, Xuan J, Yang Z (2018) Automated localization for unreproducible builds. In: Chaudron M, Crnkovic I, Chechik M, Harman M (eds) Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018. ACM, pp 71–81. https://doi.org/10.1145/3180155.3180224 Rizzi EF, Elbaum S, Dwyer MB (2016) On the techniques we create, the tools we build, and their misalignments: a study of klee. In: Proceedings of the 38th international conference on software engineering. ACM, pp 132–143 Rodeghero P, McMillan C, McBurney PW, Bosch N, D’Mello SK (2014) Improving automated source code summarization via an eye-tracking study of programmers. In: 36th International Conference on Software Engineering, ICSE ’14, Hyderabad, India - May 31 - June 07, 2014, pp 390–401. https://doi.org/10.1145/2568225.2568247 Rojas JM, White TD, Clegg BS, Fraser G (2017) Code defenders: crowdsourcing effective tests and subtle mutants with a mutation testing game. In: Proceedings of the 39th International Conference on Software Engineering, ICSE 2017, Buenos Aires, Argentina, May 20-28, 2017, pp 677–688. https://doi.org/10.1109/ICSE.2017.68 Shi A, Thummalapenta S, Lahiri SK, Bjorner N, Czerwonka J (2017) Optimizing test placement for module-level regression testing. In: Proceedings of the 39th International Conference on Software Engineering, ICSE ’17. IEEE Press, Piscataway, pp 689–699, DOI https://doi.org/10.1109/ICSE.2017.69, (to appear in print) Siegmund J, Siegmund N, Apel S (2015) Views on internal and external validity in empirical software engineering. In: Proceedings of the 37th international conference on software engineering-Volume 1. IEEE Press, pp 9–19 Sousa L, Oliveira A, Oizumi W, Barbosa S, Garcia A, Lee J, Kalinowski M, de Mello R, Fonseca B, Oliveira R, et al. (2018) Identifying design problems in the source code: a grounded theory. In: Proceedings of the 40th International Conference on Software Engineering. ACM, pp 921–931 van Tonder R, Goues CL (2018) Static automated program repair for heap properties. In: Proceedings of the 40th International Conference on Software Engineering, ICSE ’18. ACM, New York, pp 151–162, DOI https://doi.org/10.1145/3180155.3180250, (to appear in print) Tsantalis N, Mazinanian D, Rostami S (2017) Clone refactoring with lambda expressions. In: Proceedings of the 39th International Conference on Software Engineering, ICSE ’17. IEEE Press, Piscataway, pp 60–70, DOI https://doi.org/10.1109/ICSE.2017.14, (to appear in print) Tufano M, Palomba F, Bavota G, Oliveto R, Di Penta M, De Lucia A, Poshyvanyk D (2015) When and why your code starts to smell bad. In: Proceedings of the 37th International Conference on Software Engineering - Volume 1, ICSE ’15. IEEE Press, Piscataway, pp 403–414 Wang X, Sun J, Chen Z, Zhang P, Wang J, Lin Y (2018) Towards optimal concolic testing. In: Proceedings of the 40th International Conference on Software Engineering, ICSE ’18. ACM, New York, pp 291–302, DOI https://doi.org/10.1145/3180155.3180177, (to appear in print) Waterman M, Noble J, Allan G (2015) How much up-front?: A grounded theory of agile architecture. In: Proceedings of the 37th International Conference on Software Engineering - Volume 1, ICSE ’15. IEEE Press, Piscataway, pp 347–357 Yan H, Sui Y, Chen S, Xue J (2018) Spatio-temporal context reduction: a pointer-analysis-based static approach for detecting use-after-free vulnerabilities. In: Chaudron M, Crnkovic I, Chechik M, Harman M (eds) Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018. ACM, pp 327–337. https://doi.org/10.1145/3180155.3180178 Ye X, Shen H, Ma X, Bunescu R, Liu C (2016) From word embeddings to document similarities for improved information retrieval in software engineering. In: Proceedings of the 38th International Conference on Software Engineering, ICSE ’16. ACM, New York, pp 404–415, DOI https://doi.org/10.1145/2884781.2884862, (to appear in print) Yu T, Qu X, Cohen MB (2016) Vdtest: an automated framework to support testing for virtual devices. In: Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016, pp 583–594. https://doi.org/10.1145/2884781.2884866