An information retrieval-based regression test selection technique
Tóm tắt
Regression testing (RT) is the crucial part of the software testing process. It is applied after a bug fix or a change in the functionality of the codebase. The main goal is to ensure that the modified software has the desired outcome and does not cause adverse effects in other parts of the software. RT may be costly depending on the test’s quantity and complexity. Therefore, regression test selection (RTS) can be introduced to minimize these costs. RTS runs only the test cases related to the modified parts of the software. Currently, various RTS studies focus on compiled languages such as Java, C/C++, and C#, and they mostly rely on direct code dependency between tests and the system under test. In this study, we have introduced a new RTS tool called Smartest to reduce the number of selected integration tests. Former RTS tools were focused mainly on unit tests according to dependencies of modified source files. Smartest is the first RTS tool that works for software written in JavaScript and can select integration tests written in natural language by the quality assurance team. Smartest is tested on three commercial projects and observed that it picks 13% of all test cases on average. Experiments show that Smartest minimizes the selected integration tests on RTS processes, although it does not use file-level code dependency.
Tài liệu tham khảo
Rothermel, G., Harrold, M.J.: A Safe, Efficient Regression Test Selection Technique. ACM Trans. Softw. Eng. Methodol. 2(6), 173–210 (1997)
Zhang., L.: Hybrid regression test selection. In: IEEE/ACM 40th International Conference on Software Engineering (ICSE), pp. 199–209 (2018)
Shi, A., Hadzo-Tanovic, M., Zhang, L., Marinov, D., Legunsen, O.: Reflection-aware static regression test selection. In: Proc. ACM Program. Lang. 3, OOPSLA, Article 187, p. 29 (2019)
Zhong, H., Zhang, L., Khurshid, S.: TestSage: regression test selection for large-scale web service testing. In: 12th IEEE Conference on Software Testing, Validation and Verification (ICST) (2019)
Sawant, A.A., Bari, P.H., Chawan, P.M.: Software testing techniques and strategies. Int. J. Eng. Res. Appl. 2(3), 980–986 (2012)
Sakamoto, K., Tomohiro, K., Hamura, D., Washizaki, H., Fukazawa, Y.: POGen: a test code generator based on template variable coverage in gray-box integration testing for web applications. In: International Conference on Fundamental Approaches to Software Engineering, pp. 343–358. Springer, Berlin (2013)
Khan, I.A., Singh, R.: Quality assurance and integration testing aspects in web based applications (2012). arXiv:1207.3213
Elbaum, S., Rothermel, G., Penix, J.: Techniques for improving regression testing in continuous integration development environments. In: FSE 2014: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp 235–245 (2014)
Celik, A., Lee, Y.C., Gligoric, M.: Regression test selection for TizenRT. In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2018), pp. 845–850. Association for Computing Machinery, New York
Gligoric, M., Eloussi, L., Marinov, D.: Practical regression test selection with dynamic file dependencies. In: ISSTA 2015: Proceedings of the 2015 International Symposium on Software Testing and Analysis, pp. 211–222 (2015)
Stack Overflow Developer Survey.: (2021). https://insights.stackoverflow.com/survey/2021/
Kauhanen, E., Nurminen, J.K., Mikkonen, T., Pashkovskiy, M.: Regression test selection tool for python in continuous integration process. In: 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 618–621 (2021)
Jest: Delightful JavaScript Testing. https://jestjs.io/
Gligoric, M.: Regression Test Selection: Theory and Practice. University of Illinois at Urbana-Champaign (2015)
Blondeau, V., Etien, A., Anquetil, N., Cresson, S., Croisy, P., Ducasse, S.: What are the testing habits of developers? A case study in a large IT company. In: 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 58–68 (2017)
Gligoric, M., Negara, S., Legunsen, O., Marinov, D.: An empirical evaluation and comparison of manual and automated test selection. In: Proceedings of the 29th ACM/IEEE international conference on Automated software engineering, pp. 361–72 (2014)
Zhang, L., Kim, M., Khurshid, S.: Localizing failure-inducing program edits based on spectrum information. In: 2011 27th IEEE International Conference on Software Maintenance (ICSM), pp. 23–32 (2011)
Ren, X., Shah, F., Top, F., Ryder, B.G., Chianti, C.O.: A tool for change impact analysis of java programs. In: OOPSLA ’04: Proceedings of the 19th annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, pp. 432–448 (2004)
Chen, Y.F., Rosenblum, D.S., Vo, K.P.: TestTube: a system for selective regression testing. In: Proceedings of 16th International Conference on Software Engineering, pp. 211–220 (1994)
Chen, Y.: NodeSRT: a selective regression testing tool for Node.js application. In: 2021 IEEE/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 126–128. IEEE (2021)
Legunsen, O., Hariri, F., Shi, A., Lu, Y., Zhang, L., Marinov, D.: An Extensive Study of Static Regression Test Selection in Modern Software Evolution. Association for Computing Machinery (2016)
Gligoric, M., Eloussi, L., Marinov, D.: Ekstazi: lightweight test selection. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, pp. 713–716 (2015)
Vasic, M., Parvez, Z., Milicevic, A., Gligoric, M.: File-level vs. module-level regression test selection for .NET. In: ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, pp. 848–853 (2017)
Shi, A., Zhao, P., Marinov, D.: Understanding and improving regression test selection in continuous integration. In: 2019 IEEE 30th International Symposium on Software Reliability Engineering (ISSRE), pp. 228–238 (2019)
Rosenblum, D.S., Rothermel, G.: A comparative study of regression test selection techniques. In: Proceedings of the IEEE Computer Society 2nd International Workshop on Empirical Studies of Software Maintenance (1997)
Legunsen, O., Shi, A., Marinov, D.: STARTS: STAtic regression test selection. In: 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE), pp. 949–954 (2017)
Zheng, J., Williams, L., Pallino, R.B.: Automation to support regression test selection for COTS-based applications. In: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering (ASE ’07), pp. 224–233. Association for Computing Machinery, New York (2006)
Bible, J., Rothermel, G., Rosenblum, D.S.: A comparative study of coarse- and fine-grained safe regression test selection techniques. ACM Trans. Softw. Eng. Methodol. (2001)
Hirzel, M., Klaeren, H.: Graph-walk-based selective regression testing of web applications created with google web toolkit
Raulji, J.K., Jatinderkumar, R.S.: Stop-word removal algorithm and its implementation for Sanskrit Language. Int. J. Comput. Appl. (0975-8887) 150(2), 15–17 (2016)
Białecki, A., Muir, R., Ingersoll, G., Imagination, L.: Apache Lucene 4. In: InSIGIR 2012 Workshop on Open Source Information Retrieval, p. 17 (2012)
The Apache Lucene.: https://lucene.apache.org/
Thisted, R.A.: Elements of Statistical Computing: Numerical Computation. Routledge, New York (1988)
Erşahin, M., Utku, S., Kılınç, D., Erşahin, B.: Information retrieval-based bug localization approach with adaptive attribute weighting
Erşahin, B., Aktaş, O., Kılınç, D., Erşahin, M.: A hybrid sentiment analysis method for Turkish. Turk. J. Electr. Eng. Comput. Sci. 27(3), 1780–1793 (2019). https://doi.org/10.3906/elk-1808-189