A graph-based code representation method to improve code readability classification

Empirical Software Engineering - Tập 28 - Trang 1-26 - 2023
Qing Mi1, Yi Zhan1, Han Weng1, Qinghang Bao1, Longjie Cui1, Wei Ma1
1Faculty of Information Technology, Beijing University of Technology, Beijing, China

Tóm tắt

Code readability is crucial for developers since it is closely related to code maintenance and affects developers’ work efficiency. Code readability classification refers to the source code being classified as pre-defined certain levels according to its readability. So far, many code readability classification models have been proposed in existing studies, including deep learning networks that have achieved relatively high accuracy and good performance. However, in terms of representation, these methods lack effective preservation of the syntactic and semantic structure of the source code. To extract these features, we propose a graph-based code representation method. Firstly, the source code is parsed into a graph containing its abstract syntax tree (AST) combined with control and data flow edges to reserve the semantic structural information and then we convert the graph nodes’ source code and type information into vectors. Finally, we train our graph neural networks model composing Graph Convolutional Network (GCN), DMoNPooling, and K-dimensional Graph Neural Networks (k-GNNs) layers to extract these features from the program graph. We evaluate our approach to the task of code readability classification using a Java dataset provided by Scalabrino et al. (2016). The results show that our method achieves 72.5% and 88% in three-class and two-class classification accuracy, respectively. We are the first to introduce graph-based representation into code readability classification. Our method outperforms state-of-the-art readability models, which suggests that the graph-based code representation method is effective in extracting syntactic and semantic information from source code, and ultimately improves code readability classification.

Tài liệu tham khảo

Alawad DM, Panta M, Zibran MF, et al (2019) An empirical study of the relationships between code readability and software complexity. arXiv:1909.01760 Allamanis M, Barr ET, Sutton C (2014) Learning natural coding conventions. In: Proceedings of the 22nd ACM SIGSOFT international symposium on foundations of software engineering Allamanis M, Brockschmidt M, Khademi M (2018) Learning to represent programs with graphs. arXiv:1711.00740 Alon U, Zilberstein M, Levy O, et al (2018) A general path-based representation for predicting program properties. CoRR abs/1803.09544. http://arxiv.org/abs/1803.09544 Brunner E, Munzel U (2000) The nonparametric behrens-fisher problem: Asymptotic theory and a small-sample approximation. Biom J 42:17–25. https://doi.org/10.1002/(SICI)1521-4036(200001)42:1<17::AID-BIMJ17>3.0.CO;2-U Buse R, Weimer W (2010) Learning a metric for code readability. Softw Eng IEEE Trans 36:546–558. https://doi.org/10.1109/TSE.2009.70 Cao S, Sun X, Bo L et al (2021) Bgnn4vd: Constructing bidirectional graph neural-network for vulnerability detection. Inf Softw Technol 136:106576 Devlin J, Chang MW, Lee K, et al (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 Dinella E, Dai H, Li Z, Naik M, Song L, Wang K (2020) Hoppity: Learning graph transformations to detect and fix bugs in programs. International Conference on Learning Representations. https://iclr.cc/virtual_2020/poster_SJeqs6EFvB.html Dorn J (2012) A general software readability model. MCS Thesis available from (http://www.cs.virginia.edu/weimer/students/dorn-mcs-paper.pdf) 5:11–14 Fakhoury S, Roy D, Hassan SA, et al (2019) Improving source code readability: Theory and practice. In: 2019 IEEE/ACM 27th international conference on program comprehension (ICPC). pp 2–12 Feng Z, Guo D, Tang D, et al (2020) Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:2002.08155 Fernandes P, Allamanis M, Brockschmidt M (2019) Structured neural summarization. In: 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https://openreview.net/forum?id=H1ersoRqtm Gilmer J, Schoenholz SS, Riley PF, et al (2017) Neural message passing for quantum chemistry. arXiv:1704.01212 Hindle A, Barr ET, Su Z, et al (2012) On the naturalness of software. In: 2012 34th International Conference on Software Engineering (ICSE), pp 837–847. https://doi.org/10.1109/ICSE.2012.6227135 Hu Z, Dong Y, Wang K, et al (2020) Heterogeneous graph transformer. In: Huang Y, King I, Liu T, et al (eds) WWW ’20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020. ACM / IW3C2, pp 2704–2710. https://doi.org/10.1145/3366423.3380027 Johnson J, Lubo S, Yedla N, et al (2019) An empirical study assessing source code readability in comprehension. In: 2019 IEEE International conference on software maintenance and evolution (ICSME). pp 513–523 Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 LeClair A, Haque S, Wu LL, et al (2020) Improved code summarization via a graph neural network. In: Proceedings of the 28th international conference on program comprehension Lee T, Lee JB, In H (2013) A study of different coding styles affecting code readability. Int J Softw Eng Appl 7:413–422. https://doi.org/10.14257/ijseia.2013.7.5.36 Ling C, Huang J, Zhang H (2003) Auc: a statistically consistent and more discriminating measure than accuracy. In: Proc 18th Int’l joint conf artificial intelligence (IJCAI) Li Y, Tarlow D, Brockschmidt M, et al (2016) Gated graph sequence neural networks. In: Bengio Y, LeCun Y (eds) 4th international conference on learning representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. http://arxiv.org/abs/1511.05493 Maddison CJ, Tarlow D (2014) Structured generative models of natural source code. ArXiv abs/1401.0514 Mannan UA, Ahmed I, Sarma A (2018) Towards understanding code readability and its impact on design quality. In: Proceedings of the 4th ACM SIGSOFT international workshop on NLP for software engineering. pp 18–21 Ma Y, Wang S, Aggarwal CC, et al (2019) Graph convolutional networks with eigenpooling. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. pp 723–731 Mi Q, Keung J, Xiao Y et al (2018) Improving code readability classification using convolutional neural networks. Inf Softw Technol 104. https://doi.org/10.1016/j.infsof.2018.07.006 Mi Q, Hao Y, Ou L, Ma W (2022a) Towards using visual, semantic and structural features to improve code readability classification. J Syst Softw 193:11. https://doi.org/10.1016/j.jss.2022.111454 Mi Q, Hao Y, Wu M, et al (2022b) An enhanced data augmentation approach to support multi-class code readability classification. In: International conference on software engineering and knowledge engineering Morris C, Ritzert M, Fey M, et al (2019) Weisfeiler and leman go neural: Higher-order graph neural networks. In: Proceedings of the AAAI conference on artificial intelligence. pp 4602–4609 Pantiuchina J, Lanza M, Bavota G (2018) Improving code: The (mis) perception of quality metrics. In: 2018 IEEE international conference on software maintenance and evolution (ICSME). pp 80–91 Piantadosi V, Fierro F, Scalabrino S et al (2020) How does code readability change during software evolution? Empir Softw Eng 25:5374–5412 Posnett D, Hindle A, Devanbu P (2011) A simpler model of software readability. In: Proceedings of the 8th working conference on mining software repositories. pp 73–82 Raychev V, Bielik P, Vechev MT (2016) Probabilistic model for code with decision trees. In: Visser E, Smaragdakis Y (eds) Proceedings of the 2016 ACM SIGPLAN international conference on object-oriented programming, systems, languages, and applications, OOPSLA 2016, part of SPLASH 2016, Amsterdam, The Netherlands, October 30 - November 4, 2016. ACM, pp 731–747. https://doi.org/10.1145/2983990.2984041 Scalabrino S, Linares-Vásquez M, Oliveto R et al (2018) A comprehensive model for code readability. J Softw Evol Process 30. https://doi.org/10.1002/smr.1958 Scalabrino S, Linares-Vasquez M, Poshyvanyk D, et al (2016) Improving code readability models with textual features. In: 2016 IEEE 24th International conference on program comprehension (ICPC), IEEE, pp 1–10 Sedano T (2016) Code readability testing, an empirical study. In: 2016 IEEE 29th International conference on software engineering education and training (CSEET). pp 111–117 Tsitsulin A, Palowitch J, Perozzi B, et al (2020) Graph clustering with graph neural networks. arXiv preprint arXiv:2006.16904 Vagavolu D, Swarna KC, Chimalakonda S (2021) A mocktail of source code representations. In: 2021 36th IEEE/ACM International conference on automated software engineering (ASE). pp 1296–1300 Wang X, Ji H, Shi C, et al (2019) Heterogeneous graph attention network. In: The world wide web conference. pp 2022–2032 Wang W, Li G, Ma B, et al (2020a) Detecting code clones with graph neural network and flow-augmented abstract syntax tree. In: Kontogiannis K, Khomh F, Chatzigeorgiou A, et al (eds) 27th IEEE International Conference on Software Analysis, Evolution and Reengineering, SANER 2020, London, ON, Canada, February 18-21, 2020. IEEE, pp 261–271. https://doi.org/10.1109/SANER48275.2020.9054857 Wang W, Zhang K, Li G, et al (2020b) Learning to represent programs with heterogeneous graphs. CoRR abs/2012.04188. https://arxiv.org/abs/2012.04188 Xia X, Bao L, Lo D et al (2017) Measuring program comprehension: A large-scale field study with professionals. IEEE Trans Softw Eng 44(10):951–976 Xu K, Hu W, Leskovec J, et al (2018a) How powerful are graph neural networks? CoRR abs/1810.00826. http://arxiv.org/abs/1810.00826 Xu K, Li C, Tian Y, et al (2018b) Representation learning on graphs with jumping knowledge networks. arXiv:abs/1806.03536 Yamaguchi F, Golde N, Arp D, et al (2014) Modeling and discovering vulnerabilities with code property graphs. In: 2014 IEEE Symposium on security and privacy, SP 2014, Berkeley, CA, USA, May 18-21, 2014. IEEE Computer Society, pp 590–604, https://doi.org/10.1109/SP.2014.44 Zhang C, Song D, Huang C, et al (2019) Heterogeneous graph neural network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining. pp 793–803 Zhou Y, Liu S, Siow J, Du X, Liu Y (2019) Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Adv Neural Inf Process Syst 915:11