Usage and attribution of Stack Overflow code snippets in GitHub projects

Empirical Software Engineering - Tập 24 Số 3 - Trang 1259-1295 - 2019
Baltes, Sebastian1, Diehl, Stephan1
1University of Trier, Trier, Germany

Tóm tắt

Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of copyable code snippets. Using those snippets raises maintenance and legal issues. SO’s license (CC BY-SA 3.0) requires attribution, i.e., referencing the original question or answer, and requires derived work to adopt a compatible license. While there is a heated debate on SO’s license model for code snippets and the required attribution, little is known about the extent to which snippets are copied from SO without proper attribution. We present results of a large-scale empirical study analyzing the usage and attribution of non-trivial Java code snippets from SO answers in public GitHub (GH) projects. We followed three different approaches to triangulate an estimate for the ratio of unattributed usages and conducted two online surveys with software developers to complement our results. For the different sets of projects that we analyzed, the ratio of projects containing files with a reference to SO varied between 3.3% and 11.9%. We found that at most 1.8% of all analyzed repositories containing code from SO used the code in a way compatible with CC BY-SA 3.0. Moreover, we estimate that at most a quarter of the copied code snippets from SO are attributed as required. Of the surveyed developers, almost one half admitted copying code from SO without attribution and about two thirds were not aware of the license of SO code snippets and its implications.

Tài liệu tham khảo

citation_journal_title=Inf Softw Technol; citation_title=On code reuse from StackOverflow: An exploratory study on Android apps; citation_author=R Abdalkareem, E Shihab, J Rilling; citation_volume=88; citation_publication_date=2017; citation_pages=148-158; citation_doi=10.1016/j.infsof.2017.04.005; citation_id=CR1 Acar Y, Backes M, Fahl S, Kim D, Mazurek ML, Stransky C (2016) You get where you’re looking for: The impact of information sources on code security. In: Locasto M, Shmatikov V, Erlingsson Ú (eds) 2016 IEEE Symposium on Security and Privacy (S&P 2016), IEEE Computer Society, San Jose, CA, USA, pp 289–305 Achte Z (2016) AZ I-8 O 294/15. Landgericht Bochum http://www.justiz.nrw.de/nrwe/lgs/bochum/lg_bochum/j2016/I_8_O_294_15_Urteil_20160303.html citation_title=An introduction to categorical data analysis; citation_publication_date=2007; citation_id=CR4; citation_author=A Agresti; citation_publisher=Wiley Aioobe (2010) How to convert byte size into human readable format in java? http://stackoverflow.com/a/3758880 Allamanis M, Sutton C (2013) Why, when, and what: Analyzing Stack Overflow questions by topic, type, and code. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 53–56 citation_journal_title=Empir Softw Eng; citation_title=Investigating whether and how software developers understand open source software licensing; citation_author=DA Almeida, GC Murphy, G Wilson, M Hoye; citation_volume=11; citation_issue=11; citation_publication_date=2018; citation_pages=730; citation_id=CR7 Alsup W (2012) Oracle America, Inc v. Google, Inc. United States District Court for the Northern District of California An L, Mlouki O, Khomh F, Antoniol G (2017) Stack Overflow: A Code Laundering Platform? In: Pinzger M, Bavota G, Marcus A (eds) 24th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2017) IEEE Computer Society, Klagenfurt, Austria, pp 283–293 Badashian AS, Esteki A, Gholipour A, Hindle A, Stroulia E (2014) Involvement, Contribution and Influence in GitHub and Stack Overflow. In: Ng J, Li J, Wong K (eds) 24th international conference on computer science and software engineering (CASCON 2014), IBM / ACM, Markham, ON, Canada, pp 19–33 Baltes S (2017). sbaltes/api-retriever on GitHub. https://doi.org/10.5281/zenodo.1049419 Baltes S (2018) Usage and Attribution of Stack Overflow Code Snippets in GitHub Projects — Supplementary Material. https://doi.org/10.5281/zenodo.1148069 Baltes S, Dumani L (2018) SOTorrent Dataset. https://doi.org/10.5281/zenodo.1135262 Baltes S, Dumani L, Treude C, Diehl S (2018) SOTOrrent: Reconstructing and analyzing the evolution stack overflow posts. In: Zaidman A, Hill E, Kamei Y (eds) 15th international conference on mining software repositories (MSR 2018) ACM, Gothenburg, Sweden, pp 1–12 citation_journal_title=Information technology Learning, and Performance Journal; citation_title=Organizational research: Determining appropriate sample size in survey research; citation_author=JEII Bartlett, JW Kotrlik, CC Higgins; citation_volume=19; citation_issue=1; citation_publication_date=2001; citation_pages=43-50; citation_id=CR15 Bosu A, Corley CS, Heaton D, Chatterji D, Carver JC, Kraft NA (2013) Building reputation in StackOverflow: An empirical investigation. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 89–92 Brandt J, Dontcheva M, Weskamp M, Klemmer SR (2010) Example-centric programming: Integrating web search into the development environment. In: Mynatt E, Edwards K, Rodden T (eds) 2010 Conference on human factors in computing systems (CHI 2010), ACM, Atlanta, GA, USA, pp 513–522 citation_journal_title=Software—Practice and Experience; citation_title=Efficient plagiarism detection for large code repositories; citation_author=S Burrows, SMM Tahaghoghi, J Zobel; citation_volume=37; citation_issue=2; citation_publication_date=2007; citation_pages=151-176; citation_doi=10.1002/spe.750; citation_id=CR18 Campbell BA, Treude C (2017) NLP2Code: Code snippet content assist via natural language tasks. In: Mei H, Zhang L, Zimmermann T (eds) 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME 2017) IEEE Computer Society, Shanghai, China, pp 628–632 Cavaretta MJ (2015) Open source issues in mergers & acquisitions. http://www.mbbp.com/news/open-source-issues citation_title=Sampling techniques; citation_publication_date=1977; citation_id=CR21; citation_author=WG Cochran; citation_publisher=Wiley Corley JS (2017) Artifex Software Inc v. Hancom, Inc. United States District Court for the Northern District of California Creative Commons Corporation (2007) Attribution-ShareAlike 3.0 Unported. https://creativecommons.org/licenses/by-sa/3.0/legalcode Creative Commons Corporation (2017a) Compatible Licenses. https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/ Creative Commons Corporation (2017b) Frequently Asked Questions. https://creativecommons.org/faq/#can-i-apply-a-creative-commons-license-to-software Duijn M, Kucera A, Bacchelli A (2015) Quality questions need quality code: Classifying code fragments on stack overflow. In: Di Penta M, Pinzger M, Robbes R (eds) 12th working conference on mining software repositories (MSR 2015) IEEE Computer Society, Florence, Italy, pp 410–413 Electronic Frontier Foundation (2018) Oracle v. Google. https://www.eff.org/cases/oracle-v-google Engelfriet A (2016) What is the license status of StackOverflow code snippets? https://legalict.com/software/what-is-the-license-status-of-stackoverflow-code-snippets/ Fazunenko D (2016) Get rid of the humanReadableByteCount() method in openjdk/hotspot. https://bugs.openjdk.java.net/browse/JDK-8170860 Fischer F, Böttinger K, Xiao H, Stransky C, Acar Y, Backes M, Fahl S (2017) Stack overflow considered harmful? the impact of Copy&Paste on android application security. In: Butler KRB, Erlingsson Ú, Parno B (eds) 2017 IEEE Symposium on security and privacy (S&P 2017) IEEE Computer Society, San Jose, CA, USA, pp 121–136 German DM, Hassan AE (2009) License integration patterns: Addressing license mismatches in component-based development. In: Fickas S, Atlee J M, Inverardi P (eds) 31St international conference on software engineering (ICSE 2009) IEEE Computer Society, Vancouver, BC, Canada, pp 188–198 German DM, Di Penta M, Gueheneuc YG, Antoniol G (2009) Code siblings: Technical and legal implications of copying code between applications. In: Godfrey M W, Whitehead J (eds) 6th international working conference on mining software repositories (MSR 2009), IEEE Computer Society, Vancouver, BC, Canada, pp 81–90 Gharehyazie M, Ray B, Filkov V (2017) Some From Here, Some From There: Cross-project Code Reuse in GitHub. In: Gonzalez-Barahona JM, Hindle A, Tan L (eds) 14th international conference on mining software repositories (MSR 2017), IEEE Computer Society, Buenos Aires, Argentina, pp 291–301 GitHub Inc (2017a) Choosealicense.com: No License. https://choosealicense.com/no-license/ GitHub Inc (2017b) GitHub Developer – API. https://developer.github.com/v3/ GitHub Inc (2018) The State of the Octoverse 2017. https://octoverse.github.com/ Google Cloud Platform (2017a) GitHub Data. https://cloud.google.com/bigquery/public-data/github Google Cloud Platform (2017b) Stack Overflow Data. https://cloud.google.com/bigquery/public-data/stackoverflow Gousios G (2013) The GHTorrent dataset and tool suite. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013), IEEE, San Francisco, CA, USA, pp 233–236 Gousios G (2017) GHTorrent on the Google cloud. http://ghtorrent.org/gcloud.html Kaess J, Müller J, Rieger J (2004) Welte v. Sitecom Deutschland GmbH. District Court of Munich I Kalliamvakou E, Gousios G, Blincoe K, Singer L, Germán DM, Damian D (2014) The promises and perils of mining GitHub. In: Devanbu PT, Kim S, Pinzger M (eds) 11th working conference on mining software repositories (MSR 2014), ACM, Hyderabad, India, pp 92–101 citation_journal_title=Comput Sci Educ; citation_title=A comparison of source code plagiarism detection engines; citation_author=T Lancaster, F Culwin; citation_volume=14; citation_issue=2; citation_publication_date=2004; citation_pages=101-112; citation_doi=10.1080/08993400412331363843; citation_id=CR43 citation_journal_title=Proc ACM Program Lang; citation_title=DéJàvu: A map of code duplicates on GitHub; citation_author=CV Lopes, P Maj, P Martins, V Saini, Yang Di, J Zitny, H Sajnani, J Vitek; citation_volume=1; citation_issue=OOPSLA; citation_publication_date=2017; citation_pages=84:1-84:28; citation_doi=10.1145/3133908; citation_id=CR44 Martins VT, Fonte D, Henriques PR, Dd Cruz (2014) Plagiarism detection: a tool survey and comparison. In: Pereira MJV, Leal JP, Simoes A (eds) 3rd symposium on languages, applications and technologies (SLATE 2014), Schloss Dagstuhl–Leibniz-Zentrum Fuer Informatik, Bragança, Portugal, Openaccess Series In Informatics (OASIcs), vol 38, pp 143–158 Meloca R, Pinto G, Baiser L, Mattos M, Polato I, Wiese IS, German D (2018) Understanding the usage, impact, and adoption of non-OSI approved licenses. In: Zaidman A, Hill E, Kamei Y (eds) 15th international conference on mining software repositories (MSR 2018) ACM, Gothenburg, Sweden, pp 1–11 Morrison P, Murphy-Hill E (2013) Is programming knowledge related to age? An exploration of Stack Overflow. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 69–72 citation_journal_title=Empir Softw Eng; citation_title=Curating GitHub for engineered software projects; citation_author=N Munaiah, S Kroh, C Cabrey, M Nagappan; citation_volume=22; citation_issue=6; citation_publication_date=2017; citation_pages=3219-3253; citation_doi=10.1007/s10664-017-9512-6; citation_id=CR48 Nasehi SM, Sillito J, Maurer F, Burns C (2012) What makes a good code example? A study of programming Q&A in StackOverflow. In: Tonella P, Di Penta M, Maletic JI (eds) 28th IEEE International Conference on Software Maintenance (ICSM 2012) IEEE Computer Society, Trento, Italy, pp 25–34 citation_journal_title=Eur J Soc Psychol; citation_title=Methods of coping with social desirability bias: a review; citation_author=AJ Nederhof; citation_volume=15; citation_issue=3; citation_publication_date=1985; citation_pages=263-280; citation_doi=10.1002/ejsp.2420150303; citation_id=CR50 PMD (2016) Finding duplicated code. http://pmd.github.io/pmd-5.5.1/usage/cpd-usage.html Ponzanelli L, Bacchelli A, Lanza M (2013) Seahawk: Stack overflow in the IDEa. In: Notkin D, Cheng BHC, Pohl K (eds) 35th international conference on software engineering (ICSE 2013) IEEE Computer Society, San Francisco, CA, USA, pp 1295–1298 Ponzanelli L, Mocci A, Bacchelli A, Lanza M (2014) Understanding and classifying the quality of technical forum questions. In: Wong WE, McMillin B (eds) 14th international conference on quality software (QSIC 2014) IEEE, Allen, TX, USA, pp 343–352 Poteat H (2016) GitHub’s 2015 Transparency Report. https://github.com/blog/2202-github-s-2015-transparency-report citation_journal_title=Journal of Universal Computer Science; citation_title=Finding plagiarisms among a set of programs with JPlag; citation_author=L Prechelt, G Malpohl, M Philippsen; citation_volume=8; citation_issue=11; citation_publication_date=2002; citation_pages=1016-1038; citation_id=CR55 Ragkhitwetsagul C (2016) Measuring code similarity in Large-Scaled code corpora. In: Kraft NA, Menzies T, Adams B, Poshyvanyk D (eds) 2016 IEEE International conference on software maintenance and evolution (ICSME 2016) IEEE Computer Society, Raleigh, NC, USA, pp 626–630 citation_journal_title=Sci Comput Program; citation_title=Comparison and evaluation of code clone detection techniques and tools: a qualitative approach; citation_author=CK Roy, JR Cordy, R Koschke; citation_volume=74; citation_issue=7; citation_publication_date=2009; citation_pages=470-495; citation_doi=10.1016/j.scico.2009.02.007; citation_id=CR57 Sajnani H, Saini V, Svajlenko J, Roy CK, Lopes CV (2016) SourcererCC: Scaling code clone detection to big-code. In: Dillon L, Visser W, Williams L (eds) 38th international conference on software engineering (ICSE 2016), ACM, Austin, TX, USA, pp 1157–1168 Scalabrino S, Bavota G, Vendome C, Linares-Vásquez M, Poshyvany D, Oliveto R (2017) Automatically assessing code understandability: How far are we? In: Rosu G, Penta MD, Nguyen TN (eds) 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2017) IEEE Computer Society, Urbana, IL, USA, pp 417–427 Schenk D, Lungu M (2013) Geo-locating the knowledge transfer in StackOverflow. In: Ali R, Begel A, Maalej W (eds) 2013 International workshop on social software engineering (SSE 2013) ACM, Saint Petersburg, Russian Federation, pp 21–24 Silvestri G, Yang J, Bozzon A, Tagarelli A (2015) Linking Accounts across Social Networks: The Case of StackOverflow, GitHub and Twitter. In: Armano G, Bozzon A, Giuliani A (eds) 1st International Workshop on Knowledge Discovery on the WEB (KDWeb 2015) CEUR-WS.org, Cagliari, Italy, CEUR Workshop Proceedings, pp 41–52 Software Freedom Law Center (2008) Free Software Foundation Inc v. Cisco Systems, Inc. United States District Court for the Southern District of New York citation_journal_title=Commun ACM; citation_title=License risks from ad hoc reuse of code from the internet; citation_author=M Sojer, J Henkel; citation_volume=54; citation_issue=12; citation_publication_date=2011; citation_pages=74-81; citation_doi=10.1145/2043174.2043193; citation_id=CR63 citation_title=Understanding Open Source and Free Software Licensing; citation_publication_date=2004; citation_id=CR64; citation_author=AM St Laurent; citation_publisher=O’Reilly Media Stack Exchange Inc (2015) Stack Exchange Data Dump: August 18, 2015. https://archive.org/details/stackexchange/ Stack Exchange Inc (2016) Stack Exchange API v2.2. https://api.stackexchange.com/docs Stack Exchange Inc (2017a) Stack Exchange Data Dump 2017-12-01. https://archive.org/details/stackexchange/ Stack Exchange Inc (2017b) Stack Exchange Data Dump: March 14, 2017. https://archive.org/details/stackexchange/ Stack Exchange Inc (2018a) Stack exchange network terms of service. https://web.archive.org/web/20180228075555/http://stackexchange.com/legal Stack Exchange Inc (2018b) Stack exchange network terms of service. http://stackexchange.com/legal Stack Exchange Meta (2009) What is up with the source code license on Stack Overflow? http://meta.stackexchange.com/q/25956 Stack Exchange Meta (2013) Do I have to worry about copyright issues for code posted on Stack Overflow? http://meta.stackexchange.com/q/12527 Stack Exchange Meta (2015) Can we get some explicit clarification on the *intended* legal usage of code from SO answers? http://meta.stackoverflow.com/q/286582 Stack Exchange Meta (2016) A new code license: The MIT, this time with attribution required. http://meta.stackexchange.com/q/272956 Subramanian S, Holmes R (2013) Making sense of online code snippets. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 85–88 Tim Post (2018) A new (2018) update to our terms of service is here. https://meta.stackexchange.com/questions/309746/a-new-2018-update-to-our-terms-of-service-is-here TIOBE software BV (2017) TIOBE Index for February 2017. http://www.tiobe.com/tiobe-index/ Treude C, Robillard MP (2016) Augmenting API documentation with insights from stack overflow. In: Dillon L, Visser W, Williams L (eds) 38th international conference on software engineering (ICSE 2016), ACM, Austin, TX, USA, pp 392–403 Treude C, Robillard MP (2017) Understanding stack overflow code fragments. In: Mei H, Zhang L, Zimmermann T (eds) 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME 2017) IEEE Computer Society, Shanghai, China, pp 509–513 Treude C, Barzilay O, Storey MAD (2011) How do programmers ask and answer questions on the web? In: Taylor RN, Gall HC, Medvidovic N (eds) 33rd international conference on software engineering (ICSE 2011) ACM, Waikiki, Honolulu, pp 804–807 Vasilescu B, Capiluppi A, Serebrenik A (2012) Gender, representation and online participation: A quantitative study of stackoverflow. In: Aberer K, Flache A, Jager W, Liu L, Tang J, Gueret C (eds) 4th International Conference on Social Informatics (SocInfo 2012) Springer, Lausanne, Switzerland, Lecture Notes in Computer Science, pp 332–338 Vasilescu B, Filkov V, Serebrenik A (2013) StackOverflow and GitHub: Associations between Software Development and Crowdsourced Knowledge. In: Chang LW, Srivastava J, Zhan J (eds) 2013 International conference on social computing (SocialCom 2013) IEEE Computer Society, Washington, DC, USA, pp 188–195 Vendome C (2015) A large scale study of license usage on GitHub. In: Bertolino A, Canfora G, Elbaum S (eds) 37th international conference on software engineering (ICSE 2015) IEEE, Florence, Italy, pp 772–774 Wang S, Lo D, Jiang L (2013) An empirical study on developer interactions in StackOverflow. In: Shin SY, Maldonado JC (eds) 28th annual ACM symposium on applied computing (SAC 2013) ACM, Coimbra, Portugal, pp 1019–1024 White JS (2008) Jacobsen v. Katzer, 535 F.3d 1373, 1379. United States Court of Appeals for the Federal Circuit Wikipedia (2017) Free Software Foundation, Inc v. Cisco Systems, Inc. https://en.wikipedia.org/wiki/Free_Software_Foundation,_Inc._v._Cisco_Systems,_Inc. citation_journal_title=Empir Softw Eng; citation_title=What do developers search for on the web?; citation_author=X Xia, L Bao, D Lo, PS Kochhar, AE Hassan, Z Xing; citation_volume=22; citation_issue=6; citation_publication_date=2017; citation_pages=3149-3185; citation_doi=10.1007/s10664-017-9514-4; citation_id=CR87 Yang D, Hussain A, Lopes CV (2016) From query to usable code: an analysis of stack overflow code snippets. In: Kim M, Robbes R, Bird C (eds) 13th international conference on mining software repositories (MSR 2016), ACM, Austin, TX, USA, pp 391–402 Yang D, Martins P, Saini V, Lopes CV (2017) Stack Overflow in GitHub: Any snippets there? In: Gonzalez-Barahona JM, Hindle A, Tan L (eds) 14th international conference on mining software repositories (MSR 2017) IEEE Computer Society, Buenos Aires, Argentina, pp 280–290 Yang J, Hauff C, Bozzon A, Houben GJ (2014) Asking the right question in collaborative Q&A systems. In: Ferres L, Rossi G, Almeida VAF, Herder E (eds) 25th ACM conference on hypertext and social media (HT 2014) ACM, Santiago, Chile, pp 179–189 Zagalsky A, Barzilay O, Yehudai A (2012) Example Overflow: Using social media for code recommendation. In: Maalej W, Robillard MP, Walker RJ, Zimmermann T (eds) 3rd international workshop on recommendation systems for software engineering (RSSE 2012) IEEE, Zurich, Switzerland, pp 38–42 Zhang T, Upadhyaya G, Reinhardt A, Rajan H, Kim M (2018) Are Code Examples on an Online Q&A Forum Reliable? A Study of API Misuse on Stack Overflow. In: Crnkovic I, Chechik M, Harman M (eds) 40th international conference on software engineering (ICSE 2018) ACM, Gothenburg, Sweden, pp 1–11