Usage and attribution of Stack Overflow code snippets in GitHub projects
Tóm tắt
Stack Overflow (SO) is the most popular question-and-answer website for software developers, providing a large amount of copyable code snippets. Using those snippets raises maintenance and legal issues. SO’s license (CC BY-SA 3.0) requires attribution, i.e., referencing the original question or answer, and requires derived work to adopt a compatible license. While there is a heated debate on SO’s license model for code snippets and the required attribution, little is known about the extent to which snippets are copied from SO without proper attribution. We present results of a large-scale empirical study analyzing the usage and attribution of non-trivial Java code snippets from SO answers in public GitHub (GH) projects. We followed three different approaches to triangulate an estimate for the ratio of unattributed usages and conducted two online surveys with software developers to complement our results. For the different sets of projects that we analyzed, the ratio of projects containing files with a reference to SO varied between 3.3% and 11.9%. We found that at most 1.8% of all analyzed repositories containing code from SO used the code in a way compatible with CC BY-SA 3.0. Moreover, we estimate that at most a quarter of the copied code snippets from SO are attributed as required. Of the surveyed developers, almost one half admitted copying code from SO without attribution and about two thirds were not aware of the license of SO code snippets and its implications.
Tài liệu tham khảo
citation_journal_title=Inf Softw Technol; citation_title=On code reuse from StackOverflow: An exploratory study on Android apps; citation_author=R Abdalkareem, E Shihab, J Rilling; citation_volume=88; citation_publication_date=2017; citation_pages=148-158; citation_doi=10.1016/j.infsof.2017.04.005; citation_id=CR1
Acar Y, Backes M, Fahl S, Kim D, Mazurek ML, Stransky C (2016) You get where you’re looking for: The impact of information sources on code security. In: Locasto M, Shmatikov V, Erlingsson Ú (eds) 2016 IEEE Symposium on Security and Privacy (S&P 2016), IEEE Computer Society, San Jose, CA, USA, pp 289–305
Achte Z (2016) AZ I-8 O 294/15. Landgericht Bochum
http://www.justiz.nrw.de/nrwe/lgs/bochum/lg_bochum/j2016/I_8_O_294_15_Urteil_20160303.html
citation_title=An introduction to categorical data analysis; citation_publication_date=2007; citation_id=CR4; citation_author=A Agresti; citation_publisher=Wiley
Aioobe (2010) How to convert byte size into human readable format in java?
http://stackoverflow.com/a/3758880
Allamanis M, Sutton C (2013) Why, when, and what: Analyzing Stack Overflow questions by topic, type, and code. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 53–56
citation_journal_title=Empir Softw Eng; citation_title=Investigating whether and how software developers understand open source software licensing; citation_author=DA Almeida, GC Murphy, G Wilson, M Hoye; citation_volume=11; citation_issue=11; citation_publication_date=2018; citation_pages=730; citation_id=CR7
Alsup W (2012) Oracle America, Inc v. Google, Inc. United States District Court for the Northern District of California
An L, Mlouki O, Khomh F, Antoniol G (2017) Stack Overflow: A Code Laundering Platform? In: Pinzger M, Bavota G, Marcus A (eds) 24th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2017) IEEE Computer Society, Klagenfurt, Austria, pp 283–293
Badashian AS, Esteki A, Gholipour A, Hindle A, Stroulia E (2014) Involvement, Contribution and Influence in GitHub and Stack Overflow. In: Ng J, Li J, Wong K (eds) 24th international conference on computer science and software engineering (CASCON 2014), IBM / ACM, Markham, ON, Canada, pp 19–33
Baltes S (2017). sbaltes/api-retriever on GitHub.
https://doi.org/10.5281/zenodo.1049419
Baltes S (2018) Usage and Attribution of Stack Overflow Code Snippets in GitHub Projects — Supplementary Material.
https://doi.org/10.5281/zenodo.1148069
Baltes S, Dumani L (2018) SOTorrent Dataset.
https://doi.org/10.5281/zenodo.1135262
Baltes S, Dumani L, Treude C, Diehl S (2018) SOTOrrent: Reconstructing and analyzing the evolution stack overflow posts. In: Zaidman A, Hill E, Kamei Y (eds) 15th international conference on mining software repositories (MSR 2018) ACM, Gothenburg, Sweden, pp 1–12
citation_journal_title=Information technology Learning, and Performance Journal; citation_title=Organizational research: Determining appropriate sample size in survey research; citation_author=JEII Bartlett, JW Kotrlik, CC Higgins; citation_volume=19; citation_issue=1; citation_publication_date=2001; citation_pages=43-50; citation_id=CR15
Bosu A, Corley CS, Heaton D, Chatterji D, Carver JC, Kraft NA (2013) Building reputation in StackOverflow: An empirical investigation. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 89–92
Brandt J, Dontcheva M, Weskamp M, Klemmer SR (2010) Example-centric programming: Integrating web search into the development environment. In: Mynatt E, Edwards K, Rodden T (eds) 2010 Conference on human factors in computing systems (CHI 2010), ACM, Atlanta, GA, USA, pp 513–522
citation_journal_title=Software—Practice and Experience; citation_title=Efficient plagiarism detection for large code repositories; citation_author=S Burrows, SMM Tahaghoghi, J Zobel; citation_volume=37; citation_issue=2; citation_publication_date=2007; citation_pages=151-176; citation_doi=10.1002/spe.750; citation_id=CR18
Campbell BA, Treude C (2017) NLP2Code: Code snippet content assist via natural language tasks. In: Mei H, Zhang L, Zimmermann T (eds) 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME 2017) IEEE Computer Society, Shanghai, China, pp 628–632
Cavaretta MJ (2015) Open source issues in mergers & acquisitions.
http://www.mbbp.com/news/open-source-issues
citation_title=Sampling techniques; citation_publication_date=1977; citation_id=CR21; citation_author=WG Cochran; citation_publisher=Wiley
Corley JS (2017) Artifex Software Inc v. Hancom, Inc. United States District Court for the Northern District of California
Creative Commons Corporation (2007) Attribution-ShareAlike 3.0 Unported.
https://creativecommons.org/licenses/by-sa/3.0/legalcode
Creative Commons Corporation (2017a) Compatible Licenses.
https://creativecommons.org/share-your-work/licensing-considerations/compatible-licenses/
Creative Commons Corporation (2017b) Frequently Asked Questions.
https://creativecommons.org/faq/#can-i-apply-a-creative-commons-license-to-software
Duijn M, Kucera A, Bacchelli A (2015) Quality questions need quality code: Classifying code fragments on stack overflow. In: Di Penta M, Pinzger M, Robbes R (eds) 12th working conference on mining software repositories (MSR 2015) IEEE Computer Society, Florence, Italy, pp 410–413
Electronic Frontier Foundation (2018) Oracle v. Google.
https://www.eff.org/cases/oracle-v-google
Engelfriet A (2016) What is the license status of StackOverflow code snippets?
https://legalict.com/software/what-is-the-license-status-of-stackoverflow-code-snippets/
Fazunenko D (2016) Get rid of the humanReadableByteCount() method in openjdk/hotspot.
https://bugs.openjdk.java.net/browse/JDK-8170860
Fischer F, Böttinger K, Xiao H, Stransky C, Acar Y, Backes M, Fahl S (2017) Stack overflow considered harmful? the impact of Copy&Paste on android application security. In: Butler KRB, Erlingsson Ú, Parno B (eds) 2017 IEEE Symposium on security and privacy (S&P 2017) IEEE Computer Society, San Jose, CA, USA, pp 121–136
German DM, Hassan AE (2009) License integration patterns: Addressing license mismatches in component-based development. In: Fickas S, Atlee J M, Inverardi P (eds) 31St international conference on software engineering (ICSE 2009) IEEE Computer Society, Vancouver, BC, Canada, pp 188–198
German DM, Di Penta M, Gueheneuc YG, Antoniol G (2009) Code siblings: Technical and legal implications of copying code between applications. In: Godfrey M W, Whitehead J (eds) 6th international working conference on mining software repositories (MSR 2009), IEEE Computer Society, Vancouver, BC, Canada, pp 81–90
Gharehyazie M, Ray B, Filkov V (2017) Some From Here, Some From There: Cross-project Code Reuse in GitHub. In: Gonzalez-Barahona JM, Hindle A, Tan L (eds) 14th international conference on mining software repositories (MSR 2017), IEEE Computer Society, Buenos Aires, Argentina, pp 291–301
GitHub Inc (2017a) Choosealicense.com: No License.
https://choosealicense.com/no-license/
GitHub Inc (2017b) GitHub Developer – API.
https://developer.github.com/v3/
GitHub Inc (2018) The State of the Octoverse 2017.
https://octoverse.github.com/
Google Cloud Platform (2017a) GitHub Data.
https://cloud.google.com/bigquery/public-data/github
Google Cloud Platform (2017b) Stack Overflow Data.
https://cloud.google.com/bigquery/public-data/stackoverflow
Gousios G (2013) The GHTorrent dataset and tool suite. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013), IEEE, San Francisco, CA, USA, pp 233–236
Gousios G (2017) GHTorrent on the Google cloud.
http://ghtorrent.org/gcloud.html
Kaess J, Müller J, Rieger J (2004) Welte v. Sitecom Deutschland GmbH. District Court of Munich I
Kalliamvakou E, Gousios G, Blincoe K, Singer L, Germán DM, Damian D (2014) The promises and perils of mining GitHub. In: Devanbu PT, Kim S, Pinzger M (eds) 11th working conference on mining software repositories (MSR 2014), ACM, Hyderabad, India, pp 92–101
citation_journal_title=Comput Sci Educ; citation_title=A comparison of source code plagiarism detection engines; citation_author=T Lancaster, F Culwin; citation_volume=14; citation_issue=2; citation_publication_date=2004; citation_pages=101-112; citation_doi=10.1080/08993400412331363843; citation_id=CR43
citation_journal_title=Proc ACM Program Lang; citation_title=DéJàvu: A map of code duplicates on GitHub; citation_author=CV Lopes, P Maj, P Martins, V Saini, Yang Di, J Zitny, H Sajnani, J Vitek; citation_volume=1; citation_issue=OOPSLA; citation_publication_date=2017; citation_pages=84:1-84:28; citation_doi=10.1145/3133908; citation_id=CR44
Martins VT, Fonte D, Henriques PR, Dd Cruz (2014) Plagiarism detection: a tool survey and comparison. In: Pereira MJV, Leal JP, Simoes A (eds) 3rd symposium on languages, applications and technologies (SLATE 2014), Schloss Dagstuhl–Leibniz-Zentrum Fuer Informatik, Bragança, Portugal, Openaccess Series In Informatics (OASIcs), vol 38, pp 143–158
Meloca R, Pinto G, Baiser L, Mattos M, Polato I, Wiese IS, German D (2018) Understanding the usage, impact, and adoption of non-OSI approved licenses. In: Zaidman A, Hill E, Kamei Y (eds) 15th international conference on mining software repositories (MSR 2018) ACM, Gothenburg, Sweden, pp 1–11
Morrison P, Murphy-Hill E (2013) Is programming knowledge related to age? An exploration of Stack Overflow. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 69–72
citation_journal_title=Empir Softw Eng; citation_title=Curating GitHub for engineered software projects; citation_author=N Munaiah, S Kroh, C Cabrey, M Nagappan; citation_volume=22; citation_issue=6; citation_publication_date=2017; citation_pages=3219-3253; citation_doi=10.1007/s10664-017-9512-6; citation_id=CR48
Nasehi SM, Sillito J, Maurer F, Burns C (2012) What makes a good code example? A study of programming Q&A in StackOverflow. In: Tonella P, Di Penta M, Maletic JI (eds) 28th IEEE International Conference on Software Maintenance (ICSM 2012) IEEE Computer Society, Trento, Italy, pp 25–34
citation_journal_title=Eur J Soc Psychol; citation_title=Methods of coping with social desirability bias: a review; citation_author=AJ Nederhof; citation_volume=15; citation_issue=3; citation_publication_date=1985; citation_pages=263-280; citation_doi=10.1002/ejsp.2420150303; citation_id=CR50
PMD (2016) Finding duplicated code.
http://pmd.github.io/pmd-5.5.1/usage/cpd-usage.html
Ponzanelli L, Bacchelli A, Lanza M (2013) Seahawk: Stack overflow in the IDEa. In: Notkin D, Cheng BHC, Pohl K (eds) 35th international conference on software engineering (ICSE 2013) IEEE Computer Society, San Francisco, CA, USA, pp 1295–1298
Ponzanelli L, Mocci A, Bacchelli A, Lanza M (2014) Understanding and classifying the quality of technical forum questions. In: Wong WE, McMillin B (eds) 14th international conference on quality software (QSIC 2014) IEEE, Allen, TX, USA, pp 343–352
Poteat H (2016) GitHub’s 2015 Transparency Report.
https://github.com/blog/2202-github-s-2015-transparency-report
citation_journal_title=Journal of Universal Computer Science; citation_title=Finding plagiarisms among a set of programs with JPlag; citation_author=L Prechelt, G Malpohl, M Philippsen; citation_volume=8; citation_issue=11; citation_publication_date=2002; citation_pages=1016-1038; citation_id=CR55
Ragkhitwetsagul C (2016) Measuring code similarity in Large-Scaled code corpora. In: Kraft NA, Menzies T, Adams B, Poshyvanyk D (eds) 2016 IEEE International conference on software maintenance and evolution (ICSME 2016) IEEE Computer Society, Raleigh, NC, USA, pp 626–630
citation_journal_title=Sci Comput Program; citation_title=Comparison and evaluation of code clone detection techniques and tools: a qualitative approach; citation_author=CK Roy, JR Cordy, R Koschke; citation_volume=74; citation_issue=7; citation_publication_date=2009; citation_pages=470-495; citation_doi=10.1016/j.scico.2009.02.007; citation_id=CR57
Sajnani H, Saini V, Svajlenko J, Roy CK, Lopes CV (2016) SourcererCC: Scaling code clone detection to big-code. In: Dillon L, Visser W, Williams L (eds) 38th international conference on software engineering (ICSE 2016), ACM, Austin, TX, USA, pp 1157–1168
Scalabrino S, Bavota G, Vendome C, Linares-Vásquez M, Poshyvany D, Oliveto R (2017) Automatically assessing code understandability: How far are we? In: Rosu G, Penta MD, Nguyen TN (eds) 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE 2017) IEEE Computer Society, Urbana, IL, USA, pp 417–427
Schenk D, Lungu M (2013) Geo-locating the knowledge transfer in StackOverflow. In: Ali R, Begel A, Maalej W (eds) 2013 International workshop on social software engineering (SSE 2013) ACM, Saint Petersburg, Russian Federation, pp 21–24
Silvestri G, Yang J, Bozzon A, Tagarelli A (2015) Linking Accounts across Social Networks: The Case of StackOverflow, GitHub and Twitter. In: Armano G, Bozzon A, Giuliani A (eds) 1st International Workshop on Knowledge Discovery on the WEB (KDWeb 2015) CEUR-WS.org, Cagliari, Italy, CEUR Workshop Proceedings, pp 41–52
Software Freedom Law Center (2008) Free Software Foundation Inc v. Cisco Systems, Inc. United States District Court for the Southern District of New York
citation_journal_title=Commun ACM; citation_title=License risks from ad hoc reuse of code from the internet; citation_author=M Sojer, J Henkel; citation_volume=54; citation_issue=12; citation_publication_date=2011; citation_pages=74-81; citation_doi=10.1145/2043174.2043193; citation_id=CR63
citation_title=Understanding Open Source and Free Software Licensing; citation_publication_date=2004; citation_id=CR64; citation_author=AM St Laurent; citation_publisher=O’Reilly Media
Stack Exchange Inc (2015) Stack Exchange Data Dump: August 18, 2015.
https://archive.org/details/stackexchange/
Stack Exchange Inc (2016) Stack Exchange API v2.2.
https://api.stackexchange.com/docs
Stack Exchange Inc (2017a) Stack Exchange Data Dump 2017-12-01.
https://archive.org/details/stackexchange/
Stack Exchange Inc (2017b) Stack Exchange Data Dump: March 14, 2017.
https://archive.org/details/stackexchange/
Stack Exchange Inc (2018a) Stack exchange network terms of service.
https://web.archive.org/web/20180228075555/http://stackexchange.com/legal
Stack Exchange Inc (2018b) Stack exchange network terms of service.
http://stackexchange.com/legal
Stack Exchange Meta (2009) What is up with the source code license on Stack Overflow?
http://meta.stackexchange.com/q/25956
Stack Exchange Meta (2013) Do I have to worry about copyright issues for code posted on Stack Overflow?
http://meta.stackexchange.com/q/12527
Stack Exchange Meta (2015) Can we get some explicit clarification on the *intended* legal usage of code from SO answers?
http://meta.stackoverflow.com/q/286582
Stack Exchange Meta (2016) A new code license: The MIT, this time with attribution required.
http://meta.stackexchange.com/q/272956
Subramanian S, Holmes R (2013) Making sense of online code snippets. In: Zimmermann T, Di Penta M, Kim S (eds) 10th international working conference on mining software repositories (MSR 2013) IEEE, San Francisco, CA, USA, pp 85–88
Tim Post (2018) A new (2018) update to our terms of service is here.
https://meta.stackexchange.com/questions/309746/a-new-2018-update-to-our-terms-of-service-is-here
TIOBE software BV (2017) TIOBE Index for February 2017.
http://www.tiobe.com/tiobe-index/
Treude C, Robillard MP (2016) Augmenting API documentation with insights from stack overflow. In: Dillon L, Visser W, Williams L (eds) 38th international conference on software engineering (ICSE 2016), ACM, Austin, TX, USA, pp 392–403
Treude C, Robillard MP (2017) Understanding stack overflow code fragments. In: Mei H, Zhang L, Zimmermann T (eds) 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME 2017) IEEE Computer Society, Shanghai, China, pp 509–513
Treude C, Barzilay O, Storey MAD (2011) How do programmers ask and answer questions on the web? In: Taylor RN, Gall HC, Medvidovic N (eds) 33rd international conference on software engineering (ICSE 2011) ACM, Waikiki, Honolulu, pp 804–807
Vasilescu B, Capiluppi A, Serebrenik A (2012) Gender, representation and online participation: A quantitative study of stackoverflow. In: Aberer K, Flache A, Jager W, Liu L, Tang J, Gueret C (eds) 4th International Conference on Social Informatics (SocInfo 2012) Springer, Lausanne, Switzerland, Lecture Notes in Computer Science, pp 332–338
Vasilescu B, Filkov V, Serebrenik A (2013) StackOverflow and GitHub: Associations between Software Development and Crowdsourced Knowledge. In: Chang LW, Srivastava J, Zhan J (eds) 2013 International conference on social computing (SocialCom 2013) IEEE Computer Society, Washington, DC, USA, pp 188–195
Vendome C (2015) A large scale study of license usage on GitHub. In: Bertolino A, Canfora G, Elbaum S (eds) 37th international conference on software engineering (ICSE 2015) IEEE, Florence, Italy, pp 772–774
Wang S, Lo D, Jiang L (2013) An empirical study on developer interactions in StackOverflow. In: Shin SY, Maldonado JC (eds) 28th annual ACM symposium on applied computing (SAC 2013) ACM, Coimbra, Portugal, pp 1019–1024
White JS (2008) Jacobsen v. Katzer, 535 F.3d 1373, 1379. United States Court of Appeals for the Federal Circuit
Wikipedia (2017) Free Software Foundation, Inc v. Cisco Systems, Inc.
https://en.wikipedia.org/wiki/Free_Software_Foundation,_Inc._v._Cisco_Systems,_Inc.
citation_journal_title=Empir Softw Eng; citation_title=What do developers search for on the web?; citation_author=X Xia, L Bao, D Lo, PS Kochhar, AE Hassan, Z Xing; citation_volume=22; citation_issue=6; citation_publication_date=2017; citation_pages=3149-3185; citation_doi=10.1007/s10664-017-9514-4; citation_id=CR87
Yang D, Hussain A, Lopes CV (2016) From query to usable code: an analysis of stack overflow code snippets. In: Kim M, Robbes R, Bird C (eds) 13th international conference on mining software repositories (MSR 2016), ACM, Austin, TX, USA, pp 391–402
Yang D, Martins P, Saini V, Lopes CV (2017) Stack Overflow in GitHub: Any snippets there? In: Gonzalez-Barahona JM, Hindle A, Tan L (eds) 14th international conference on mining software repositories (MSR 2017) IEEE Computer Society, Buenos Aires, Argentina, pp 280–290
Yang J, Hauff C, Bozzon A, Houben GJ (2014) Asking the right question in collaborative Q&A systems. In: Ferres L, Rossi G, Almeida VAF, Herder E (eds) 25th ACM conference on hypertext and social media (HT 2014) ACM, Santiago, Chile, pp 179–189
Zagalsky A, Barzilay O, Yehudai A (2012) Example Overflow: Using social media for code recommendation. In: Maalej W, Robillard MP, Walker RJ, Zimmermann T (eds) 3rd international workshop on recommendation systems for software engineering (RSSE 2012) IEEE, Zurich, Switzerland, pp 38–42
Zhang T, Upadhyaya G, Reinhardt A, Rajan H, Kim M (2018) Are Code Examples on an Online Q&A Forum Reliable? A Study of API Misuse on Stack Overflow. In: Crnkovic I, Chechik M, Harman M (eds) 40th international conference on software engineering (ICSE 2018) ACM, Gothenburg, Sweden, pp 1–11