Hourly Wages in Crowdworking: A Meta-Analysis
Tóm tắt
In the past decade, crowdworking on online labor market platforms has become an important source of income for a growing number of people worldwide. This development has led to increasing political and scholarly interest in the wages people can earn on such platforms. This study extends the literature, which is often based on a single platform, region, or category of crowdworking, through a meta-analysis of prevalent hourly wages. After a systematic literature search, the paper considers 22 primary empirical studies, including 105 wages and 76,765 data points from 22 platforms, eight different countries, and 10 years. It is found that, on average, microtasks results in an hourly wage of less than $6. This wage is significantly lower than the mean wage of online freelancers, which is roughly three times higher when not factoring in unpaid work. Hourly wages accounting for unpaid work, such as searching for tasks and communicating with requesters, tend to be significantly lower than wages not considering unpaid work. Legislators and researchers evaluating wages in crowdworking need to be aware of this bias when assessing hourly wages, given that the majority of literature does not account for the effect of unpaid work time on crowdworking wages. To foster the comparability of different research results, the article suggests that scholars consider a wage correction factor to account for unpaid work. Finally, researchers should be aware that remuneration and work processes on crowdworking platforms can systematically affect the data collection method and inclusion of unpaid work.
Tài liệu tham khảo
Adams A, Berg J (2017) When home affects pay: an analysis of the gender pay gap among crowdworkers. SSRN Electron J. https://doi.org/10.2139/ssrn.3048711
Agrawal A, Horton J, Lacetera N, Lyons E (2015) Digitization and the contract labor market: a research agenda. In: Goldfarb A et al (eds) Economic analysis of the digital economy. National Bureau of Economic Research conference report. University of Chicago Press, Chicago
Ahmed I, Sutton AJ, Riley RD (2012) Assessment of publication bias, selection bias, and unavailable data in meta-analyses using individual participant data: a database survey. BMJ (clin Res Edn). https://doi.org/10.1136/bmj.d7762
Aleksynska M, Bastrakova A, Kharchenko N (2019) Working conditions on digital labour platforms: evidence from a leading labour supply economy. IZA Discuss Pap. https://doi.org/10.2139/ssrn.3390235
Aloisi A (2015) Commoditized workers the rising of on-demand work, a case study research on a set of online platforms and apps. SSRN Electron J. https://doi.org/10.2139/ssrn.2637485
Auer EM, Behrend TS, Collmus AB, Landers RN, Miles AF (2021) Pay for performance, satisfaction and retention in longitudinal crowdsourced research. PLoS One 16(1):e0245460. https://doi.org/10.1371/journal.pone.0245460
Barzilay AR, Ben-David A (2017) Platform inequality: gender in the gig-economy. Seton Hall Law Rev. https://doi.org/10.2139/ssrn.2995906
Bayudan-Dacuycuy C, Kryz Baje L (2021) Decent work in crowdwork: gendered takeaways from an online survey in the Philippines. PDIS: Discussion Paper Series 2021–11. https://pidswebs.pids.gov.ph/CDN/PUBLICATIONS/pidsdps2111.pdf. Accessed 2 May 2021
Beerepoot N, Lambregts B (2015) Competition in online job marketplaces: towards a global labour market for outsourcing services? Glob Netw 15(2):236–255. https://doi.org/10.1111/glob.12051
Berg J (2016) Income security in the on-demand economy: findings and policy lessons from a survey of crowdworkers. Comp Labor Law Policy J 37(3):506–543
Berg J., Rani U (2021) Working conditions, geography and gender in global crowdwork. In: Work and labour relations in global platform capitalism. Elgar. https://www.elgaronline.com/view/edcoll/9781802205121/9781802205121.00013.xml. Accessed 1 Apr 2022
Berg J, Furrer M, Harmon E, Rani U, Silbermann MS (2018) Digital labour platforms and the future of work, towards decent work in the online world. https://www.ilo.org/global/publications/books/WCMS_645337/lang--en/index.htm. Accessed 3 May 2022
Bertschek I, Ohnemus J, Viete S (2016) Forschungsbericht 462: Befragung zum sozioökonomischen Hintergrund und zu den Motiven von Crowdworkern: Endbericht. https://www.ssoar.info/ssoar/handle/document/47092. Accessed 2 Dec 2021
Bijmolt TH, Pieters RG (2001) Meta-analysis in marketing when studies contain multiple measurements. Mark Lett 12(2):157–169. https://doi.org/10.1023/A:1011117103381
Blanchflower D, Costa R, Machin S (2017) The return of falling real wages. Centre for Economic Performance, LSE. https://cep.lse.ac.uk/pubs/download/rwu006.pdf. Accessed 3 May 2022
Boudreau KJ, Lakhani KR (2013) Using the crowd as an innovation partner. Harv Bus Rev 91(4). https://hbr.org/2013/04/using-the-crowd-as-an-innovation-partner. Accessed 21 Sep 2021
Boudreau KJ, Lakhani KR (2016) Innovation experiments: researching technical advance, knowledge production, and the design of supporting institutions. Innov Policy Econ 16:135–167. https://doi.org/10.1086/684988
Bozarth JD, Roberts RR (1972) Signifying significant significance. Am Psychol 27(8):774–775. https://doi.org/10.1037/h0038034
Braesemann F, Stephany F, Teutloff O, Kässi O, Graham M, Lehdonvirta V (2021) The polarisation of remote work. SSRN Electron J. https://doi.org/10.2139/ssrn.3919655
Brandt A, Polom L, Danneberg M (2016) Gute digitale Arbeit: Auswirkungen der Digitalisierung im Dienstleistungsbereich: Studie im Auftrag der Vereinten Dienstleistungsgewerkschaft (ver.di) Landesbezirk Niedersachsen-Bremen und der Friedrich-Ebert-Stiftung. http://library.fes.de/pdf-files/wiso/12786.pdf. Accessed 9 Oct 2021
Brawley AM, Pury CL (2016) Work experiences on MTurk: job satisfaction, turnover, and information sharing. Comput Hum Behav 54:531–546. https://doi.org/10.1016/j.chb.2015.08.031
Brutus S, Aguinis H, Wassmer U (2013) Self-reported limitations and future directions in scholarly reports. J Manag 39(1):48–75. https://doi.org/10.1177/0149206312455245
Callison-Burch C (2014) Crowd-workers: aggregating information across turkers to help them find higher paying work. In: Proceedings of the AAAI conference on human computation and crowdsourcing, vol 2, pp 8–9. https://www.cis.upenn.edu/~ccb/publications/crowd-workers.pdf. Accessed 17 Oct 2021
Caro LP, O’Higgins N, Berg J (2021) Young people and the gig economy. In: Chacalatana J, Dasgupta S (eds) Is the future ready for youth? International Labour Organization, Geneva, pp 38–52
Casilli A (2016) Is there a global digital labor culture? Marginalization of work, global inequalities, and coloniality. In: 2nd symposium of the project for advanced research in global communication, Philadelphia
Chandler J, Shapiro D (2016) Conducting clinical research using crowdsourced convenience samples. Ann Rev Clin Psychol 12:53–81. https://doi.org/10.1146/annurev-clinpsy-021815-093623
Choi BCK, Pak AWP (2005) A catalog of biases in questionnaires. Prov Chronic Dis 2(1):1–13
Codagnone C, Abadie F, Biagi F (2016) The future of work in the sharing economy: market efficiency and equitable opportunities or unfair precarisation? Publications Office of the European Union, Luxembourg
Cunningham-Parmeter K (2019) Gig-dependence: finding the real independent contractors of platform work. N Ill Univ Law Rev 39(3):379–427
Cutolo D, Kenney M (2019) The emergence of platform-dependent entrepreneurs: power asymmetries, risk, and uncertainty. Acad Manag Perspect. https://doi.org/10.5465/amp.2019.0103
D’Cruz P, Noronha E (2016) Positives outweighing negatives: the experiences of Indian crowdsourced workers. Work Org Labour Glob. https://doi.org/10.13169/workorgalaboglob.10.1.0044
De Stefano V (2015) The rise of the “just-in-time workforce”: on-demand work, crowd work and labour protection in the “gig-economy.” Comp Labor Law Policy J 37(3):471–503
De Groen WP, Maselli I (2016) The impact of the collaborative economy on the labour market. CEPS Special Report 138. http://aei.pitt.edu/76467/1/SR138CollaborativeEconomy_0.pdf. Accessed 5 May 2021
De Groen WP, Maselli I, Fabo B (2016) The digital market for loca services: a one night stand for workers? An example from the on demand economy. CEPS Publ Off, Luxembourg
Deng X, Joshi KD, Galliers RD (2016) The duality of empowerment and marginalization in microtask crowdsourcing: giving voice to the less powerful through value sensitive design. MIS Q 40(2):279–302. https://doi.org/10.25300/MISQ/2016/40.2.01
Dengler K, Matthes B (2015) Folgen der Digitalisierung für die Arbeitswelt: Substituierbarkeitspotenziale von Berufen in Deutschland. IAB-Forschungsbericht 11/2015
DGB (2021) DGB-Position zur Plattformarbeit. https://www.dgb.de/downloadcenter/++co++231aa4e0-a1ea-11eb-a42d-001a4a160123. Accessed 5 May 2022
Difallah DE, Catasta M, Demartini G, Ipeirotis PG, Cudré-Mauroux P (2015) The dynamics of micro-task crowdsourcing. In: WWW’15 companion: Proceedings of the 24th International Conference on World Wide Web. ACM, New York, pp 238–247. https://doi.org/10.1145/2740908.2744109
Drahokoupil J, Fabo B (2016) The platform economy and the disruption of the employment relationship. ETUI Policy Brief. https://doi.org/10.13140/RG.2.2.11554.63687
Dunn M (2017) Digital work: new opportunities or lost wages? Am J Manag. https://doi.org/10.5465/ambpp.2016.11689abstract
Durward D, Blohm I, Leimeister JM (2020) The nature of crowd work and its effects on individuals’ work perception. J Manag Inf Syst 37(1):66–95. https://doi.org/10.1080/07421222.2019.1705506
Elbanna A, Idowu A (2021) Crowdwork, digital liminality and the enactment of culturally recognised alternatives to Western precarity: beyond epistemological terra nullius. Eur J Inf Syst 31:128–144. https://doi.org/10.1080/0960085X.2021.1981779
Elvik R (1998) Evaluating the statistical conclusion validity of weighted mean results in meta-analysis by analysing funnel graph diagrams. Accid Anal Prev 30(2):255–266. https://doi.org/10.1016/S0001-4575(97)00076-6
European Commission (2021) Commission proposals to improve the working conditions of people working through digital labour platforms. Press release. https://ec.europa.eu/commission/presscorner/detail/en/ip_21_6605. Accessed 4 Apr 2022
Fabo B, Karanovic J, Dukova K (2017) In search of an adequate European policy response to the platform economy. Transf Eur Rev Labour Res 23(2):163–175. https://doi.org/10.1177/1024258916688861
FairCrowdWork (2017) Fair crowd work: platform reviews, 2 April. http://faircrowd.work/platform-reviews/. Accessed 2 Apr 2022
Forbes K, Gagnon J, Collins C (2021) Low inflation bends the Phillips curve around the world. Natl Bur Econ Res Camb. https://doi.org/10.3386/w29323
Funke C, Picot G (2021) Platform work in a coordinated market economy. Ind Relat J 52(4):348–363. https://doi.org/10.1111/irj.12339
Gao Y, Chen Y, Liu KJR (2015) On cost-effective incentive mechanisms in microtask crowdsourcing. IEEE Trans Comput Intell AI Games 7(1):3–15. https://doi.org/10.1109/TCIAIG.2014.2298361
Gegenhuber T, Schüßler E, Reischauer G, Thäter L (2021) Building collective institutional infrastructures for decent platform work: the development of a crowdwork agreement in Germany. Res Soc Org 79:43–68
Gerber C (2022) Gender and precarity in platform work: old inequalities in the new world of work. New Technol Work Employ (forthcoming). https://doi.org/10.1111/ntwe.12233
Giard N, Brunsmeier S, Hemsen P, Hesse M, Löken N, Nouri Z, Reimann M, Schlicher K, Schneider M, Schulte J (2021) Research Report. Erkenntnisse zur Arbeitsrealität deutscher Crowdworker. https://pub.uni-bielefeld.de/record/2953491. Accessed 6 Apr 2022
Goodman JK, Cryder CE, Cheema A (2013) Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples. J Behav Decis Mak 26(3):213–224. https://doi.org/10.1002/bdm.1753
Graham M, Anwar MA (2019) The global gig economy: towards a planetary labour market? First Monday. https://doi.org/10.5210/fm.v24i4.9913
Greef S, Schroeder W, Akel A, Berzel A, D’Antonio O, Kiepe L, Schreiter B, Sperling HJ (2017) Forschungsbericht 500: Plattformökonomie und Crowdworking: Eine Analyse der Strategien und Positionen zentraler Akteure. https://www.ssoar.info/ssoar/bitstream/handle/document/55503/ssoar-2017-greef_et_al-Plattformokonomie_und_Crowdworking_Eine_Analyse.pdf?sequence=1. Accessed 4 May 2022
Haidar J, Keune M (2021) Work and labour relations in global platform capitalism. ILERA Publication Series, Elgar
Hanrahan BV, Chen A, Ma J, Ma NF, Squicciarini A, Savage S (2021) The expertise involved in deciding which HITs are worth doing on Amazon Mechanical Turk. In: Proceedings of the ACM on Human-Computer Interaction, vol 5, issue CSCW1, Article 128. https://doi.org/10.1145/3449202
Hara K, Adams A, Milland K, Savage S, Callison-Burch C, Bigham JP (2018) A data-driven analysis of workers’ earnings on Amazon Mechanical Turk. In: Engage with CHI: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York. https://doi.org/10.1145/3173574.3174023
Hara K, Adams A, Milland K, Savage S, Hanrahan BV, Bigham JP, Callison-Burch C (2019) Worker demographics and earnings on Amazon Mechanical Turk. In: Extended abstracts of the 2019 CHI conference on human factors in computing systems, ACM, New York. https://doi.org/10.1145/3290607.3312970
Healy J, Pekarek A, Vromen A (2020) Sceptics or supporters? Consumers’ views of work in the gig economy. New Technol Work Employ 35(1):1–19. https://doi.org/10.1111/ntwe.12157
Heeks R (2017) Decent work and the digital gig economy: a developing country perspective on employment impacts and standards in online outsourcing, crowdwork, etc. Development Informatics Working Paper, No. 71. https://doi.org/10.2139/ssrn.3431033
Heeks R, Graham M, Mungai P, van Belle JP, Woodcock J (2021) Systematic evaluation of gig work against decent work standards: the development and application of the Fairwork framework. Inf Soc 37(5):267–286. https://doi.org/10.1080/01972243.2021.1942356
Henrich J, Heine S, Norenzayan A (2010) The weirdest people in the world? RatSWD Working Papers, 139. https://ideas.repec.org/p/rsw/rswwps/rswwps139.html
Hornuf L, Jeworrek S (2022) The effect of community managers on online idea crowdsourcing activities. J Assoc Inf Syst, forthcoming
Horton JJ, Chilton LB (2010) The labor economics of paid crowdsourcing. In: Proceedings of the 11th ACM conference on electronic commerce. ACM, New York. https://doi.org/10.1145/1807342.1807376
Idowu A, Elbanna A (2021) Institutionalizing crowdwork as a mode of employment: the case of crowdworkers in Nigeria. In: New Ways of Working: Palgrave Macmillan, Cham, pp 77–107. https://doi.org/10.1007/978-3-030-61687-8_4. Accessed 6 Apr 2022
ILO (2018) Job quality in the platform economy. Issue Brief No. 5 for the 2nd meeting of the Global Commission on the Future of Work. https://www.ilo.org/global./topics/future-of-work/publications/issue-briefs/WCMS_618167/lang--en/index.htm. Accessed 3 May 2022
International Monetary Fund (2021) World Economic Outlook Database, April 2021, 4 October. https://www.imf.org/en/Publications/WEO/weo-database/2021/April. Accessed 4 Apr 2022
Ipeirotis PG (2010) Analyzing the Amazon Mechanical Turk marketplace. XRDS Crossroads 17(2):16–21. https://doi.org/10.1145/1869086.1869094
Irani LC, Silberman MS (2013) Turkopticon: interrupting worker invisibility in Amazon Mechanical Turk. In: CHI ’13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, pp 611–620. https://doi.org/10.1145/2470654.2470742
Jäger G, Zilian LS, Hofer C, Füllsack M (2019) Crowdworking: working with or against the crowd? J Econ Interact Coord 14(4):761–788. https://doi.org/10.1007/s11403-019-00266-1
Jarrahi MH, Sutherland W, Nelson SB, Sawyer S (2020) Platformic management, boundary resources for gig work, and worker autonomy. Comput Supported Coop Work 29(1–2):153–189. https://doi.org/10.1007/s10606-019-09368-7
Jiang L, Wagner C, Chen X (2021) Taking time into account: understanding microworkers’ continued participation in microtasks. J Assoc Inf Syst 22(4):893–930. https://doi.org/10.17705/1jais.00684
Johnston H, Land-Kazlauskas C (2019) Organizing on-demand representation, voice, and collective bargaining in the gig economy. Conditions of Work and Employment, No. 94. https://ideas.repec.org/p/ilo/ilowps/994981993502676.html. Accessed 8 Dec 2021
Kaganer E, Carmel E, Hirschheim R, Olsen T (2013) Managing the human cloud. MIT Sloan Manag Rev 54(2):23–32
Kaplan T, Saito S, Hara K, Bigham JP (2018) Striving to earn more: a survey of work strategies and tool use among crowd workers. In: Sixth AAAI Conference on Human Computation and Crowdsourcing, Zürich
Kässi O, Lehdonvirta V (2018) Online labour index: measuring the online gig economy for policy and research. Technol Forecast Soc Change 137:241–248. https://doi.org/10.1016/j.techfore.2018.07.056
Kässi O, Lehdonvirta V, Stephany F (2021) How many online workers are there in the world? A data-driven assessment. Open Res Eur. https://doi.org/10.12688/openreseurope.13639.1
Kaufmann N, Schulze T, Veit D (2011) More than fun and money worker motivation in crowdsourcing – a study on Mechanical Turk. In: Proceedings of the Seventeenth Americas Conference on Information Systems, Detroit. https://www.semanticscholar.org/paper/More-than-fun-and-money.-Worker-Motivation-in-A-on-Kaufmann-Schulze/ff279098481a87faa4498fa9088cd9bd835e9a3e. Accessed 5 Apr 2022
Khovanskaya V, Dombrowski L, Rzeszotarski J, Sengers P (2019) The tools of management: adapting historical union tactics to platform-mediated labor. In: Proceedings of the ACM on Human-Computer Interaction, vol 3, issue CSCW. https://doi.org/10.1145/3359310
Kingsley SC, Gray ML, Suri S (2015) Accounting for market frictions and power asymmetries in online labor markets. Policy Internet 7(4):383–400. https://doi.org/10.1002/poi3.111
Kittur A, Nickerson JV, Bernstein M, Gerber E, Shaw A, Zimmerman J, Lease M, Horton J (2013) The future of crowd work. In: Proceedings of the 2013 conference on computer supported cooperative work. ACM, New York, pp 1301–1318. https://doi.org/10.1145/2441776.2441923
Koester G, Lis E, Nickel C, Osbat C, Smet F (2021) Understanding low inflation in the Euro area from 2013 to 2019: cyclical and structural drivers. ECB Occas Pap. https://doi.org/10.2139/ssrn.3928302
Kuek SC, Paradi-Guilford C, Fayomi T, Imaizumi S, Ipeirotis P, Pina P, Singh M (2015) The global opportunity in online outsourcing. https://openknowledge.worldbank.org/bitstream/handle/10986/22284/The0global0opp0n0online0outsourcing.pdf?sequence=1&isAllowed=y. Accessed 22 Apr 2022
Lascău L, Gould SJ, Brumby DP, Cox AL (2022) Crowdworkers’ temporal flexibility is being traded for the convenience of requesters through 19 “invisible mechanisms” employed by crowdworking platforms: a comparative analysis study of nine platforms. In: CHI conference on human factors in computing systems extended abstracts, ACM, New York. https://www.sjjg.uk/pdfs/crowd-flexibility.pdf. Accessed 3 Apr 2022
Lee WK, Cui Y (2020) Should gig platforms decentralize dispute resolution? SSRN Electron J. https://doi.org/10.2139/ssrn.3719630
Lee CH, Cook S, Lee JS, Han B (2016) Comparison of two meta-analysis methods: inverse-variance-weighted average and weighted sum of Z-scores. Genom Inform 14(4):173–180. https://doi.org/10.5808/GI.2016.14.4.173
Lehdonvirta V, Kässi O, Hjorth I, Barnard H, Graham M (2019) The global platform economy: a new offshoring institution enabling emerging-economy microproviders. J Manag 45(2):567–599. https://doi.org/10.1177/0149206318786781
Leimeister JM, Durward D, Zogaj S (2016) Crowd Worker in Deutschland: Eine empirische Studie zum Arbeitsumfeld auf externen Crowdsourcing-Plattformen, Study der Hans-Böckler-Stiftung, No. 323. https://www.econstor.eu/bitstream/10419/146407/1/867095172.pdf. Accessed 4 May 2022
Leist D, Hießl C, Schlachter M (2017) Forschungsbericht 499: Plattformökonomie – Eine Literaturauswertung. https://www.bmas.de/SharedDocs/Downloads/DE/Publikationen/Forschungsberichte/fb499-plattformoekonomie-eine-literaturauswertung.pdf?__blob=publicationFile&v=1. Accessed 1 May 2022
Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6(7):e1000100. https://doi.org/10.1371/journal.pmed.1000100
Lioznova A, Drutsa A, Kukushkin V, Bezzubtseva A (2020) Prediction of hourly earnings and completion time on a crowdsourcing platform. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, New York, pp 3172–3182. https://doi.org/10.1145/3394486.3403369
Litman L, Robinson J, Rosen Z, Rosenzweig C, Waxman J, Bates LM (2020) The persistence of pay inequality: the gender pay gap in an anonymous online labor market. PLoS One. https://doi.org/10.1371/journal.pone.0229383
Lübker M (2020) Europäischer Tarifbericht des WSI. WSI Report, No. 58, https://www.wsi.de/de/faust-detail.htm?sync_id=HBS-007729. Accessed 3 May 2022
Lynn T, Fowley F, Fox G, Rosati P, Endo PT, Ogunsanya L (2021) Nanojobs: towards an open crowd working platform for refugees and displaced persons. In: 2021 IEEE International Humanitarian Technology Conference. IEEE, pp 1–6. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9698930. Accessed 5 May 2022
Maier MF, Viete S (2017) Forschungsbericht 498: Plattformbasierte Erwerbsarbeit: Stand der empirischen Forschung. https://d-nb.info/1192198085/34. Accessed 3 Oct 2021
Mazar N, Amir O, Ariely D (2008) The dishonesty of honest people: a theory of self-concept maintenance. J Mark Res 45(6):633–644. https://doi.org/10.1509/jmkr.45.6.633
Moore JC, Stinson L, Welniak EJ (2000) Income measurement error in surveys: a review. J Off Stat 16(4):331–361. https://www.scb.se/contentassets/ca21efb41fee47d293bbee5bf7be7fb3/income-measurement-error-in-surveys-a-review.pdf. Accessed 3 May 2022
Mrass V, Peters C, Leimeister JM (2018) Good Practices deutscher Crowdworking-Plattformen. ITeG technical reports, volume 6: Herausforderung cloud & crowd: good practices und Handlungsempfehlungen. University Press, Kassel, pp 13–20
Muszyński K, Pulignano V, Domecka M, Mrozowicki A (2021) Coping with precarity during COVID-19: a study of platform work in Poland. Int Labour Rev. https://doi.org/10.1111/ilr.12224
Newlands G, Lutz C (2021) Crowdwork and the mobile underclass: barriers to participation in India and the United States. New Media Soc 23(6):1341–1361. https://doi.org/10.1177/1461444820901847
Nickerson JV (2013) Crowd work and collective learning. In: Littlejohn A, Margaryan A (eds) Technology-enhanced professional learning: processes, practices, and tools. Routledge, London
Nikzad A (2017) Thickness and competition in ride-sharing markets. SSRN Electron J. https://doi.org/10.2139/ssrn.3065672
O’Higgins N, Caro LP (2022) Crowdwork for young people: risks and opportunities. IZA Discussion Papers, No. 14933. https://docs.iza.org/dp14933.pdf. Accessed 3 May, 2022
Otey v. CrowdFlower, Inc. (2013) Case No. 12-cv-05524-JST (N.D. Cal. Aug. 27)
Pallais A (2014) Inefficient hiring in entry-level labor markets. Am Econ Rev 104(11):3565–3599. https://doi.org/10.1257/aer.104.11.3565
Paolacci G, Chandler JJ, Ipeirotis PG (2010) Running experiments on Amazon Mechanical Turk. Judgm Decis Mak. https://doi.org/10.1037/t69659-000
Pongratz HJ, Bormann S (2017) Online-Arbeit auf Internet-Plattformen: empirische Befunde zum, Crowdworking‘ in Deutschland. AIS Studien 10(2):151–181. https://doi.org/10.21241/SSOAR.64850
Prassl J, Risak M (2016) Uber, Taskrabbit, and Co.: platforms as employers – rethinking the legal analysis of crowdwork. Comp Labor Law Policy J 37(8):619–652
Preis U (2016) Forschungsbericht 490: Sozialversicherungsrechtliche Einordnung neuer Beschäftigungsformen mit Schwerpunkt Plattform-bzw. Gig-Ökonomie. https://www.bmas.de/SharedDocs/Downloads/DE/Publikationen/Forschungsberichte/fb499-plattformoekonomie-eine-literaturauswertung.pdf?__blob=publicationFile&v=1. Accessed 1 May 2022
Pulignano V, Marà C (2021) Working for nothing in the platform economy – thematic publication. https://lirias.kuleuven.be/3648337?limo=0. Accessed 5 Apr 2022
Rani U, Dhir KR, Furrer M, Gőbel N, Moraiti A, Cooney S, Manus MC, Coddou A (2021) World employment and social outlook – the role of digital labour platforms in transforming the world of work: ILO. https://www.ilo.org/global/research/global-reports/weso/2021/WCMS_771749/lang--en/index.htm. Accessed 27 Dec 2021
Rani U, Furrer M (2019) On-demand digital economy: can experience ensure work and income security for microtask workers? Jahrbücher Für Nationalökonomie Und Statistik 239(3):565–597. https://doi.org/10.1515/jbnst-2018-0019
Rani U, Furrer M (2021) Digital labour platforms and new forms of flexible work in developing countries: algorithmic management of work and workers. Compet Change 25(2):212–236. https://doi.org/10.1177/1024529420905187
Revelli C, Viviani JL (2015) Financial performance of socially responsible investing (SRI): what have we learned? A meta-analysis. Bus Ethics Eur Rev 24(2):158–185. https://doi.org/10.1111/beer.12076
Ross J, Irani L, Silberman MS, Zaldivar A, Tomlinson B (2010) Who are the crowdworkers? In: CHI ‘10 extended abstracts on human factors in computing systems. ACM, New York, pp 2863–2872. https://doi.org/10.1145/1753846.1753873
Saito S, Chiang CW, Savage S, Nakano T, Kobayashi T, Bigham JP (2019) TurkScanner: predicting the hourly wage of microtasks. In: Proceedings of the 28th international conference on world wide web. ACM, New York, pp 3187–3193. https://doi.org/10.1145/3308558.3313716
Salehi N, Irani LC, Bernstein MS, Alkhatib A, Ogbe E, Milland K, Clickhappier (2015) We are dynamo. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems. ACM, New York, pp 1621–1630. https://doi.org/10.1145/2702123.2702508
Sannon S, Cosley D (2019) Privacy, power, and invisible labor on Amazon Mechanical Turk. In: Proceedings of the 2019 CHI conference on human factors in computing systems. ACM, New York, pp 1–12. https://doi.org/10.1145/3290605.3300512
Satterthwaite FE (1946) An approximate distribution of estimates of variance components. Biom Bull 2(6):110–114. https://doi.org/10.2307/3002019
Schmidt FA (2017) Conclusion: towards an ethics of creative crowdwork. In: Schmidt FA (ed) Crowd design: from tools for empowerment to platform capitalism. Birkhäuser, Zurich, pp 211–236
Schmidt FL, Hunter JE (2015) Methods of meta-analysis: correcting error and bias in research findings. Sage, Thousand Oaks. https://doi.org/10.4135/9781483398105
Schmidt FA (2016) Arbeitsmärkte in der Plattformökonomie – zur Funktionsweise und den Herausforderungen von Crowdwork und Gigwork: Friedrich-Ebert-Stiftung. https://library.fes.de/pdf-files/wiso/12826.pdf. Accessed 5 May 2021
Schoukens P (2020) Digitalisation and social security in the EU. The case of platform work: from work protection to income protection? Eur J Soc Secur 22(4):434–451. https://doi.org/10.1177/1388262720971300
Schriner A, Oerther D (2014) No really, (crowd) work is the silver bullet. Procedia Eng 78:224–228. https://doi.org/10.1016/j.proeng.2014.07.060
Segev E (2020) Crowdsourcing contests. Eur J Oper Res 281(2):241–255. https://doi.org/10.1016/j.ejor.2019.02.057
Serfling O (2018) Crowdworking Monitor No. 1. https://www.bmas.de/SharedDocs/Downloads/DE/Meldungen/2018/crowdworking-monitor.pdf?__blob=publicationFile&v=1. Accessed 3 May 2022
Shafiei Gol E, Stein MK, Avital M (2018) Why take the risk? Motivations of highly skilled workers to participate in crowdworking platforms. In: Proceedings of the 39th international conference on information systems. ACM, New York. https://aisel.aisnet.org/icis2018/crowd/Presentations/7. Accessed 17 Oct 2021
Shmueli B, Fell J, Ray S, Ku LW (2021) Beyond fair pay: ethical implications of NLP crowdsourcing. https://arxiv.org/pdf/2104.10097. Accessed 1 Apr 2022
Silberman MS, Tomlinson B, LaPlante R, Ross J, Irani L, Zaldivar A (2018) Responsible research with crowds. Commun ACM 61(3):39–41. https://doi.org/10.1145/3180492
Silberman MS, Irani L (2015) Operating an employer reputation system: lessons from Turkopticon, 2008–2015. Comp Labor Law Policy J 37(3):505–542
Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ (2000) Publication and related biases. Health Technol Assess. https://doi.org/10.3310/hta4100
Spindeldreher K, Schlagwein D (2016) What drives the crowd? A meta-analysis of the motivation of participants in crowdsourcing. In: Proceedings of the 20th Pacific Asia Conference on Information Systems. Chiayi
Stephany F, Dunn M, Sawyer S, Lehdonvirta V (2020) Distancing bonus or downscaling loss? The changing livelihood of Us online workers in times of COVID-19. J Econ Soc Geogr 111(3):561–573. https://doi.org/10.1111/tesg.12455
Stephany F (2021) When does it pay off to learn a new skill? Revealing the complementary benefit of cross-skilling, Working Paper. https://doi.org/10.31235/osf.io/sv9de
Tay P, Large O (2022) Making it work: understanding the gig economy’s shortcomings and opportunities. Tony Blair Institute for Global Change, London. https://institute.global/sites/default/files/articles/Making-It-Work-Understanding-the-Gig-Economy-s-Shortcomings-and-Opportunities.pdf. Accessed 12 Apr 2022
Teevan J (2016) The future of microwork. XRDS Crossroads 23(2):26–29. https://doi.org/10.1145/3019600
Toxtli C, Suri S, Savage S (2021) Quantifying the invisible labor in crowd work. In: Proceedings of the ACM on human–computer interaction, vol 5, issue CSCW2. https://doi.org/10.1145/3476060
Wei W, MacDonald IT (2021) Modeling the job quality of “work relationships” in China’s gig economy. Asia Pac J Hum Resour. https://doi.org/10.1111/1744-7941.12310
Whiting ME, Hugh G, Bernstein MS (2019) Fair work: crowd work minimum wage with one line of code. In: Proceedings of the AAAI conference on human computation and crowdsourcing, Stevenson, pp. 197–206. https://ojs.aaai.org//index.php/HCOMP/article/view/5283. Accessed 1 Apr 2022
Wong SI, Fieseler C, Kost D (2020) Digital labourers’ proactivity and the venture for meaningful work: fruitful or fruitless? J Occup Org Psychol 93(4):887–911. https://doi.org/10.1111/joop.12317
Wood AJ, Graham M, Lehdonvirta V, Hjorth I (2019a) Good gig, bad gig: autonomy and algorithmic control in the global gig economy. Work Employ Soc 33(1):56–75. https://doi.org/10.1177/0950017018785616
Wood AJ, Graham M, Lehdonvirta V, Hjorth I (2019b) Networked but commodified: the (dis)embeddedness of digital labour in the gig economy. Sociology 53(5):931–950. https://doi.org/10.1177/0038038519828906
Zhang C, Chen J, Raghunathan S (2022) Two-sided platform competition in a sharing economy. Manag Sci. https://doi.org/10.1287/mnsc.2022.4302