Future directions for chatbot research: an interdisciplinary research agenda

Computing - Tập 103 Số 12 - Trang 2915-2942 - 2021
Asbjørn Følstad1, Theo Araujo2, Effie Lai‐Chong Law, Petter Bae Brandtzæg1, Symeon Papadopoulos3, Lea Reis, Marcos Báez4, Guy Laban5, Patrick McAllister6, Carolin Ischen2, Rebecca Wald2, Fabio Catania7, Raphael Meyer von Wolff8, Sebastian Hobert8, Ewa Luger9
1SINTEF, Oslo, Norway
2University of Amsterdam, Amsterdam, The Netherlands
3CERTH, Thessaloníki, Greece
4Claude Bernard University Lyon 1, Villeurbanne, France
5University of Glasgow, Glasgow, UK
6Ulster University, Jordanstown campus, Newtownabbey, UK
7Politecnico di Milano, Milan, Italy
8University of Goettingen, Göttingen, Germany
9University of Edinburgh, Edinburgh, UK

Tóm tắt

AbstractChatbots are increasingly becoming important gateways to digital services and information—taken up within domains such as customer service, health, education, and work support. However, there is only limited knowledge concerning the impact of chatbots at the individual, group, and societal level. Furthermore, a number of challenges remain to be resolved before the potential of chatbots can be fully realized. In response, chatbots have emerged as a substantial research area in recent years. To help advance knowledge in this emerging research area, we propose a research agenda in the form of future directions and challenges to be addressed by chatbot research. This proposal consolidates years of discussions at the CONVERSATIONS workshop series on chatbot research. Following a deliberative research analysis process among the workshop participants, we explore future directions within six topics of interest: (a) users and implications, (b) user experience and design, (c) frameworks and platforms, (d) chatbots for collaboration, (e) democratizing chatbots, and (f) ethics and privacy. For each of these topics, we provide a brief overview of the state of the art, discuss key research challenges, and suggest promising directions for future research. The six topics are detailed with a 5-year perspective in mind and are to be considered items of an interdisciplinary research agenda produced collaboratively by avid researchers in the field.

Từ khóa


Tài liệu tham khảo

Adam M, Wessel M, Benlian A (2020) AI-based chatbots in customer service and their effects on user compliance. Electron Markets. https://doi.org/10.1007/s12525-020-00414-7

Adiwardana D, Luong MT, So DR, Hall J, Fiedel N, Thoppilan R, Le QV (2020) Towards a human-like open-domain chatbot. arXiv preprint. arXiv:2001.09977

Amershi S, Weld D, Vorvoreanu M, Fourney A, Nushi B, Collisson P, Teevan J (2019) Guidelines for human–AI interaction. In: Proceedings of the CHI 2019 (paper no. 3). ACM, New York

Araujo T (2018) Living up to the chatbot hype: the influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Comput Hum Behav 85:183–189

Araujo T (2020) Conversational agent research toolkit: an alternative for creating and managing chatbots for experimental research. Comput Commun Res 2(1):35–51

Ashktorab Z, Jain M, Liao QV, Weisz JD (2019) Resilient chatbots: repair strategy preferences for conversational breakdowns. In: Proceedings of CHI 2019 (paper no. 254). ACM, New York

Baez M, Daniel F, Casati F (2019) Conversational web interaction: proposal of a dialog-based natural language interaction paradigm for the Web. In: Chatbot research and design. Third international workshop, CONVERSATIONS. Springer, Cham, pp 94–11

Bavaresco R, Silveira D, Reis E, Barbosa J, Righi R, Costa C, Moreira C (2020) Conversational agents in business: a systematic literature review and future research directions. Comput Sci Rev 36:100239. https://doi.org/10.1016/j.cosrev.2020.100239

Bender EM, Gebru T, McMillan-Major A, Mitchell M (2021) On the dangers of stochastic parrots: can language models be too big? In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency. ACM, New York, pp 610–623

Bickmore T, Picard RW (2005) Establishing and maintaining long-term human–computer relationships. ACM Trans Comput Hum Interact 12(2):293–327. https://doi.org/10.1145/1067860.1067867

Bittner E, Shoury O (2019) Designing automated facilitation for design thinking: a chatbot for supporting teams in the empathy map method. In: Proceedings of the 52nd Hawaii international conference on system sciences. Scholar Space, Honolulu, pp 227–236

Bobrow DG, Kaplan RM, Kay M, Norman DA, Thompson H, Winograd T (1977) GUS, a frame-driven dialog system. Artif Intell 8(2):155–173

Bott N, Wexler S, Drury L, Pollak C, Wang V, Scher K, Narducci S (2019) A protocol-driven, bedside digital conversational agent to support nurse teams and mitigate risks of hospitalization in older adults: case control pre-post study. J Med Internet Res 21(10):e13440

Brandtzaeg PB, Følstad A (2017) Why people use chatbots. In: Proceeedings of the international conference on internet science—INSCI 2017. Springer, Cham, pp 377–392

Breazeal C (2003) Toward sociable robots. Robot Auton Syst 42(3–4):167–175

Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Amodei D (2020) Language models are few-shot learners. arXiv preprint. arXiv:2005.14165

Candello H, Pinhanez C, Pichiliani M, Cavalin P, Figueiredo F, Vasconcelos M, Carmo DH (2019) The effect of audiences on the user experience with conversational interfaces in physical spaces. In: Proceedings of CHI 2019 (paper no. 90). ACM, New York

Cassell J, Bickmore T, Billinghurst M, Campbell L, Chang K, Vilhjálmsson H, Yan H (1999) Embodiment in conversational interfaces: Rea. In: Proceedings of CHI ‘99. ACM, New York, pp 520–527

Catania F, Di Nardo N, Garzotto F, Occhiuto D (2019) Emoty: an emotionally sensitive conversational agent for people with neurodevelopmental disorders. In Proceedings of the 52nd Hawaii international conference on system sciences. Scholar Space, Honolulu, pp 2014–2023

Chen H, Liu X, Yin D, Tang J (2017) A survey on dialogue systems: recent advances and new frontiers. ACM SIGKDD Explor Newsl 19(2):25–35

Chung H, Iorga M, Voas J, Lee S (2017) Alexa, can I trust you? Computer 50(9):100–104

Ciechanowski L, Przegalinska A, Magnuski M, Gloor P (2019) In the shades of the uncanny valley: an experimental study of human–chatbot interaction. Future Gener Comput Syst 92:539–548

Corkrey R, Parkinson L (2002) Interactive voice response: review of studies 1989–2000. Behav Res Methods Instruments Comput 34(3):342–353

Cranor LF (2008) A framework for reasoning about the human in the loop. In :UPSec’08: Proceedings of the 1st conference on usability, psychology, and security. USENIX Association, Berkeley

Dale R (2016) The return of the chatbots. Nat Lang Eng 22(5):811–817

Daniel G, Cabot J, Deruelle L, Derras M (2020) Xatkit: a multimodal low-code chatbot development framework. IEEE Access 8:15332–15346

DeVault D, Artstein R, Benn G, Dey T, Fast E, Gainer A, Morency LP (2014) SimSensei Kiosk: a virtual human interviewer for healthcare decision support. In Proceedings of the 2014 international conference on autonomous agents and multi-agent systems, pp 1061–1068

Deriu J, Rodrigo A, Otegi A, Echegoyen G, Rosset S, Agirre E, Cieliebak M (2021) Survey on evaluation methods for dialogue systems. Artif Intell Rev 54:755–810

Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint, arXiv:1810.04805

European Commission (2019) Ethics guidelines for trustworthy AI. Retrieved from: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai

Farooq U, Grudin J (2016) Human–computer integration. Interactions 23(6):26–32

Feine J, Gnewuch U, Morana S, Maedche A (2019a) A taxonomy of social cues for conversational agents. Int J Hum Comput Stud 132:138–161

Feine J, Gnewuch U, Morana S, Maedche A (2019b) Gender bias in chatbot design. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 79–93

Ferrara E (2020) What types of COVID-19 conspiracies are populated by Twitter bots? First Monday. https://doi.org/10.5210/fm.v25i6.10633

Fitzpatrick KK, Darcy A, Vierhile M (2017) Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Mental Health 4(2):e19

Floridi L, Chiriatti M (2020) GPT-3: Its nature, scope, limits, and consequences. Mind Mach 30(4):681–694

Frey CB, Osborne MA (2017) The future of employment: How susceptible are jobs to computerisation? Technol Forecast Soc Chang 114:254–280

Følstad A, Brandtzæg PB (2017) Chatbots and the new world of HCI. Interactions 24(4):38–42

Følstad A, Brandtzaeg PB (2020) Users’ experiences with chatbots: findings from a questionnaire study. Quality User Exp. https://doi.org/10.1007/s41233-020-00033-2

Følstad A, Brandtzaeg PB, Feltwell T, Law ELC, Tscheligi M, Luger E (2018) Chatbots for social good. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems (paper no. SIG06). ACM, New York

Følstad A, Araujo T, Papadopoulos S, Law EL-C, Luger E, Goodwin M, Brandtzaeg PB (eds) (2020) Chatbot research and design 4th international workshop, CONVERSATIONS 2020. Springer, Cham

Go E, Sundar SS (2019) Humanizing chatbots: the effects of visual, identity and conversational cues on humanness perceptions. Comput Hum Behav 97:304–316

Goel AK, Polepeddi L (2016) Jill Watson: A virtual teaching assistant for online education. Georgia Institute of Technology. Retrieved from https://smartech.gatech.edu/handle/1853/59104

Gorwa R, Guilbeault D (2020) Unpacking the social media bot: a typology to guide research and policy. Policy Internet 12(2):225–248

Grudin J, Jacques R (2019) Chatbots, humbots, and the quest for artificial general intelligence. In: Proceedings of CHI 2019 (paper no. 209). ACM, New York

Guzman AL, Lewis SC (2019) Artificial intelligence and communication: a human–machine communication research agenda. New Media Soc 22(1):70–86

Hagendorff T (2020) The ethics of AI ethics: An evaluation of guidelines. Mind Mach 30(1):99–120

Hall E (2018) Conversational Design. A Book Apart, New York

Hao K (2020) We read the paper that forced Timnit Gebru out of Google. Here’s what it says. MIT Technology Review. Retrieved from https://www.technologyreview.com/2020/12/04/1013294/google-ai-ethics-research-paper-forced-out-timnit-gebru/

Harms JG, Kucherbaev P, Bozzon A, Houben GJ (2018) Approaches for dialog management in conversational agents. IEEE Internet Comput 23(2):13–22

Harwell D (2018) The accent gap. The Washington Post. Retrieved from https://www.washingtonpost.com/graphics/2018/business/alexa-does-not-understand-your-accent/

Ho A, Hancock J, Miner AS (2018) Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. J Commun 68(4):712–733

Hobert S, Berens F (2019) Small talk conversations and the long-term use of chatbots in educational settings–experiences from a field study. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 260–272

Hobert S, Meyer von Wolff, R (2019) Say hello to your new automated tutor—a structured literature review on pedagogical conversational agents. In: Proceedings of the 14th international conference on Wirtschaftsinformatik. Association for Information Systems, pp.301–313

Hoffman G, Breazeal C (2004) Collaboration in human–robot teams. In: AIAA 1st intelligent systems technical conference. https://doi.org/10.2514/6.2004-6434

Hsieh T, Chaudhury B, Cross ES (2020) Human–robot cooperation in prisoner dilemma games: people behave more reciprocally than prosocially toward robots. In: Companion of the 2020 ACM/IEEE international conference on human–robot interaction. ACM, New York, pp 257–259

Jain M, Kumar P, Kota R, Patel SN (2018) Evaluating and informing the design of chatbots. In: Proceedings of the 2018 designing interactive systems conference. ACM, New York, pp 895–906

McTear MF (2002) Spoken dialogue technology: enabling the conversational user interface. ACM Comput Surv (CSUR) 34(1):90–169

Jokinen K, McTear M (2009) Spoken dialogue systems. Synthesis lectures on human language technologies. Morgan & Claypool Publishers, Williston

Jovanovic M, Baez M, Casati F (2020) Chatbots as conversational healthcare services. IEEE Internet Comput. https://doi.org/10.1109/MIC.2020.3037151

Jurafsky D, Martin JH (2020) Dialogue systems and chatbots. In: Speech and language processing. Draft of December 30, 2020. https://web.stanford.edu/~jurafsky/slp3/24.pdf

Kim S, Lee J, Gweon G (2019) Comparing data from chatbot and web surveys: effects of platform and conversational style on survey response quality. In: Proceedings of CHI 2019 (paper no. 86). ACM, New York

Kocaballi AB, Laranjo L, Coiera E (2019) Understanding and measuring user experience in conversational interfaces. Interact Comput 31(2):192–207

Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I (2005) A conversational agent as museum guide–design and evaluation of a real-world application. In: International workshop on intelligent virtual agents. Springer, Heidelberg, pp 329–343

Kretzschmar K, Tyroll H, Pavarini G, Manzini A, Singh I (2019) Can your phone be your therapist? Young people’s ethical perspectives on the use of fully automated conversational agents (chatbots) in mental health support. Biomed Inform Insights. https://doi.org/10.1177/1178222619829083

Kvale K, Sell OA, Hodnebrog S, Følstad A (2019) Improving conversations: lessons learnt from manual analysis of chatbot dialogues. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 187–200

Laban G, George J, Morrison V, Cross ES (2021) Tell me more! Assessing interactions with social robots from speech. Paladyn J Behav Robot 12(1):136–159

Laban G, Araujo T (2019) Working together with conversational agents: the relationship of perceived cooperation with service performance evaluations. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 215–228

Laban G, Araujo T (2020) The effect of personalization techniques in users’ perceptions of conversational recommender system. In: IVA ’20: Proceedings of the 20th ACM international conference on intelligent virtual agents (paper no. 34). ACM, New York

Latour B (2005) Reassembling the Social - An introduction to actor-network-theory. Oxford University Press, Oxford, UK

Lee S, Choi J (2017) Enhancing user experience with conversational agent for movie recommendation: effects of self-disclosure and reciprocity. Int J Hum Comput Stud 103:95–105

Lester J, Branting K, Mott B (2004) Conversational agents. In: Singh MP (ed) The Practical Handbook of Internet Computing. Chapman & Hall/CRC, Boca Raton

Liao QV, Geyer W, Muller M, Khazaen Y (2020) Conversational interfaces for information search. Understanding and improving information search. Springer, Cham, pp 267–287

Liao QV, Mas-ud Hussain M, Chandar P, Davis M, Khazaeni Y, Crasso MP, Geyer W (2018) All work and no play? In Proceedings of CHI 2018 (paper no. 3). ACM,New York

Luger E, Sellen A (2016) “Like having a really bad PA”—the gulf between user expectation and experience of conversational agents. In: Proceedings of CHI 2016. ACM, New York, pp 5286–5297

McAfee A, Brynjolfsson E (2017) Machine, platform, crowd: harnessing our digital future. W.W. Norton & Company, New York

McTear M (2021) Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots. Morgan & Claypool, Williston

Meyer von Wolff R, Hobert S, Masuch K, Schumann M (2020) Chatbots at digital workplaces - A grounded-theory approach for surveying application areas and objectives. Pac Asia J Assoc Inf Syst 12(2):64–102

Moore RJ, Arar R (2019) Conversational UX design: a practitioner’s guide to the natural conversation framework. Morgan & Claypool, Williston

Nordheim CB, Følstad A, Bjørkli CA (2019) An initial model of trust in chatbots for customer service—findings from a questionnaire study. Interact Comput 31(3):317–335

Ntoutsi E, Fafalios P, Gadiraju U, Iosifidis V, Nejdl W, Vidal ME, Ruggieri S, Turini F, Papadopoulos S, Krasanakis E, Kompatsiaris I (2020) Bias in data-driven artificial intelligence systems—an introductory survey. Wiley Interdiscipl Rev Data Min Knowl Discov 10(3):e1356

Oh CS, Bailenson JN, Welch GF (2018) A systematic review of social presence: definition, antecedents, and implications. Front Robot AI 5:114

Paikens P, Znotiņš A, Bārzdiņš G (2020) Human-in-the-loop conversation agent for customer service. In: Proceedings of the 25th international conference on applications of natural language to information systems, NLDB 2020. Springer, Cham, pp 277–284

Pérez JQ, Daradoumis T, Puig JMM (2020) Rediscovering the use of chatbots in education: a systematic literature review. Comput Appl Eng Educ 28(6):1549–1565

Perski O, Crane D, Beard E, Brown J (2019) Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digital Health 5:2055207619880676

Piccolo LS, Mensio M, Alani H (2018) Chasing the chatbots. In Internet Science. INSCI 2018 international workshops. Springer, Cham, pp 157–169

Porcheron M, Fischer JE, Reeves S, Sharples S (2018) Voice interfaces in everyday life. In: Proceedings of CHI 2018 (paper no. 640). ACM,New York

Press G (2019) AI Stats News: 62 % of US Consumers like using chatbots to interact with businesses. Forbes. Retrieved from https://www.forbes.com/sites/gilpress/2019/10/25/ai-stats-news-us-consumers-interest-in-using-chatbots-to-interact-with-businesses-rise-to-62/

Rodríguez-Gil L, García-Zubia J, Orduña P, Villar-Martinez A, López-De-Ipiña D (2019) New approach for conversational agent definition by non-programmers: a visual domain-specific language. IEEE Access 7:5262–5276

Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, Boureau YL (2020) Recipes for building an open-domain chatbot. arXiv preprint. arXiv:2004.13637

Ruane E, Birhane A, Ventresque A (2019) Conversational AI: social and ethical considerations. In: Proceedings for the 27th AIAI Irish conference on artificial intelligence and cognitive science—AICS 2019, pp 104–115. CEUR Workshop Proceedings, Vol-2563. http://ceur-ws.org/Vol-2563/

Rubin VL, Chen Y, Thorimbert LM (2010) Artificially intelligent conversational agents in libraries. Library Hi Tech 28(4):496–522

Salem M, Kopp S, Wachsmuth I, Rohlfing K, Joublin F (2012) Generation and evaluation of communicative robot gesture. Int J Social Robot 4(2):201–217

Schaake M (2021). The European commission’s artificial intelligence act. Policy Brief, Stanford HAI. Retrieved from https://hai.stanford.edu/sites/default/files/2021-06/HAI_Issue-Brief_The-European-Commissions-Artificial-Intelligence-Act.pdf

Schuller BW (2018) Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends. Commun ACM 61(5):90–99

Seering J, Luria M, Ye C, Kaufman G, Hammer J (2020) It takes a village: integrating an adaptive chatbot into an online gaming community. In: Proceedings of CHI 2020 (paper no. 579). ACM,New York

Shevat A (2017) Designing bots: creating conversational experiences. O’Reilly Media, Boston

Shum HY, He XD, Li D (2018) From Eliza to XiaoIce: challenges and opportunities with social chatbots. Front Inf Technol Electron Eng 19(1):10–26

Strubell E, Ganesh A, McCallum A (2019) Energy and policy considerations for deep learning in NLP. arXiv preprint, arXiv:1906.02243

Sundar SS (2020) Rise of machine agency: a framework for studying the psychology of Human–AI Interaction (HAII). J Comput Mediat Commun 25(1):74–88

Suresh H, Guttag JV (2019) A framework for understanding unintended consequences of machine learning. arXiv preprint, arXiv:1901.10002

Syvänen S, Valentini C (2020) Conversational agents in online organization–stakeholder interactions: a state-of-the-art analysis and implications for further research. J Commun Manag 339–362

Ta V, Griffith C, Boatfield C, Wang X, Civitello M, Bader H, Loggarakis A (2020) User experiences of social support from companion chatbots in everyday contexts: thematic analysis. J Med Internet Res 22(3):e16235

Toxtli C, Monroy-Hernández A, Cranshaw J (2018) Understanding chatbot-mediated task management. In: Proceedings of CHI 2018 (paper no. 58). ACM,New York

Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB (2019) Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 64(7):456–464. https://doi.org/10.1177/0706743719828977

Van der Goot MJ, Pilgrim T (2019) Exploring age differences in motivations for and acceptance of chatbot communication in a customer service context. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 173–186

Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Conference on neural information processing systems—NIPS 2017, pp 5998–6008

Vaziri M, Mandel L, Shinnar A, Siméon J, Hirzel M (2017) Generating chat bots from web API specifications. In: Proceedings of the 2017 ACM SIGPLAN international symposium on new ideas, new Pparadigm, and reflections on programming and software. ACM, New York, pp 44–57

Vinyals O, Le Q (2015) A neural conversational model. arXiv preprint, arXiv:1506.05869

Väänänen K, Hiltunen A, Varsaluoma J, Pietilä I (2019) CivicBots–Chatbots for supporting youth in societal participation. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 143–157

Wallace R (2003) The elements of AIML style. Alice AI Foundation, 139

Weizenbaum J (1967) Contextual understanding by computers. Commun ACM 10(8):474–480

Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Brew J (2019) HuggingFace’s transformers: state-of-the-art natural language processing. arXiv preprint, arxiv:1910.03771

Xu A, Liu Z, Guo Y, Sinha V, Akkiraju R (2017) A new chatbot for customer service on social media. In: Proceedings of CHI 2017. ACM, New York, pp 3506–3510

Yaghoub-Zadeh-Fard MA, Zamanirad S, Benatallah B, Casati F (2020) REST2Bot: bridging the gap between bot platforms and REST APIs. In: Companion proceedings of the web conference 2020. ACM, New York, pp 245–248

Yang Q, Steinfeld A, Rosé C, Zimmerman J (2020) Re-examining whether, why, and how Human-AI Interaction is uniquely difficult to design. In: Proceedings of CHI 2020 (paper no. 174). ACM,New York

Zamora J (2017) I’m sorry, Dave, I’m afraid I can’t do that: Chatbot perception and expectations. In: Proceedings of the 5th international conference on human agent Interaction. ACM,New York,pp 253–260

Zarouali B, Van den Broeck E, Walrave M, Poels K (2018) Predicting consumer responses to a chatbot on Facebook. Cyberpsychol Behav Soc Netw 21(8):491–497

Zierau N, Engel C, Söllner M, Leimeister JM (2020) Trust in smart personal assistants: A systematic literature review and development of a research agenda. In: Proceedings of the 15th international conference on Wirtschaftsinformatik—WI2020. Association for Information Systems,pp 99–11

Zou J, Schiebinger L (2018) AI can be sexist and racist—it’s time to make it fair. Nature 559:324–326