Interactive machine learning for health informatics: when do we need the human-in-the-loop?

Samuel AL (1959) Some studies in machine learning using the game of checkers. IBM J Res Dev 3(3):210–229

Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255–260

LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

Bayes T (1763) An essay towards solving a problem in the doctrine of chances (posthumous communicated by Richard Price). Philos Trans 53:370–418

Barnard GA, Bayes T (1958) Studies in the history of probability and statistics: IX. Thomas Bayes’s essay towards solving a problem in the doctrine of chances. Biometrika 45(3/4):293–315

Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York

Murphy KP (2012) Machine learning: a probabilistic perspective. MIT press, Cambridge

Silver D, Huang A, Maddison CJ, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489

Zhong N, Liu JM, Yao YY, Wu JL, Lu SF, Qin YL, Li KC, Wah B (2007) Web intelligence meets brain informatics. In: Zhong N, Liu JM, Yao YY, Wu JL, Lu SF, Li KC (eds) Web intelligence meets brain informatics., Lecture Notes in Artificial Intellience 4845Springer, Berlin, pp 1–31

Holzinger A (2014) Trends in interactive knowledge discovery for personalized medicine: cognitive science meets machine learning. IEEE Intell Inform Bull 15(1):6–14

Mitchell TM (1997) Machine learning. McGraw Hill, New York

Petz G, Karpowicz M, Frschu H, Auinger A, Sttesk V, Holzinger A (2015) Computational approaches for mining users opinions on the web 2.0. Inf Process Manag 51(4):510–519

Holzinger A, Dehmer M, Jurisica I (2014) Knowledge discovery and interactive data mining in bioinformatics-state-of-the-art, future challenges and research directions. BMC Bioinform 15(S6):I1

Spinrad N (2014) Google car takes the test. Nature 514(7523):528–528

Holzinger A (2014) Biomedical informatics: discovering knowledge in big data. Springer, New York

Holzinger A (2013) Human–computer interaction and knowledge discovery (HCI-KDD): what is the benefit of bringing those two fields to work together? In: Cuzzocrea A, Kittl C, Simos DE, Weippl E, Lida X (eds) Multidisciplinary research and practice for information systems., Springer Lecture Notes in Computer Science LNCS 8127Springer, Heidelberg, pp 319–328

Settles B (2011) From theories to queries: Active learning in practice. In: Guyon I, Cawley G, Dror G, Lemaire V, Statnikov A (eds) Active Learning and Experimental Design Workshop 2010, vol 16, JMLR Proceedings, Sardinia, pp 1–18

Fahlman SE, Hinton GE, Sejnowski TJ (1983) Massively parallel architectures for Al: NETL, Thistle, and Boltzmann machines. In: Genesereth MR (ed) AAAI-83. AAAI, Washington, DC, pp 109–113

Hubel DH, Wensveen J, Wick B (1995) Eye, brain, and vision. Scientific American Library, New York

Hofmann T (2001) Unsupervised learning by probabilistic latent semantic analysis. Mach Learn 42(1–2):177–196

Holzinger A, Schantl J, Schroettner M, Seifert C, Verspoor Karin (2014) Biomedical text mining: state-of-the-art, open problems and future challenges. In: Holzinger A, Jurisica I (eds) Interactive knowledge discovery and data mining in biomedical informatics., Lecture Notes in Computer Science LNCS 8401, volume 8401Springer, Berlin, pp 271–300

Akgul CB, Rubin DL, Napel S, Beaulieu CF, Greenspan H, Acar B (2011) Content-based image retrieval in radiology: current status and future directions. j digit imaging 24(2):208–222

Gigerenzer G, Gaissmaier W (2011) Heuristic decision making. Annu Rev Psychol 62:451–482

Wilson AG, Dann C, Lucas, Xing EP (2015) The human kernel. arXiv preprint arXiv:1510.07389

Kieseberg P, Schantl J, Früwirt P, Weippl E, Holzinger A (2015) Witnesses for the doctor in the loop. In: Guo Y, Friston K, Aldo F, Hill S, Peng H (eds) Brain informatics and health., Lecture Notes in Artificial Intelligence LNAI 9250Springer, heidelberg, pp 369–378

Atzmüller M, Baumeister J, Puppe F (2006) Introspective subgroup analysis for interactive knowledge refinement. In Sutcliffe G, Goebel R (ed), FLAIRS nineteenth international florida artificial intelligence research society conference. AAAI Press, pp 402–407

G Toderici, H Aradhye, M Paca, L Sbaiz, J Yagnik (2010) Finding meaning on YouTube: tag recommendation and category discovery. In IEEE conference on computer vision and pattern recognition (CVPR 2010), IEEE, pp 3447–3454

Sturm W, Schreck T, Holzinger A, Ullrich T (2015) Discovering medical knowledge using visual analytics a survey on methods for systems biology and omics data. In Bühler K, Linsen L, John NW (eds) Eurographics workshop on visual computing for biology and medicine, Eurographics EG, pp 71–81

Müller E, Assent I, Krieger R, Jansen T, Seidl T (2008) Morpheus: interactive exploration of subspace clustering. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining KDD 08, ACM, pp 1089–1092

Hund M, Sturm W, Schreck T, Ullrich T, Keim D, Majnaric L, Holzinger A (2015) Analysis of patient groups and immunization results based on subspace clustering. In: Guo Y, Friston K, Aldo F, Hill S, Peng H (eds) Brain informatics and health., Lecture Notes in Artificial Intelligence LNAI 9250, vol 9250Springer International Publishing, Cham, pp 358–368

Shepard RN (1962) The analysis of proximities: multidimensional scaling with an unknown distance function. Psychometrika 27(2):125–140

Goldstone RL, Day S, Son JY (2010) Comparison. In: Glatzeder B, Goel V, Müller A (eds) Towards a theory of thinking. Springer, Berlin, pp 103–121

Awasthi P, Balcan M-F, Voevodski K (2014) Local algorithms for interactive clustering. In: Proceedings of the 31th international conference on machine learning, pp 550–558

Balcan M-F, Blum A (2008) Clustering with interactive feedback. In: Freund Y, Gyrfi L, Turn G, Zeugmann T (eds) Proceedings of the algorithmic learning theory: 19th international conference, ALT 2008, Budapest, Hungary. Springer, Berlin. pp 316–328. 13–16 Oct 2008

Lathrop RH (1994) The protein threading problem with sequence amino-acid interaction preferences is np-complete. Protein Eng 7(9):1059–1068

Good BM, Andrew I (2013) Crowdsourcing for bioinformatics. Bioinformatics 29(16):1925–1933

Cooper S, Khatib F, Treuille A, Barbero J, Lee J, Beenen M, Leaver-Fay A, Baker D, Popovic Z (2010) Predicting protein structures with a multiplayer online game. Nature 466(7307):756–760

Jane Budge E, Maria Tsoti S, James Howgate D, Sivakumar S, Jalali M (2015) Collective intelligence for translational medicine: Crowdsourcing insights and innovation from an interdisciplinary biomedical research community. Ann Med 47(7):570–575

Aggarwal CC (2005) On k-anonymity and the curse of dimensionality. In: Proceedings of the 31st international conference on Very large data bases VLDB, pp 901–909

Samarati P, Sweeney L (1998) Generalizing data to provide anonymity when disclosing information. In: Mendelzon AO, Paredaens J (eds) PODS ’98 17th ACM SIGACT-SIGMOD-SIGART symposium on principles of database systems, ACM, p 188

Sweeney L (2002) Achieving k-anonymity privacy protection using generalization and suppression. Int J Uncertain Fuzziness Knowl Based Syst 10(5):571–588

Machanavajjhala A, Kifer D, Gehrke J, Venkitasubramaniam M (2007) l-diversity: privacy beyond k-anonymity. ACM Trans Knowl Discov Data 1(1):1–52

Li N, Li T, Venkatasubramanian S (2007) t-Closeness: privacy beyond k-anonymity and l-diversity. In: IEEE 23rd international conference on data engineering, ICDE 2007, IEEE, pp 106–115

Nergiz ME, Clifton C (2010) Delta-presence without complete world knowledge. IEEE Trans Knowl Data Eng 22(6):868–883

Turing AM (1950) Computing machinery and intelligence. Mind 59(236):433–460

Littman ML (2015) Reinforcement learning improves behaviour from evaluative feedback. Nature 521(7553):445–451

Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT press, Cambridge

Szepesvri C (2010) Algorithms for reinforcement learning. Synth Lect Artif Intell Mach Learn 4(1):1–103

Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285

Busoniu L, Babuska R, De Schutter B (2008) A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C 38(2):156–172

Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, Petersen S, Beattie C, Sadik A, Antonoglou I, King H, Kumaran D, Wierstra D, Legg S, Hassabis D (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533

Thomaz A, Hoffman G, Breazeal C (2005) Real-time interactive reinforcement learning for robots. In: Oblinger D, Lau T, Gil Y, Bauer M (eds) AAAI 2005 workshop on human comprehensible machine learning. AAAI Press, Menlo Park, pp 9–13

Suay HB, Chernova S (2011) Effect of human guidance and state space size on interactive reinforcement learning. In: 20th IEEE international symposium on robot and human interactive communication. IEEE, pp 1–6

Yang M, Yang YX, Wang W, Ding HY, Chen J (2014) Multiagent-based simulation of temporal-spatial characteristics of activity-travel patterns using interactive reinforcement learning. Math Probl Eng. doi: 10.1155/2014/951367

Holland S, Ester M, Kiessling W (2003) Preference mining: a novel approach on mining user preferences for personalized applications. In: Lavrac N, Gamberger D, Todorovski L, Blockeel H (eds) Proceedings LNAI of the knowledge discovery in databases: PKDD 2003, vol 2838. Springer, Berlin, pp 204–216

Trotman A (2005) Learning to rank. Inf Retr 8(3):359–381

Liu Tie-Yan (2011) Learning to rank for information retrieval. Springer, Berlin

Agarwal S, Cortes C, Herbrich R (2005) Learning to rank. In: Weiss Y, Schlkopf B, Platt JC (eds) NIPS Workshop

Fürnkranz J, Hüllermeier E (2010) Preference learning. Springer, Berlin

Hunt EB (1962) Concept learning: an information processing problem. Wiley, Hoboken

Feldman J (2000) Minimization of Boolean complexity in human concept learning. Nature 407(6804):630–633

Lake BM, Salakhutdinov R, Tenenbaum JB (2015) Human-level concept learning through probabilistic program induction. Science 350(6266):1332–1338

Brochu E, Freitas ND, Ghosh A (2007) Active preference learning with discrete choice data. In: Platt JC, Koller D, Singer Y, Roweis ST (eds) Advances in neural information processing systems 20 (NIPS 2007). Neural Information Processing Systems Foundation, pp 409–416

Chu W, Ghahramani Z (2005) Preference learning with gaussian processes. In: Proceedings of the 22nd international conference on machine learning. ACM, pp 137–144

Thurstone LL (1927) A law of comparative judgment. Psychol Rev 34(4):273–286

Bush RR, Mosteller F (1951) A mathematical model for simple learning. Psychol Rev 58(5):313–323

Fürnkranz J, Hüllermeier E, Cheng W, Park S-H (2012) Preference-based reinforcement learning: a formal framework and a policy iteration algorithm. Mach Learn 89(1–2):123–156

Zhao Y, Kosorok MR, Zeng D (2009) Reinforcement learning design for cancer clinical trials. Stat Med 28(26):3294–3315

Knox WB, Stone P, Breazeal C (2013) Training a robot via human feedback: a case study. In: Herrmann G, Pearson M, Lenz A, Bremner P, Spiers A, Leonards U (eds) Social robotics. Lecture Notes in Artificial Intelligence LNAI 8239. Springer, Heidelberg, pp 460–470

Wilson A, Fern A, Tadepalli P (2012) A Bayesian approach for policy learning from trajectory preference queries. In: Pereira F, Burges C, Bottou L, Weinberger K (eds) Advances in neural information processing systems 25 (NIPS 2012), pp 1133–1141

Jain A, Wojcik B, Joachims T, Saxena A (2013) Learning trajectory preferences for manipulators via iterative improvement. In: Advances in neural information processing systems, pp 575–583

Yimam SM, Biemann C, Majnaric L, Sabanovic S, Holzinger A (2015) Interactive and iterative annotation for biomedical entity recognition. In: Yike G, Karl F, Faisal A, Sean H, Hanchuan P (eds) Brain Informatics and Health. Lecture Notes in Artificial Intelligence LNAI 9250. Springer, Cham, pp 347–357

Settles B (2012) Active learning. Morgan and Claypool, San Rafael

Cohn DA, Ghahramani Z, Jordan MI (1996) Active learning with statistical models. J Artif Intell Res 4:129–145

Warmuth MK, Liao J, Raetsch G, Mathieson M, Putta S, Lemmen C (2003) Active learning with support vector machines in the drug discovery process. J Chem Inf Comp Sci 43(2):667–673

Yue Y, Joachims T (2009) Interactively optimizing information retrieval systems as a dueling bandits problem. In: Proceedings of the 26th annual international conference on machine learning (ICML), ACM, pp 1201–1208

Yue Y, Broder J, Kleinberg R, Joachims T (2012) The k-armed dueling bandits problem. J Comput Syst Sci 78(5):1538–1556

Viappiani P, Boutilier C (2010) Optimal bayesian recommendation sets and myopically optimal choice query sets. In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A (eds) Advances in neural information processing systems 23. Curran, pp 2352–2360

Valiant LG (1984) A theory of the learnable. Commun ACM 27(11):1134–1142

Baxter J (2000) A model of inductive bias learning. J Artif Intell Res 12:149–198

Evgeniou T, Pontil M (2004) Regularized multi-task learning. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 109–117

Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244

Parameswaran S, Weinberger KQ (2010) Large margin multi-task metric learning. In: Lafferty J, Williams C, Shawe-Taylor J, Zemel R, Culotta A (eds) Advances in neural information processing systems 23 (NIPS 2010), pp 1867–1875

McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: the sequential learning problem. In: Bower GH (ed) The psychology of learning and motivation, vol 24. Academic Press, San Diego, pp 109–164

French Robert M (1999) Catastrophic forgetting in connectionist networks. Trends Cogn Sci 3(4):128–135

Goodfellow IJ, Mirza M, Xiao D, Courville A, Bengio Y (2015) An empirical investigation of catastrophic forgeting in gradient-based neural networks. hyperimagehttp://arxiv.org/abs/1312.6211v3arXiv:1312.6211v3

Pan SJ, Yang QA (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22(10):1345–1359

Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10:1633–1685

Sycara KP (1998) Multiagent systems. AI Mag 19(2):79

Lynch NA (1996) Distributed algorithms. Morgan Kaufmann, San Francisco

DeGroot MH (1974) Reaching a consensus. J Am Stat Assoc 69(345):118–121

Benediktsson JA, Swain PH (1992) Consensus theoretic classification methods. IEEE Trans Syst Man Cybern 22(4):688–704

Weller SC, Mann NC (1997) Assessing rater performance without a gold standard using consensus theory. Med Decis Mak 17(1):71–79

Olfati-Saber R, Fax JA, Murray RM (2007) Consensus and cooperation in networked multi-agent systems. In: Proceedings of the IEEE, vol 95(1), pp 215–233

Roche B, Guegan JF, Bousquet F (2008) Multi-agent systems in epidemiology: a first step for computational biology in the study of vector-borne disease transmission. BMC Bioinform 9:435

Kok JR, Vlassis N (2006) Collaborative multiagent reinforcement learning by payoff propagation. J Mach Learn Res 7:1789–1828

Freund R, Neubauer M, Summerer M, Gruber S, Schaffer J, Swoboda R (2000) A hybrid system for the recognition of hand-written characters. In: Ferri FJ, Iñesta JM, Amin A, Pudil P (eds) Advances in pattern recognition, LNCS, vol 1876. Springer, Heidelberg, pp 67–76. doi: 10.1007/3-540-44522-6_7

Parunak HVD, Nielsen P, Brückner S, Alonso R (2007) Hybrid multi-agent systems: integrating swarming and BDI agents. In: Brueckner SA., Hassas S, Jelasity M, Yamins D (eds) Engineering self-organising systems, LNCS, vol 4335. Springer, Berlin, pp 1–14. doi: 10.1007/978-3-540-69868-5_1