On the Applicability of Brain Reading for Predictive Human-Machine Interfaces in Robotics

PLoS ONE - Tập 8 Số 12 - Trang e81732
Elsa Andrea Kirchner1,2, Su Kyoung Kim1,2, Sirko Straube2, Anett Seeland3, Hendrik Wöhrle3, Mario Michael Krell2, Marc Tabie2, Manfred Fahle4,5
1Robotics Innovation Center (RIC), German Research Center for Artificial Intelligence (DFKI), Bremen, Germany
2Robotics Lab, University of Bremen, Bremen, Germany
3Robotics Innovation Center (RIC), German Research Center for Artificial Intelligence (DFKI), Bremen, Germany#TAB#
4Brain Research Institute IV, University of Bremen, Bremen, Germany
5The Henry Wellcome Laboratories for Vision Sciences, City University, London, United Kingdom

Tóm tắt

Từ khóa


Tài liệu tham khảo

JE Young, 2009, Toward acceptable domestic robots: Applying insights from social psychology, Int J Soc Robot, 1, 95, 10.1007/s12369-008-0006-y

T Kaupp, 2010, Human-robot communication for collaborative decision making making – A probabilistic approach, Rob Auton Syst, 58, 444, 10.1016/j.robot.2010.02.003

P Prodanov, 2005, Bayesian networks based multi-modality fusion for error handling in human-robot dialogues under noisy conditions, Speech Communication, 45, 231, 10.1016/j.specom.2004.10.015

Graf B, Parlitz C, Hägele M (2009) Robotic home assistant Care-O-bot ® product vision and innovation platform. In: Proceedings of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and Techniques. Berlin, Heidelberg: Springer-Verlag, 312–320. <comment>doi:<ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="http://dx.doi.org/10.1007/978-3-642-02577-834" xlink:type="simple">10.1007/978-3-642-02577-8 34</ext-link></comment>

K Kurihara, 2009, Facial emotion and gesture reproduction method for substitute robot of remote person, Computers and Industrial Engineering, 56, 631, 10.1016/j.cie.2008.10.004

J Allanson, 2004, A research agenda for physiological computing, Interact Comput, 16, 857, 10.1016/j.intcom.2004.08.001

DD Woods, 1996, Decomposing Automation: Apparent Simplicity, Real Complexity, CRC, chapter, 1, 3

L Parra, 2003, Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring, IEEE Trans Neural Syst Rehabil Eng, 11, 173, 10.1109/TNSRE.2003.814446

TO Zander, 2010, Combining Eye Gaze Input With a Brain-Computer Interface for Touchless Human-Computer Interaction, Int J Hum Comput Interact, 27, 38, 10.1080/10447318.2011.535752

JR Wolpaw, 2002, Brain-computer interfaces for communication and control, Clin Neurophysiol, 113, 767, 10.1016/S1388-2457(02)00057-3

Guger C, Harkam W, Hertnaes C, Pfurtscheller G (1999) Prosthetic control by an EEG-based brain-computer interface (BCI). In: Proceedings of the 5th European AAATE Conference. 3–6.

N Birbaumer, 2006, Breaking the silence: brain-computer interfaces (BCI) for communication and motor control, Psychophysiology, 6, 517, 10.1111/j.1469-8986.2006.00456.x

R Leeb, 2006, Walking by thinking: the brainwaves are crucial, not the muscles! Presence: Teleoperators and Virtual Environments, 15, 500

C Enzinger, 2008, Brain motor system function in a patient with complete spinal cord injury following extensive brain-computer interface training, Exp Brain Res, 190, 215, 10.1007/s00221-008-1465-y

B Blankertz, 2006, The Berlin Brain-Computer Interface: EEG-based communication without subject training, IEEE Trans Neural Syst Rehabil Eng, 14, 147, 10.1109/TNSRE.2006.875557

LA Farwell, 1988, Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials, Electroencephalogr Clin Neurophysiol, 70, 510, 10.1016/0013-4694(88)90149-6

Blankertz B, Dornhege G, Krauledat M, Schröder M, Williamson J, <etal>et al</etal>.. (2006) The Berlin Brain-Computer Interface presents the novel mental typewriter Hex-o-Spell. In: Verlag der Technischen Universität Graz. 108–109.

Allison B, Gräser A, Graimann B (2007) Why use a BCI if you are healthy? In: ACE Workshop-Brain Computer Interfaces and Games. 7–11.

B Reuderink, 2008, Games and Brain-Computer Interfaces: The State of the Art, WP2 BrainGain Deliverable HMI University of Twente September, 2008, 1

Nijholt A, Tan D, Allison BZ, Millán JdR, Graimann B (2008). Brain-computer interfaces for HCI and games.

Summerer L, Izzo D, Rossini L, editors (2009) Brain Machine Interfaces for Space Applications: Enhancing Astronaut Capabilities, volume 86 of International Review of Neurobiology. New York: Academic Press.

Millán JdR, Galán F, Vanhooydonck D, Lew E, Philips J, <etal>et al</etal>.. (2009) Asynchronous non-invasive brain-actuated control of an intelligent wheelchair. In: Proc. 31st Annu. Conf. IEEE Eng. Med. Biol. Soc. Minneapolis, MN, 3361–3364.

Trieu HT, Willey K, Nguyen HT (2009) Adaptive shared control strategies based in the Bayesian recursive technique in an intelligent wheelchair. In: Proc. 31st Annu. Conf. IEEE Eng. Med. Biol. Soc. Minneapolis, MN, 7118–7121.

Cutrell E, Tan D (2008) Bci for passive input in hci. In: CHI 2008– Workshop Brain-Computer Interfaces for HCI and Games. 1–3.

Zander TO, Kothe C, Jatzev S, Gärtner M (2010) Enhancing human-computer interaction with input from active and passive brain-computer interfaces. Brain-Computer Interfaces: 181–199.

George L, Lécuyer A (2010) An overview of research on “passive” brain-computer interfaces for implicit human-computer interaction. In: International Conference on Applied Bionics and Biomechanics ICABB 2010– Workshop W1 “Brain-Computer Interfacing and Virtual Reality”. Venice, Italy.

S Haufe, 2011, EEG potentials predict upcoming emergency brakings during simulated driving, J Neural Eng, 8, 056001, 10.1088/1741-2560/8/5/056001

G Pfurtscheller, 2010, The hybrid BCI, Front Neurosci, 4, 1

BZ Allison, 2012, Toward smarter BCIs: extending BCIs through hybridization and intelligent control, J Neural Eng, 9, 013001, 10.1088/1741-2560/9/1/013001

EA Kirchner, 2013, A Formal Model for Embedded Brain Reading, Industrial Robot: An International Journal, 40, 530, 10.1108/IR-01-2013-318

M Coles, 1989, Modern mind-brain reading: Psychophysiology, physiology, and cognition, Psychophysiology, 26, 251, 10.1111/j.1469-8986.1989.tb01916.x

JD Haynes, 2006, Decoding mental states from brain activity in humans, Nat Rev Neurosci, 7, 523, 10.1038/nrn1931

Y Kamitani, 2005, Decoding the visual and subjective contents of the human brain, Nature Neuroscience, 8, 679, 10.1038/nn1444

Y Miyawaki, 2008, Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders, Neuron, 60, 915, 10.1016/j.neuron.2008.11.004

T Naselaris, 2009, Bayesian Reconstruction of Natural Images from Human Brain Activity, Neuron, 63, 902, 10.1016/j.neuron.2009.09.006

SM Polyn, 2005, Category-specific cortical activity precedes retrieval during memory search, Science (New York, NY), 310, 1963, 10.1126/science.1117645

P Suppes, 2009, Partial orders of similarity differences invariant between EEG-recorded brain and perceptual representations of language, Neural Comput, 21, 3228, 10.1162/neco.2009.04-08-764

J Isreal, 1980, P300 and tracking diffculty: Evidence for multiple resources in dual-task performance, Psychophysiology, 17, 259, 10.1111/j.1469-8986.1980.tb00146.x

NK Squires, 1975, Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man, Electroencephalogr Clin Neurophysiol, 38, 387, 10.1016/0013-4694(75)90263-1

R Verleger, 1994, Reduction of P3b in patients with temporo-parietal lesions, Brain Res Cogn Brain Res, 2, 103, 10.1016/0926-6410(94)90007-8

J Polich, 2007, Updating P300: an integrative theory of P3a and P3b, Clin Neurophysiol, 118, 2128, 10.1016/j.clinph.2007.04.019

M Kutas, 1977, Augmenting mental chronometry: the P300 as a measure of stimulus evaluation time, Science, 197, 792, 10.1126/science.887923

D Salisbury, 2001, Button-pressing Affects P300 Amplitude and Scalp Topography, Clin Neurophysiol, 112, 1676, 10.1016/S1388-2457(01)00607-1

Kirchner EA, Metzen JH, Duchrow T, Kim SK, Kirchner F (2009) Assisting Telemanipulation Operators via Real-Time Brain Reading. In: Lohweg V, Niggemann O, editors, Proc. Mach. Learning in Real-time Applicat. Workshop 2009. Paderborn, Germany, Lemgoer Schriftenreihe zur industriellen Informationstechnik.

B Rolke, 2001, Missed prime words within the attentional blink evoke an n400 semantic priming effect, Psychophysiology, 38, 165, 10.1111/1469-8986.3820165

R West, 2011, The temporal dynamics of prospective memory: a review of the ERP and prospective memory literature, Neuropsychologia, 49, 2233, 10.1016/j.neuropsychologia.2010.12.028

Kirchner EA, Kim SK (2012) EEG in Dual-Task Human-Machine Interaction: Target Recognition and Prospective Memory. In: Proceedings of the 18th Annual Meeting of the Organization for Human Brain Mapping.

Kirchner EA, Kim SK, Fahle M (2013) EEG in Dual-Task Human-Machine Interaction: On the Feasibility of EEG based Support of Complex Human-Machine Interaction. Perception 42 ECVP Abstract Supplement: 220.

Woehrle H, Kirchner EA (2014) Online Detection of P300 related Target Recognition Processes During a Demanding Teleoperation Task. In: Proc. International Conference on Physiological Computing Systems, (PhyCS 2014). Lissabon, Portugal: SCITEPRESS Digital Library.

Folgheraiter M, Bongardt B, Schmidt S, de Gea Fernandéz J, Albiez J, <etal>et al</etal>.. (2009) Design of an Arm Exoskeleton using an hybrid Motion-Capture and Model-Based Technique. In: IEEE Int. Conf. Robotics and Automation. Kobe.

Folgheraiter M, Bongardt B, Albiez J, Kirchner F (2008) A bio-inspired haptic interface for telerobotics applications. In: IEEE Int. Conf. Robotics and Biomemetics. Bangkok, 560–565.

M Folgheraiter, 2012, Measuring the improvement of the interaction comfort of a wearable exoskeleton, International Journal of Social Robotics, 4, 285, 10.1007/s12369-012-0147-x

Seeland A, Wöhrle H, Straube S, Kirchner EA (2013) Online movement prediction in a robotic application scenario. In: 6th International IEEE EMBS Conference on Neural Engineering (NER). San Diego, California, 41–44.

Kirchner EA, Albiez JC, Seeland A, Jordan M, Kirchner F (2013) Towards assistive robotics for home rehabilitation. In: Proceedings of the International Conference on Biomedical Electronics and Devices. SciTePress, 168–177.

HH Kornhuber, 1965, Hirnpotentialänderungen bei Willkürbewegungen und passiven Bewegungen des Menschen: Bereitschaftspotential und reafferente Potentiale, Pflügers Archives, 284, 1, 10.1007/BF00412364

L Deecke, 1969, Distribution of readiness potential, pre-motion positivity, and motor potential of the human cerebral cortex preceding voluntary finger movements, Exp Brain Res, 7, 158, 10.1007/BF00235441

E Balconi, 2009, The multicomponential nature of movement-related cortical potentials: functional generators and psychological factors, Neurophysiol Trends, 5, 59

N Miller, 1969, Learning of visceral and gladular responses, Science, 163, 434, 10.1126/science.163.3866.434

SJ Pan, 2010, A Survey on Transfer Learning, IEEE Trans Knowl Data Eng, 22, 1345, 10.1109/TKDE.2009.191

I Iturrate, 2013, Task-dependent signal variations in EEG error-related potentials for brain-computer interfaces, J Neural Eng, 10, 026024, 10.1088/1741-2560/10/2/026024

Kim SK, Kirchner EA (2013) Classifier transferability in the detection of error related potentials from observation to interaction. In: Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, SMC 2013, Manchester, UK, October 13–16, 2013.

Tabie M, Kirchner EA (2013) EMG onset detection – Comparison of different methods for a movement prediction task based on emg. In: Alvarez S, Solé-Casals J, Fred A, Gamboa H, editors, Proceedings of the 6th International Conference on Bio-inspired Systems and Signal Processing (BIOSIGNALS-13). Barcelona: SciTePress, 242–247.

T Kaufmann, 2011, ERPs contributing to classification in the P300 BCI, In: 5th International BCI Conference, 2011, 136

B Jansen, 2004, An exploratory study of factors affecting single trial P300 detection, IEEE Trans Biomed Eng, 51, 975, 10.1109/TBME.2004.826684

F Ghaderi, 2014, Effects of eye artifact removal methods on single trial P300 detection, a comparative study, Journal of Neuroscience Methods, 221, 41, 10.1016/j.jneumeth.2013.08.025

Krell MM, Straube S, Seeland A, Wöhrle H, Teiwes J, <etal>et al</etal>.. (2013). pySPACE. Available: <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="https://github.com/pyspace" xlink:type="simple">https://github.com/pyspace</ext-link>.

Chang CC, Lin CJ (2001) LIBSVM: a library for support vector machines. Available: <ext-link xmlns:xlink="http://www.w3.org/1999/xlink" ext-link-type="uri" xlink:href="http://www.csie.ntu.edu.tw/cjlin/libsvm" xlink:type="simple">http://www.csie.ntu.edu.tw/cjlin/libsvm</ext-link>.

JA Hanley, 1982, The meaning and use of the area under a receiver operating characteristic (ROC) curve, Radiology, 143, 29, 10.1148/radiology.143.1.7063747

Nocedal J, Wright S (1999) Numerical Optimization. Springer Series in Operations Research. Springer.

Brodersen K, Ong C, Stephan K, Buhmann JM (2010) The balanced accuracy and its posterior distribution. Proceedings of the 20th International Conference on Pattern Recognition: 3121–3124.

M Krauledat, 2008, Towards zero training for Brain-Computer interfacing, PLoS ONE, 3, e2967, 10.1371/journal.pone.0002967

S Fazli, 2009, Subject-independent mental state classification in single trials, Neural Netw, 22, 1305, 10.1016/j.neunet.2009.06.003

Lotte F, Guan C (2010) Learning from other subjects helps reducing Brain-Computer interface calibration time. In: ICASSP. 614–617.

Metzen JH, Kim SK, Kirchner EA (2011) Minimizing calibration time for brain reading. In: Mester R, Felsberg M, editors, Pattern Recognition. Springer Berlin/Heidelberg, volume 6835 of Lecture Notes in Computer Science, 366–375.

B Blankertz, 2006, The Berlin Brain-Computer Interface: Machine learning based detection of user specific brain states, J Univers Comput Sci, 12, 581

O Bai, 2011, Prediction of human voluntary movement before it occurs, Clin Neurophysiol, 122, 364, 10.1016/j.clinph.2010.07.010

Ibez J, Serrano J, del Castillo M, Barrios L, Gallego J, <etal>et al</etal>.. (2011) An EEG-Based Design for the Online Detection of Movement Intention. Adv Neural Inf Process Syst: 370–377.

B Wang, 2009, Classification of Single-Trial EEG based on support vector clustering during finger movement, Advances in Neural Networks-ISNN, 2009, 354