Ảo giác Người điều khiển rối được tạo ra bởi các tín hiệu âm học không gian ảo và các kích thích âm thanh - hình ảnh không đồng bộ ở cả người trẻ và người lớn tuổi

Multisensory Research - Tập 32 Số 8 - Trang 745-770 - 2019
Marnix Stawicki1,2,3, Piotr Majdak4,5, Deniz Başkent1,2,3
1Department of Otorhinolaryngology, Head and Neck Surgery, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
2Graduate School of Medical Sciences, Research School of Behavioral and Cognitive Neurosciences (BCN), University of Groningen, Groningen, The Netherlands
3University of Groningen
4Acoustics Research Institute, Austrian Academy of Sciences, Vienna, Austria
5Austrian Academy of Sciences,

Tóm tắt

Tóm tắt Ảo giác người điều khiển rối (ventriloquist illusion), sự thay đổi trong nhận thức về vị trí của một tín hiệu thính giác khi có thêm một tín hiệu hình ảnh đồng bộ nhưng không tương thích về không gian, đã được chứng minh trước đây trong các quần thể trẻ khỏe mạnh là một mô hình mạnh mẽ chủ yếu dựa vào các quá trình tự động. Ở đây, chúng tôi đề xuất ảo giác người điều khiển rối như một bài kiểm tra tiềm năng đơn giản để đánh giá sự tích hợp âm thanh - hình ảnh (AV) ở cả người trẻ và người lớn tuổi. Chúng tôi đã sử dụng một phiên bản được điều chỉnh của mô hình ảo giác, điều này có tính thích ứng, gần như không có bất kỳ thiên kiến nào, dựa vào việc biểu diễn tín hiệu âm thanh hai tai với các hàm chuyển giao liên quan đến đầu (HRTFs) chung thay vì nhiều loa, và được kiểm tra với các kích thích AV được trình bày đồng bộ và không đồng bộ (cả âm điệu và lời nói). Góc nghe tối thiểu (MAA), sự khác biệt nhỏ nhất có thể cảm nhận được về góc giữa hai nguồn âm thanh, đã được so sánh khi có và không có kích thích hình ảnh ở cả người trẻ và người lớn tuổi với không hoặc tối thiểu khiếm khuyết cảm giác. Hiệu ứng ảo giác, được đo bằng các MAA được thực hiện với HRTFs, đã được quan sát với cả kích thích hình ảnh đồng bộ và không đồng bộ, nhưng chỉ với tín hiệu âm điệu và không với tín hiệu lời nói. Các mẫu tương tự nhau giữa người trẻ và người lớn tuổi cho thấy tính linh hoạt của mô hình ảo giác người điều khiển rối đã được điều chỉnh.

Từ khóa


Tài liệu tham khảo

Abel, S. M., Giguère, C., Consoli, A. and Papsin, B. C. (2000). The effect of aging on horizontal plane sound localization, J. Acoust. Soc. Am. 108, 743–752.10.1121/1.429607

Alais, D. and Burr, D. (2004). The ventriloquist effect results from near-optimal bimodal integration, Curr. Biol. 3, 257–262.

Alm, M. and Behne, D. (2013). Audio-visual speech experience with age influences perceived audio-visual asynchrony in speech, J. Acoust. Soc. Am. 134, 3001–3010.10.1121/1.4820798

Alsius, A., Paré, M. and Munhall, K. G. (2018). Forty years after hearing lips and seeing voices: the McGurk effect revisited, Multisens. Res. 31, 111–144.10.1163/22134808-00002565

Altieri, N. and Hudock, D. (2014). Assessing variability in audiovisual speech integration skills using capacity and accuracy measures, Int. J. Audiol. 53, 710–718.10.3109/14992027.2014.909053

Basharat, A., Adams, M. S., Staines, W. R. and Barnett-Cowan, M. (2018). Simultaneity and temporal order judgments are coded differently and change with age: an event-related potential study, Front. Integr. Neurosci. 12, 15. DOI:10.3389/fnint.2018.00015.

Başkent, D. and Bazo, D. (2011). Audiovisual asynchrony detection and speech intelligibility in noise with moderate to severe sensorineural hearing impairment, Ear Hear. 32, 582–592.10.1097/AUD.0b013e31820fca23

Başkent, D., Clarke, J., Pals, C., Benard, M. R., Bhargava, P., Saija, J., Sarampalis, A., Wagner, A. and Gaudrain, E. (2016). Cognitive compensation of speech perception with hearing impairment, cochlear implants, and aging: how and to what degree can it be achieved?, Trends Hear. 20, 1–16. DOI:10.1177/2331216516670279.

Beauchamp, M. S. (2005). See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex, Curr. Op. Neurobiol. 15, 145–153.10.1016/j.conb.2005.03.011

Bergman, M., Blumenfeld, V. G., Cascardo, D., Dash, B., Levitt, H. and Margulies, M. K. (1976). Age-related decrement in hearing for speech: sampling and longitudinal studies, J. Gerontol. 31, 533–538.10.1093/geronj/31.5.533

Bermant, R. I. and Welch, R. B. (1976). Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition, Percept. Mot. Skills 42, 487–493.

Bertelson, P. and Aschersleben, G. (1998). Automatic visual bias of perceived auditory location, Psychon. Bull. Rev. 5, 482–489.10.3758/BF03208826

Bertelson, P., Vroomen, J., de Gelder, B. and Driver, J. (2000). The ventriloquist effect does not depend on the direction of deliberate visual attention, Percept. Psychophys. 62, 321–332.10.3758/BF03205552

Bosmana, A. J. and Smoorenburg, G. F. (1995). Intelligibility of Dutch CVC syllables and sentences for listeners with normal hearing and with three types of hearing impairment, Audiology 34, 260–284.10.3109/00206099509071918

Chen, L. and Vroomen, J. (2013). Intersensory binding across space and time: a tutorial review, Atten. Percept. Psychophys. 75, 790–811.10.3758/s13414-013-0475-4

Cienkowski, K. M. and Carney, A. E. (2002). Auditory-visual speech perception and aging, Ear Hear. 23, 439–449.10.1097/00003446-200210000-00006

Colonius, H. and Arndt, P. (2001). A two-stage model for visual–auditory interaction in saccadic latencies, Percept. Psychophys. 63, 126–147.10.3758/BF03200508

Couth, S., Gowen, E. and Poliakoff, E. (2018). Using race model violation to explore multisensory responses in older adults: enhanced multisensory integration or slower unisensory processing?, Multisens. Res. 31, 151–174.10.1163/22134808-00002588

de Boer-Schellekens, L. and Vroomen, J. (2014). Multisensory integration compensates loss of sensitivity of visual temporal order in the elderly, Exp. Brain Res. 232, 253–262.10.1007/s00221-013-3736-5

de Dieuleveult, A. L., Siemonsma, P. C., van Erp, J. B. F. and Brouwer, A.-M. (2017). Effects of aging in multisensory integration: a systematic review, Front. Aging Neurosci. 28, 80. DOI:10.3389/fnagi.2017.00080.

de Dieuleveult, A. L., Perry, S., Siemonsma, P. C., Brouwer, A. M. and van Erp, J. B. F. (2019). A simple target interception task as test for activities of daily life performance in older adults, Front. Neurosci. 13, 524. DOI:10.3389/fnins.2019.00524.

de Gelder, B. and Bertelson, P. (2003). Multisensory integration, perception and ecological validity, Tr. Cogn. Sci. 7, 460–467.10.1016/j.tics.2003.08.014

Diederich, A., Colonius, H. and Schomburg, A. (2008). Assessing age-related multisensory enhancement with the time-window-of-integration model, Neuropsychologia 46, 2556–2562.10.1016/j.neuropsychologia.2008.03.026

Driver, J. (1996). Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading, Nature 381, 66–68.10.1038/381066a0

Erber, N. P. (1975). Auditory-visual perception of speech, J. Speech Hear. Disrd. 40, 481–492.10.1044/jshd.4004.481

Ernst, M. O. and Bülthoff, H. H. (2004). Merging the senses into a robust percept, Trends Cogn. Sci. 8, 162–169.10.1016/j.tics.2004.02.002

Files, B. T., Tjan, B. S., Jiang, J. and Bernstein, L. E. (2015). Visual speech discrimination and identification of natural and synthetic consonant stimuli, Front. Psychol. 6, 878. DOI:10.3389/fpsyg.2015.00878.

Fogerty, D., Humes, L. E. and Busey, T. A. (2016). Age-related declines in early sensory memory: identification of rapid auditory and visual stimulus sequences, Front. Aging Neurosci. 8, 90. DOI:10.3389/fnagi.2016.00090.

Fozard, J. L. (1990). Vision and hearing in aging, in: Handbook of the Psychology of Aging, J. E. Birren and K. W. Schaie (Eds), pp. 150–170. Academic Press, New York, NY, USA.

Freiherr, J., Lundström, J. N., Habel, U. and Reetz, K. (2013). Multisensory integration mechanisms during aging, Front. Hum. Neurosci. 7, 863. DOI:10.3389/fnhum.2013.00863.

Gardner, W. G. and Martin, K. D. (1995). HRTF measurements of a KEMAR, J. Acoust. Soc. Am. 97, 3907. DOI:10.1121/1.412407.10.1121/1.412407

Grant, K. W. and Seitz, P. F. (1998). Measures of auditory–visual integration in nonsense syllables and sentences, J. Acoust. Soc. Am. 104, 2438–2450.10.1121/1.423751

Hay-McCutcheon, M. J., Pisoni, D. B. and Hunt, K. K. (2009). Audiovisual asynchrony detection and speech perception in hearing-impaired listeners with cochlear implants: a preliminary analysis, Int. J. Audiol. 48, 321–333.10.1080/14992020802644871

Helfer, K. S. (1998). Auditory and auditory–visual recognition of clear and conversational speech by older adults, J. Am. Acad. Audiol. 9, 234–242.

Hoffman, H. J., Dobie, R. A., Ko, C.-W., Themann, C. L. and Murphy, W. J. (2012). Hearing threshold levels at age 70 years (65–74 years) in the unscreened older adult population of the United States, 1959–1962 and 1999–2006, Ear Hear. 33, 437–440.10.1097/AUD.0b013e3182362790

Holmes, N. P. (2009). The principle of inverse effectiveness in multisensory integration: some statistical considerations, Brain Topogr. 21, 168–176.10.1007/s10548-009-0097-2

Katz, J. (2014). Handbook of Clinical Audiology, 5th edn. Lippincott, Williams, and Wilkins, Philadelphia, PA, USA.

Lalonde, K. and Holt, R. F. (2016). Audiovisual speech perception development at varying levels of perceptual processing, J. Acoust. Soc. Am. 139, 1713–1723.10.1121/1.4945590

Langendijk, E. H. A. and Bronkhorst, A. W. (2002). Contribution of spectral cues to human sound localization, J. Acoust. Soc. Am. 112, 1583–1596.10.1121/1.1501901

Laurienti, P. J., Burdette, J. H., Maldjian, J. A. and Wallace, M. T. (2006). Enhanced multisensory integration in older adults, Neurobiol. Aging 27, 1155–1163.10.1016/j.neurobiolaging.2005.05.024

Levitt, H. (1971). Transformed up-down methods in psychoacoustics, J. Acoust. Soc. Am. 49, 467–477.10.1121/1.1912375

Lovelace, C. T., Stein, B. E. and Wallace, M. T. (2003). An irrelevant light enhances auditory detection in humans: a psychophysical analysis of multisensory integration in stimulus detection, Cogn. Brain Res. 17, 447–453.10.1016/S0926-6410(03)00160-5

Macpherson, E. A. and Middlebrooks, J. C. (2002). Listener weighting of cues for lateral angle: the duplex theory of sound localization revisited, J. Acoust. Soc. Am. 111, 2219–2236.10.1121/1.1471898

Mahoney, J. R., Holtzer, R. and Verghese, J. (2014). Visual-somatosensory integration and balance: evidence for psychophysical integrative differences in aging, Multisens. Res. 27, 17–42.10.1163/22134808-00002444

Majdak, P., Iwaya, Y., Carpentier, T., Nicol, R., Parmentier, M., Roginska, A., Suzuki, Y., Watanabe, K., Wierstorf, H., Ziegelwanger, H. and Noisternig, M. (2013). Spatially oriented format for acoustics: a data exchange format representing head-related transfer functions, in: Proceedings of the 134th Convention of the Audio Engineering Society (AES) Roma, Italy, Convention Paper 8880.

Massaro, D. W., Cohen, M. M. and Smeele, P. M. T. (1996). Perception of asynchronous and conflicting visual and auditory speech, J. Acoust. Soc. Am. 100, 1777–1786.10.1121/1.417342

Mattys, S. L., Davis, M. H., Bradlow, A. R. and Scott, S. K. (2012). Speech recognition in adverse conditions: a review, Lang. Cogn. Process. 27, 953–978.10.1080/01690965.2012.705006

McGurk, H. and MacDonald, J. (1976). Hearing lips and seeing voices, Nature 264, 746–748.10.1038/264746a0

Middlebrooks, J. C. and Green, D. M. (1991). Sound localization by human listeners, Annu. Rev. Psychol. 42, 135–159.10.1146/annurev.ps.42.020191.001031

Musacchia, G., Arum, L., Nicol, T., Garstecki, D. and Kraus, N. (2009). Audiovisual deficits in older adults with hearing loss: biological evidence, Ear Hear. 30, 505–514.10.1097/AUD.0b013e3181a7f5b7

Oppenheim, A. V., Schafer, R. W. and Buck, J. R. (1999). Discrete-Time Signal Processing, 2nd edn. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.

Otte, R. J., Agterberg, M. J. H., Van Wanrooij, M. M., Snik, A. F. M. and van Opstal, J. (2013). Age-related hearing loss and ear morphology affect vertical but not horizontal sound-localization performance, J. Assoc. Res. Otolaryngol. 14, 261–273.10.1007/s10162-012-0367-7

Peiffer, A. M., Mozolic, J. L., Hugenschmidt, C. E. and Laurienti, P. J. (2007). Age-related multisensory enhancement in a simple audiovisual detection task, Neuroreport 18, 1077–1081.10.1097/WNR.0b013e3281e72ae7

Perrott, D. R. and Saberi, K. (1990). Minimum audible angle thresholds for sources varying in both elevation and azimuth, J. Acoust. Soc. Am. 87, 1728–1731.10.1121/1.399421

Pichora-Fuller, M. K., Schneider, B. A. and Daneman, M. (1995). How young and old adults listen to and remember speech in noise, J. Acoust. Soc. Am. 97, 593–608.10.1121/1.412282

Pick, H. L., Warren, D. H. and Hay, J. C. (1969). Sensory conflict in judgments of spatial direction, Percept. Psychophys. 6, 203–205.10.3758/BF03207017

Rosenblum, L. D. and Saldaña, H. M. (1996). An audiovisual test of kinematic primitives for visual speech perception, J. Exp. Psychol. Hum. Percept. Perform. 22, 318–331.10.1037/0096-1523.22.2.318

Saija, J. D., Akyürek, E. G., Andringa, T. C. and Başkent, D. (2014). Perceptual restoration of degraded speech is preserved with advancing age, J. Assoc. Res. Otolaryngol. 15, 139–148.10.1007/s10162-013-0422-z

Saija, J. D., Başkent, D., Andringa, T. C. and Akyürek, E. G. (2019). Visual and auditory temporal integration in healthy younger and older adults, Psychol. Res. 83, 951–967.10.1007/s00426-017-0912-4

Salthouse, T. A. (1996). The processing-speed theory of adult age differences in cognition, Psychol. Rev. 103, 403–428.10.1037/0033-295X.103.3.403

Setti, A., Burke, K. E., Kenny, R. A. and Newell, F. N. (2011). Is inefficient multisensory processing associated with falls in older people?, Exp. Brain Res. 209, 375–384.10.1007/s00221-011-2560-z

Shaw, E. A. G. (1974). Transformation of sound pressure level form the free field to the eardrum in the horizontal plane, J. Acoust. Soc. Am. 56, 1848–1861.10.1121/1.1903522

Sommers, M. S., Tye-Murray, N. and Spehar, B. (2005). Auditory-visual speech perception and auditory-visual enhancement in normal-hearing younger and older adults, Ear Hear. 26, 263–275.10.1097/00003446-200506000-00003

Søndergaard, P. L. and Majdak, P. (2013). The auditory modeling toolbox, in: The Technology of Binaural Listening, Modern Acoustics and Signal Processing, J. Blauert (Ed.), pp. 397–425. Springer-Verlag, Heidelberg, Berlin, Germany. Available at: http://link.springer.com/10.1007/978-3-642-37762-4.

Stein, B. E. and Meredith, M. A. (1993). The Merging of the Senses. MIT Press, Cambridge, MA, USA.

Stevenson, R. A., Nelms, C. E., Baum, S. H., Zurkovsky, L., Barense, M. D., Newhouse, P. A. and Wallace, M. T. (2015). Deficits in audiovisual speech perception in normal aging emerge at the level of whole-word recognition, Neurobiol. Aging 36, 283–291.10.1016/j.neurobiolaging.2014.08.003

Stevenson, R. A. and Wallace, M. T. (2013). Multisensory temporal integration: task and stimulus dependencies, Exp. Brain Res. 227, 249–261.10.1007/s00221-013-3507-3

Strouse, A., Ashmead, D. H., Ohde, R. N. and Grantham, D. W. (1998). Temporal processing in the aging auditory system, J. Acoust. Soc. Am. 104, 2385–2399.10.1121/1.423748

Tremblay, C., Champoux, F., Voss, P., Bacon, B. A., Lepore, F. and Théoret, H. (2007). Speech and non-speech audio-visual illusions: a developmental study, PLoS ONE 2(8), e742. DOI:10.1371/journal.pone.0000742.

Tuomainen, J., Andersen, T. S., Tiippana, K. and Sams, M. (2005). Audio-visual speech perception is special, Cognition 96, B13–B22.10.1016/j.cognition.2004.10.004

Tye-Murray, N., Sommers, M. S. and Spehar, B. (2007). Audiovisual integration and lipreading abilities of older adults with normal and impaired hearing, Ear Hear. 28, 656–668.10.1097/AUD.0b013e31812f7185

Tye-Murray, N., Sommers, M., Spehar, B., Myerson, J., Hale, S. and Rose, N. S. (2008). Auditory-visual discourse comprehension by older and young adults in favorable and unfavorable conditions, Int. J. Audiol. 47, S31–S37.10.1080/14992020802301662

Tye-Murray, N., Sommers, M., Spehar, B., Myerson, J. and Hale, S. (2010). Aging, audiovisual integration, and the principle of inverse effectiveness, Ear Hear. 31, 636–644.10.1097/AUD.0b013e3181ddf7ff

Van Opstal, A. J. (2016). The Auditory System and Human Sound-Localization Behavior. Academic Press, London, UK.

Vatakis, A. and Spence, C. (2006). Audiovisual synchrony perception for music, speech, and object actions, Brain Res. 1111, 134–142.10.1016/j.brainres.2006.05.078

Vroomen, J. and de Gelder, B. (2004). Perceptual effects of cross-modal stimulation: ventriloquism and the freezing phenomenon, in: The Handbook of Multisensory Processes, G. A. Calvert, C. Spence and B. E. Stein (Eds), pp. 141–150. The MIT Press, Cambridge, MA, USA.

Vroomen, J., Bertelson, P. and de Gelder, B. (1998). A visual influence in the discrimination of auditory location, in: Proceedings of the International Conference on Auditory–Visual Speech Processing (AVSP’98), D. Burnham, J. Robert-Ribes and E. Vatikiotis-Bateson (Eds), pp. 131–134. Causal Productions, Adelaide, South Australia, Australia.

Vroomen, J., Bertelson, P. and de Gelder, B. (2001). The ventriloquist effect does not depend on the direction of automatic visual attention, Percept. Psychophys. 63, 651–659.10.3758/BF03194427

Wenzel, E. M., Arruda, M., Kistler, D. J. and Wightman, F. L. (1993). Localization using nonindividualized head-related transfer functions, J. Acoust. Soc. Am. 94, 111–123.10.1121/1.407089

Wightman, F. L. and Kistler, D. J. (1989). Headphone simulation of free-field listening. II: psychophysical validation, J. Acoust. Soc. Am. 85, 868–878.10.1121/1.397558

Wild, C. J., Yusuf, A., Wilson, D. E., Peelle, J. E., Davis, M. H. and Johnsrude, I. S. (2012). Effortful listening: the processing of degraded speech depends critically on attention, J. Neurosci. 32, 14010–14021.10.1523/JNEUROSCI.1528-12.2012

Ziegelwanger, H. and Majdak, P. (2014). Modeling the direction-continuous time-of-arrival in head-related transfer functions, J. Acoust. Soc. Am. 135, 1278–1293.10.1121/1.4863196