Cardiac murmur grading and risk analysis of cardiac diseases based on adaptable heterogeneous-modality multi-task learning
Tóm tắt
Cardiovascular disease (CVDs) has become one of the leading causes of death, posing a significant threat to human life. The development of reliable Artificial Intelligence (AI) assisted diagnosis algorithms for cardiac sounds is of great significance for early detection and treatment of CVDs. However, there is scarce research in this field. Existing research mainly faces three major challenges: (1) They mainly limited to murmur classification and cannot achieve murmur grading, but attempting both classification and grading may lead to negative effects between different multi-tasks. (2) They mostly pay attention to unstructured cardiac sound modality and do not consider the structured demographic modality, as it is difficult to balance the influence of heterogeneous modalities. (3) Deep learning methods lack interpretability, which makes it challenging to apply them clinically. To tackle these challenges, we propose a method for cardiac murmur grading and cardiac risk analysis based on heterogeneous modality adaptive multi-task learning. Specifically, a Hierarchical Multi-Task learning-based cardiac murmur detection and grading method (HMT) is proposed to prevent negative interference between different tasks. In addition, a cardiac risk analysis method based on Heterogeneous Multi-modal feature impact Adaptation (HMA) is also proposed, which transforms unstructured modality into structured modality representation, and utilizes an adaptive mode weight learning mechanism to balance the impact between unstructured modality and structured modality, thus enhancing the performance of cardiac risk prediction. Finally, we propose a multi-task interpretability learning module that incorporates an important evaluation using random masks. This module utilizes SHAP graphs to visualize crucial murmur segments in cardiac sound and employs a multi-factor risk decoupling model based on nomograms. And then we gain insights into the cardiac disease risk in both pre-decoupled multi-modality and post-decoupled single-modality scenarios, thus providing a solid foundation for AI assisted cardiac murmur grading and risk analysis. Experimental results on a large real-world CirCor DigiScope PCG dataset demonstrate that the proposed method outperforms the state-of-the-art (SOTA) method in murmur detection, grading, and cardiac risk analysis, while also providing valuable diagnostic evidence.
Tài liệu tham khảo
Qiu D, Cheng Y, Wang X. Dual u-net residual networks for cardiac magnetic resonance images super-resolution. Comput Methods Programs Biomed. 2022;218:106707. https://doi.org/10.1016/j.cmpb.2022.106707.
Weber C, Noels H. Atherosclerosis: current pathogenesis and therapeutic options. Nat Med. 2011;17(11):1410–22.
Zhang D, Chen Y, Chen Y, Ye S, Cai W, Jiang J, Xu Y, Zheng G, Chen M. Heart disease prediction based on the embedded feature selection method and deep neural network. J Healthcare Eng. 2021;2021:1–9.
Shimizu I, Minamino T. Physiological and pathological cardiac hypertrophy. J Mol Cell Cardiol. 2016;97:245–62.
Buja LM. Myocardial ischemia and reperfusion injury. Cardiovasc Pathol. 2005;14(4):170–5.
Coffey S, Cairns BJ, Iung B. The modern epidemiology of heart valve disease. Heart. 2016;102(1):75–85.
Wang F, Syeda-Mahmood T, Beymer D. Finding disease similarity by combining ecg with heart auscultation sound. In: 2007 Computers in Cardiology, 2007; pp. 261–264. IEEE
Reed TR, Reed NE, Fritzson P. Heart sound analysis for symptom detection and computer-aided diagnosis. Simul Model Pract Theory. 2004;12(2):129–46.
Randhawa SK, Singh M. Classification of heart sound signals using multi-modal features. Procedia Computer Science 2015; 58, 165–171. https://doi.org/10.1016/j.procs.2015.08.045 . Second International Symposium on Computer Vision and the Internet (VisionNet’15)
Mustafa M, Abdalla G, Manimurugan S, Alharbi AR. Detection of heartbeat sounds arrhythmia using automatic spectral methods and cardiac auscultatory. J Supercomput. 2020;76:5899–922.
Liu J, Wang H, Yang Z, Quan J, Liu L, Tian J. Deep learning-based computer-aided heart sound analysis in children with left-to-right shunt congenital heart disease. Int J Cardiol. 2022;348:58–64. https://doi.org/10.1016/j.ijcard.2021.12.012.
Davidsen AH, Andersen S, Halvorsen PA, Schirmer H, Reierth E, Melbye H. Diagnostic accuracy of heart auscultation for detecting valve disease: a systematic review. BMJ Open. 2023;13(3): 068121.
Mangione S, Nieman LZ, Gracely E, Kaye D. The teaching and practice of cardiac auscultation during internal medicine and cardiology training: a nationwide survey. Ann Intern Med. 1993;119(1):47–54.
Voigt I, Boeckmann M, Bruder O, Wolf A, Schmitz T, Wieneke H. A deep neural network using audio files for detection of aortic stenosis. Clin Cardiol. 2022;45(6):657–63.
Levin AD, Ragazzi A, Szot SL, Ning T. Extraction and assessment of diagnosis-relevant features for heart murmur classification. Methods. 2022;202:110–6.
Biancaniello T. Innocent murmurs. Circulation. 2005;111(3):20–2.
Elola A, Aramendi E, Oliveira J, Renna F, Coimbra MT, Reyna MA, Sameni R, Clifford GD, Rad AB. Beyond heart murmur detection: automatic murmur grading from phonocardiogram. IEEE J Biomed Health Inform. 2023. https://doi.org/10.1109/JBHI.2023.3275039.
Vandenhende S, Georgoulis S, Van Gansbeke W, Proesmans M, Dai D, Van Gool L. Multi-task learning for dense prediction tasks: a survey. IEEE Trans Pattern Anal Mach Intell. 2021;44(7):3614–33.
Hao Y, Usama M, Yang J, Hossain MS, Ghoneim A. Recurrent convolutional neural network based multimodal disease risk prediction. Futur Gener Comput Syst. 2019;92:76–83.
Safara F, Doraisamy S, Azman A, Jantan A, Ramaiah ARA. Multi-level basis selection of wavelet packet decomposition tree for heart sound classification. Comput Biol Med. 2013;43(10):1407–14.
Debbal S, Bereksi-Reguig F. Computerized heart sounds analysis. Comput Biol Med. 2008;38(2):263–80.
Rath A, Mishra D, Panda G, Pal M. Development and assessment of machine learning based heart disease detection using imbalanced heart sound signal. Biomed Signal Process Control. 2022;76: 103730.
Zeinali Y, Niaki STA. Heart sound classification using signal processing and machine learning algorithms. Mach Learn Appl. 2022;7: 100206.
Chen K, Mudvari A, Barrera FG, Cheng L, Ning T. Heart murmurs clustering using machine learning. In: 2018 14th IEEE International Conference on Signal Processing (ICSP), pp. 94–98 (2018). IEEE
Delgado-Trejos E, Quiceno-Manrique A, Godino-Llorente J, Blanco-Velasco M, Castellanos-Dominguez G. Digital auscultation analysis for heart murmur detection. Ann Biomed Eng. 2009;37:337–53.
Kotb MA, Nabih H, El Zahraa F, El Falaki M, Shaker CW, Refaey MA, Rjoob K. Improving the recognition of heart murmur. Int J Adv Comput Sci Appl. 2016;7(7):283–7.
Kotb MA, Elmahdy HN, Mostafa FEZ, Shaker CW, Refaey MA, Rjoob KWY. Recognition of heart murmur based on machine learning and visual based analysis of phonocardiography. In: Intelligent Computing: Proceedings of the 2018 Computing Conference, 2019; Volume 2, pp. 188–202. Springer
Xiao B, Xu Y, Bi X, Zhang J, Ma X. Heart sounds classification using a novel 1-d convolutional neural network with extremely low parameter consumption. Neurocomputing. 2020;392:153–9.
Patwa A, Rahman MMU, Al-Naffouri TY. Heart murmur and abnormal pcg detection via wavelet scattering transform & a 1d-cnn. arXiv preprint arXiv:2303.11423 (2023).
Oh SL, Jahmunah V, Ooi CP, Tan R-S, Ciaccio EJ, Yamakawa T, Tanabe M, Kobayashi M, Acharya UR. Classification of heart sound signals using a novel deep wavenet model. Comput Methods Programs Biomed. 2020;196: 105604.
Venkataramani VV, Garg A, Priyakumar UD. Modified variable kernel length resnets for heart murmur detection and clinical outcome prediction using multi-positional phonocardiogram recording
Raza A, Mehmood A, Ullah S, Ahmad M, Choi GS, On B-W. Heartbeat sound signal classification using deep learning. Sensors. 2019;19(21):4819.
Li J, Ke L, Du Q, Ding X, Chen X. Research on the classification of ecg and pcg signals based on bilstm-googlenet-ds. Appl Sci. 2022;12(22):11762.
McDonald A, Gales MJ, Agarwal A. Detection of heart murmurs in phonocardiograms with parallel hidden semi-markov models. In: 2022 Computing in Cardiology (CinC), 2022 ; vol. 498, pp. 1–4. IEEE
Wang Z-H, Horng G-J, Hsu T-H, Aripriharta A, Jong G-J. Heart sound signal recovery based on time series signal prediction using a recurrent neural network in the long short-term memory model. J Supercomput. 2020;76:8373–90.
Freeman A, LEVINE SA. The clinical significance of the systolic murmur: a study of 1000 consecutive “non-cardiac" cases. Ann Intern Med. 1933;6(11):1371–85.
He Y, Li W, Zhang W, Zhang S, Pi X, Liu H. Research on segmentation and classification of heart sound signals based on deep learning. Appl Sci. 2021;11(2):651.
Bondareva E, Xia T, Han J, Mascolo C. Towards uncertainty-aware murmur detection in heart sounds via tandem learning. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Araujo M, Zeng D, Palotti J, Xi X, Shi Y, Pyles L, Ni Q. Maiby’s algorithm: A two-stage deep learning approach for murmur detection in mel spectrograms for automatic auscultation of congenital heart disease. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Gündüz AF, Fatih T. Pcg frame classification by classical machine learning methods using spectral features and mfcc based features. Avrupa Bilim ve Teknoloji Dergisi. 2022;42:77–82.
Chang Y, Liu L, Antonescu C. Multi-task prediction of murmur and outcome from heart sound recordings. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. https://doi.org/10.22489/CinC.2022.309
Keren R, Tereschuk M, Luan X. Evaluation of a novel method for grading heart murmur intensity. Arch Pediatr Adolesc Med. 2005;159(4):329–34. https://doi.org/10.1001/archpedi.159.4.329.
Oliveira J, Renna F, Costa P, Nogueira M, Oliveira AC, Elola A, Ferreira C, Jorge A, Rad AB, Reyna M, et al. The circor digiscope phonocardiogram dataset. version 1.0. 0 (2022)
Oliveira J, Renna F, Costa PD, Nogueira M, Oliveira C, Ferreira C, Jorge A, Mattos S, Hatem T, Tavares T, Elola A, Rad AB, Sameni R, Clifford GD, Coimbra MT. The circor digiscope dataset: From murmur detection to murmur classification. IEEE J Biomed Health Inform. 2022;26(6):2524–35. https://doi.org/10.1109/JBHI.2021.3137048.
Cornely AK, Mirsky GM. Heart murmur detection using wavelet time scattering and support vector machines. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Imran Z, Grooby E, Malgi VV, Sitaula C, Aryal S, Marzbanrad F. A fusion of handcrafted feature-based and deep learning classifiers for heart murmur detection. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Kim J, Park G, Suh B. Classification of phonocardiogram recordings using vision transformer architecture. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. https://doi.org/10.22489/CinC.2022.084
Chang Y, Liu L, Antonescu C. Multi-task prediction of murmur and outcome from heart sound recordings. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Ballas A, Papapanagiotou V, Delopoulos A, Diou C. Listen2yourheart: A self-supervised approach for detecting murmur in heart-beat sounds. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Lee J, Kang T, Kim N, Han S, Won H, Gong W, Kwak I-Y. Deep learning based heart murmur detection using frequency-time domain features of heartbeat sounds. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Lu H, Yip JB, Steigleder T, Grießhammer S, Heckel M, Jami NVSJ, Eskofier B, Ostgathe C, Koelpin A. A lightweight robust approach for automatic heart murmurs and clinical outcomes classification from phonocardiogram recordings. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Bruoth E, Bugata P, Gajdoš D, Hudák D, Kmečová V, Staňková M, Szabari A, Vozáriková G, et al.. Murmur identification using supervised contrastive learning. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Xu Y, Bao X, Lam H-K, Kamavuako EN. Hierarchical multi-scale convolutional network for murmurs detection on pcg signals. In: 2022 Computing in Cardiology (CinC), 2022; vol. 498, pp. 1–4. IEEE
Testa A, Gallo D, Langella R. On the processing of harmonics and interharmonics: using Hanning window in standard framework. IEEE Trans Power Deliv. 2004;19(1):28–34. https://doi.org/10.1109/TPWRD.2003.820437.
Hong H, Hong S. simplenomo: a python package of making nomograms for visualizable calculation of logistic regression models. Health Data Sci. 2023;3:0023.