Cross2SynNet: cross-device–cross-modal synthesis of routine brain MRI sequences from CT with brain lesion

Dill T (2008) Contraindications to magnetic resonance imaging. Heart 94(7):943–948 Oliveri S, Pricolo P, Pizzoli S, Faccio F, Lampis V, Summers P, Petralia G, Pravettoni G (2018) Investigating cancer patient acceptance of Whole Body MRI. Clin Imaging 52:246–251 Jue J, Jason H, Neelam T, Andreas R, Sean BL, Joseph DO, Harini V (2019) Integrating cross-modality hallucinated MRI with CT to aid mediastinal lung tumor segmentation. Medical image computing and computer assisted intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part VI 22. Springer, Berlin, pp 221–229 Saba L, Anzidei M, Piga M, Ciolina F, Mannelli L, Catalano C, Suri JS, Raz E (2014) Multi-modal CT scanning in the evaluation of cerebrovascular disease patients. Cardiovasc Diag Therapy 4(3):245 Boulanger M, Nunes J-C, Chourak H, Largent A, Tahri S, Acosta O, De Crevoisier R, Lafond C, Barateau A (2021) Deep learning methods to generate synthetic CT from MRI in radiotherapy: a literature review. Phys Med 89:265–281 Kong L, Lian C, Huang D, Hu Y, Zhou Q (2021) Breaking the dilemma of medical image-to-image translation. Adv Neural Inform Process Syst 34:1964–1978 Qin Z, Liu Z, Zhu P, Ling W (2022) Style transfer in conditional GANs for cross-modality synthesis of brain magnetic resonance images. Comput Biol Med 148:105928 Dar SU, Yurt M, Karacan L, Erdem A, Erdem E, Cukur T (2019) Image synthesis in multi-contrast MRI with conditional generative adversarial networks. IEEE Trans Med Imaging 38(10):2375–2388 Benzakoun J, Deslys M-A, Legrand L, Hmeydia G, Turc G, Hassen WB, Charron S, Debacker C, Naggara O, Baron J-C (2022) Synthetic FLAIR as a substitute for FLAIR sequence in acute ischemic stroke. Radiology 303(1):153–159 Kalantar R, Messiou C, Winfield JM, Renn A, Latifoltojar A, Downey K, Sohaib A, Lalondrelle S, Koh D-M, Blackledge MD (2021) Ct-based pelvic t1-weighted mr image synthesis using unet, unet++ and cycle-consistent generative adversarial network (cycle-gan). Front Oncol 11:665807 Chen RJ, Lu MY, Chen TY, Williamson DF, Mahmood F (2021) Synthetic data in machine learning for medicine and healthcare. Nat Biomed Eng 5(6):493–497 Li W, Li Y, Qin W, Liang X, Xu J, Xiong J, Xie Y (2020) Magnetic resonance image (MRI) synthesis from brain computed tomography (CT) images based on deep learning methods for magnetic resonance (MR)-guided radiotherapy. Quant Imaging Med Surg 10(6):1223 Hu N, Zhang T, Wu Y, Tang B, Li M, Song B, Gong Q, Wu M, Gu S, Lui S (2022) Detecting brain lesions in suspected acute ischemic stroke with CT-based synthetic MRI using generative adversarial networks. Ann Transl Med 10(2):35 Feng E, Qin P, Chai R, Zeng J, Wang Q, Meng Y, Wang P (2022) MRI generated from CT for acute ischemic stroke combining radiomics and generative adversarial networks. IEEE J Biomed Health Inform 26(12):6047–6057 Costa P, Galdran A, Meyer MI, Niemeijer M, Abràmoff M, Mendonça AM, Campilho A (2017) End-to-end adversarial retinal image synthesis. IEEE Trans Med Imaging 37(3):781–791 Tragakis A, Kaul C, Murray-Smith R, Husmeier D (2023) The fully convolutional transformer for medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp 3660–3669 Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18. Springer, Berlin, pp 234–241 Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1125–1134 Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:14111784 Nie D, Trullo R, Lian J, Wang L, Petitjean C, Ruan S, Wang Q, Shen D (2018) Medical image synthesis with deep convolutional adversarial networks. IEEE Trans Biomed Eng 65(12):2720–2730 Klein S, Staring M, Murphy K, Viergever MA, Pluim JP (2009) Elastix: a toolbox for intensity-based medical image registration. IEEE Trans Med Imaging 29(1):196–205 Orlenko A, Kofink D, Lyytikäinen L-P, Nikus K, Mishra P, Kuukasjärvi P, Karhunen PJ, Kähönen M, Laurikka JO, Lehtimäki T (2020) Model selection for metabolomics: predicting diagnosis of coronary artery disease using automated machine learning. Bioinformatics 36(6):1772–1778 He J, You H, Sandström E, Nittinger E, Bjerrum EJ, Tyrchan C, Czechtizky W, Engkvist O (2021) Molecular optimization by capturing chemist’s intuition using deep neural networks. J Cheminform 13(1):1–17 Weissenbacher D, Ge S, Klein A, O’Connor K, Gross R, Hennessy S, Gonzalez-Hernandez G (2021) Active neural networks to detect mentions of changes to medication treatment in social media. J Am Med Inform Assoc 28(12):2551–2561 Yu B, Zhou L, Wang L, Shi Y, Fripp J, Bourgeat P (2019) Ea-GANs: edge-aware generative adversarial networks for cross-modality MR image synthesis. IEEE Trans Med Imaging 38(7):1750–1762 Dalmaz O, Yurt M, Çukur T (2022) ResViT: residual vision transformers for multimodal medical image synthesis. IEEE Trans Med Imaging 41(10):2598–2614 Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B (2018) High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 8798–8807 Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:14126980 Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612 Zhu J-Y, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on computer vision. pp 2223–2232 Jiang J, Hu YC, Tyagi N, Zhang P, Rimner A, Deasy JO, Veeraraghavan H (2019) Cross-modality (CT-MRI) prior augmented deep learning for robust lung tumor segmentation from small MR datasets. Med Phys 46(10):4392–4404 O’Connor JP (2017) Rethinking the role of clinical imaging. Elife 6:e30563 Liu Y, Chen A, Shi H, Huang S, Zheng W, Liu Z, Zhang Q, Yang X (2021) CT synthesis from MRI using multi-cycle GAN for head-and-neck radiation therapy. Comput Med Imaging Graph 91:101953 Bahrami A, Karimian A, Arabi H (2021) Comparison of different deep learning architectures for synthetic CT generation from MR images. Phys Med 90:99–107 Yurt M, Dar SU, Erdem A, Erdem E, Oguz KK, Çukur T (2021) mustGAN: multi-stream generative adversarial networks for MR image synthesis. Med Image Anal 70:101944 Liu J, Pasumarthi S, Duffy B, Gong E, Datta K, Zaharchuk G (2023) One model to synthesize them all: multi-contrast multi-scale transformer for missing data imputation. IEEE Trans Med Imaging 42(9):2577–2591 Zhang X, He X, Guo J, Ettehadi N, Aw N, Semanek D, Posner J, Laine A, Wang Y (2021) PTNet: a high-resolution infant MRI synthesizer based on transformer. arXiv preprint arXiv:210513993 Dorent R, Haouchine N, Kogl F, Joutard S, Juvekar P, Torio E, Golby AJ, Ourselin S, Frisken S, Vercauteren T (2023) Unified brain MR-ultrasound synthesis using multi-modal hierarchical representations. International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 448–458 Oh H-J, Jeong W-K (2023) DiffMix: diffusion model-based data synthesis for nuclei segmentation and classification in imbalanced pathology image datasets. arXiv preprint arXiv:230614132 Du Y, Jiang Y, Tan S, Wu X, Dou Q, Li Z, Li G, Wan X (2023) ArSDM: colonoscopy images synthesis with adaptive refinement semantic diffusion models. International conference on medical image computing and computer-assisted intervention. Springer, Berlin, pp 339–349 Özbey M, Dalmaz O, Dar SU, Bedel HA, Özturk Ş, Güngör A, Çukur T (2023) Unsupervised medical image translation with adversarial diffusion models. IEEE Trans Med Imaging 42(12):3524–3539 Wu H, Xiao B, Codella N, Liu M, Dai X, Yuan L, Zhang L (2021) Cvt: introducing convolutions to vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 22–31 Guo J, Han K, Wu H, Tang Y, Chen X, Wang Y, Xu C (2022) Cmt: convolutional neural networks meet vision transformers. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition. pp 12175–12185 Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. Adv Neural Inf Process Syst 30:5998–6008 Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:201011929 Chen J, Lu Y, Yu Q, Luo X, Adeli E, Wang Y, Lu L, Yuille AL, Zhou Y (2021) Transunet: transformers make strong encoders for medical image segmentation. arXiv preprint arXiv:210204306 Cao H, Wang Y, Chen J, Jiang D, Zhang X, Tian Q, Wang M (2022) Swin-unet: Unet-like pure transformer for medical image segmentation. European Conference on computer vision. Springer, Berlin, pp 205–218 Nie D, Shen D (2020) Adversarial confidence learning for medical image segmentation and synthesis. Int J Comput Vision 128:2494–2513 Augustin M, Bammer R, Simbrunner J, Stollberger R, Hartung H-P, Fazekas F (2000) Diffusion-weighted imaging of patients with subacute cerebral ischemia: comparison with conventional and contrast-enhanced MR imaging. Am J Neuroradiol 21(9):1596–1602