Tổng quát hóa sâu được hướng dẫn bởi vật lý cho cảm biến nén

Springer Science and Business Media LLC - Tập 131 - Trang 2864-2887 - 2023
Bin Chen1, Jiechong Song1, Jingfen Xie1, Jian Zhang1
1Peking University Shenzhen Graduate School, Shenzhen, China

Tóm tắt

Bằng cách hấp thụ những ưu điểm của cả phương pháp dựa trên mô hình và dữ liệu, sơ đồ học sâu gắn kết với vật lý đạt được việc tái tạo hình ảnh với độ chính xác cao và có thể giải thích. Nó đã thu hút sự chú ý ngày càng tăng và trở thành xu hướng chính cho các nhiệm vụ hình ảnh ngược. Tập trung vào vấn đề cảm biến nén hình ảnh (CS), chúng tôi phát hiện ra thiếu sót nội tại của khuôn mẫu đang nổi này, được triển khai rộng rãi bởi các mạng lưới thuật toán sâu cuốn lại, trong đó nhiều vòng lặp đơn giản liên quan đến vật lý thực sẽ dẫn đến chi phí tính toán khổng lồ và thời gian suy luận dài, cản trở ứng dụng thực tiễn của chúng. Một khuôn khổ Mô hình Hồi phục Hướng dẫn bởi Vật lý sâu (RL) mới được đề xuất bằng cách tổng quát hóa mô hình hồi phục lặp đi lặp lại truyền thống từ miền hình ảnh (ID) sang miền đặc trưng nhiều chiều (FD). Sau đó, một kiến trúc cuộn đa tỉ lệ gọn nhẹ được phát triển để nâng cao khả năng của mạng lưới và duy trì tốc độ suy luận theo thời gian thực. Lấy hai góc độ khác nhau của tối ưu hóa và phân rã không gian phạm vi, thay vì xây dựng một mạng cuộn lặp cụ thể cho thuật toán, chúng tôi cung cấp hai triển khai: PRL-PGD và PRL-RND. Các thí nghiệm cho thấy hiệu suất và hiệu quả vượt trội của mạng PRL so với các phương pháp hiện đại khác, với tiềm năng lớn cho việc cải thiện thêm và ứng dụng thực tế vào các vấn đề hình ảnh ngược khác hoặc các mô hình tối ưu hóa.

Từ khóa

#Cảm biến nén #học sâu #vật lý #hồi phục hình ảnh #mạng lưới cuộn lại.

Tài liệu tham khảo

Adler, J., & Öktem, O. (2018). Learned primal-dual reconstruction. IEEE Transactions on Medical Imaging, 37(6), 1322–1332. Agustsson, E., & Timofte, R. (2017). NTIRE 2017 challenge on single image super-resolution: dataset and study. In Proceedings of IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp. 126–135. Björck, Å., Elfving, T., & Strakos, Z. (1998). Stability of Conjugate Gradient and lanczos methods for linear least squares problems. SIAM Journal on Matrix Analysis and Applications, 19(3), 720–736. Blumensath, T., & Davies, M. E. (2009). Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3), 265–274. Boufounos, Petros. T., & Baraniuk, Richard. G. (2008). 1-bit Compressive sensing. In Proceedings of IEEE conference on information sciences and systems (CISS), pp. 16–21. Cai, J.-F., Candès, E. J., & Shen, Z. (2010). A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4), 1956–1982. Candès, E. J., & Wakin, M. B. (2008). An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2), 21–30. Chen, B., & Zhang, J. (2022). Content-aware scalable deep compressed sensing. IEEE Transactions on Image Processing, 31, 5412–5426. Chen, D., & Davies, Mike. E. (2020). Deep decomposition learning for inverse imaging problems. In Proceedings of European conference on computer vision (ECCV), pp. 510–526. Chen, D., Tachella, J., & Davies, M. E. (2021a). Equivariant imaging: learning beyond the range space. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4379–4388. Chen, H., Zhang, Y., Zhang, W., Liao, P., Li, K., Zhou, J., & Wang, G. (2017). Low-dose CT via convolutional neural network. Biomedical Optics Express, 8(2), 679–694. Chen, J., Sun, Y., Liu, Q., & Huang, R. (2020). Learning memory augmented cascading network for compressed sensing of images. In Proceedings of European conference on computer vision (ECCV), pp. 513–529. Chen, Y., & Pock, T. (2016). Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1256–1272. Chen, Z., Guo, W., Feng, Y., Li, Y., Zhao, C., Ren, Y., & Shao, L. (2021). Deep-learned regularization and proximal operator for image compressive sensing. IEEE Transactions on Image Processing, 30, 7112–7126. Coban, S., Andriiashen, V., & Ganguly, P. (2020). Apple CT Data: simulated parallel-beam tomographic datasets. Zenodo. Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007). Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Transactions on Image Processing, 16(8), 2080–2095. Denker, A., Schmidt, M., Leuschner, J., Maass, P., & Behrmann, J. (2020). Conditional normalizing flows for low-dose computed tomography image reconstruction. arXiv preprint arXiv:2006.06270. Dong, C., Loy, C. C., He, K., & Tang, X. (2014a). Learning a deep convolutional network for image super-resolution. In Proceedings of European conference on computer vision (ECCV), pp. 184–199. Dong, W., Shi, G., Li, X., Ma, Y., & Huang, F. (2014b). Compressive sensing via nonlocal low-rank regularization. IEEE Transactions on Image Processing, 23(8), 3618–3632. Donoho, D. L. (2006). Compressed sensing. IEEE Transactions on Information Theory, 52(4), 1289–1306. Elad, M. (2010). Sparse and redundant representations: from theory to applications in signal and image processing (Vol. 2). Springer. Fan, Z.-E., Lian, F., & Quan, J.-N. (2022). Global sensing and measurements reuse for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8954–8963. Fowler, J. E., Mun, S., Tramel, E. W., et al. (2012). Block-based compressed sensing of images and video. Foundations and Trends in Signal Processing, 4(4), 297–416. Gan, L. (2007). Block compressed sensing of natural images. In Proceedings of IEEE international conference on digital signal processing (ICDSP), pp. 403–406. IEEE. Gilton, D., Ongie, G., & Willett, R. (2019). Neumann networks for linear inverse problems in imaging. IEEE Transactions on Computational Imaging, 6, 328–343. Gu, J., & Dong, C. (2021). Interpreting super-resolution networks with local attribution maps. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 9199–9208. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 770–778. Huang, J. -B., Singh, A., & Ahuja, N. (2015). Single image super-resolution from transformed self-exemplars. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 5197–5206. Huang, Y., Würfl, T., Breininger, K., Liu, L., Lauritsch, G., & Maier, A. (2018). Some investigations on robustness of deep learning in limited angle tomography. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 145–153. Jacques, L., Laska, J. N., Boufounos, P. T., & Baraniuk, R. G. (2013). Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transactions on Information Theory, 59(4), 2082–2102. Kafle, S., Joseph, G., & Varshney, P. K. (2021). One-bit compressed sensing using untrained network prior. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2875–2879. Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. In Proceedings of international conference on learning representations (ICLR), pp. 1–15. Kokkinos, F., & Lefkimmiatis, S. (2018). Deep image demosaicking using a cascade of convolutional residual denoising networks. In Proceedings of European conference on computer vision (ECCV), pp. 303–319. Kruse, J., Rother, C., & Schmidt, U. (2017). Learning to push the limits of efficient FFT-based image deconvolution. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4586–4594. Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., & Ashok, A. (2016). ReconNet: Non-iterative reconstruction of images from compressively sensed measurements. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 449–458. Lefkimmiatis, S. (2017). Non-local color image denoising with convolutional neural networks. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3587–3596. Lefkimmiatis, S. (2018). Universal denoising networks: A novel CNN architecture for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3204–3213. Leuschner, J., Schmidt, M., Ganguly, P. S., Andriiashen, V., Coban, S. B., Denker, A., Bauer, D., Hadjifaradji, A., Batenburg, K. J., Maass, P., et al. (2021). Quantitative comparison of deep learning-based image reconstruction methods for low-dose and sparse-angle CT applications. Journal of Imaging, 7(3), 44. Li, Y., Li, K., Zhang, C., Montoya, J., & Chen, G.-H. (2019). Learning to reconstruct computed tomography images directly from sinogram data under a variety of data acquisition conditions. IEEE Transactions on Medical Imaging, 38(10), 2469–2481. Liu, T., Chaman, A., Belius, D., & Dokmanic, I. (2020). Interpreting U-nets via task-driven multiscale dictionary learning. arXiv preprint arXiv:2011.12815. Liu, Y., Long, Z., & Zhu, C. (2018). Image completion using low tensor tree rank and total variation minimization. IEEE Transactions on Multimedia, 21(2), 338–350. Liu, Y., Long, Z., Huang, H., & Zhu, C. (2019). Low CP rank and tucker rank tensor completion for estimating missing components in image data. IEEE Transactions on Circuits and Systems for Video Technology, 30(4), 944–954. Long, Z., Liu, Y., Chen, L., & Zhu, C. (2019). Low rank tensor completion for multiway visual data. Signal Processing, 155, 301–316. Long, Z., Zhu, C., Liu, J., & Liu, Y. (2021). Bayesian low rank tensor ring for image recovery. IEEE Transactions on Image Processing, 30, 3568–3580. Long, Z., Zhu, C., Liu, J., Comon, P., & Liu, Y. (2022). Trainable subspaces for low rank tensor completion: Model and analysis. IEEE Transactions on Signal Processing, 70, 2502–2517. Lustig, M., Donoho, D., & Pauly, J. M. (2007). Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 58(6), 1182–1195. Lustig, M., Donoho, D. L., Santos, J. M., & Pauly, J. M. (2008). Compressed sensing MRI. IEEE Signal Processing Magazine, 25(2), 72–82. Ma, K., Duanmu, Z., Qingbo, W., Wang, Z., Yong, H., Li, H., & Zhang, L. (2016). Waterloo exploration database: New challenges for image quality assessment models. IEEE Transactions on Image Processing, 26(2), 1004–1016. Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of IEEE international conference on computer vision (ICCV), 2, pp. 416–423. Mousavi, A., & Baraniuk, R. G. (2017). Learning to invert: Signal recovery via deep convolutional networks. In Proceedings of IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 2272–2276. Mousavi, A., Patel, A. B., & Baraniuk, R. G. (2015). A deep learning approach to structured signal recovery. In Proceedings of IEEE allerton conference on communication, control, and computing, pp. 1336–1343. Mun, S., & Fowler, J. E. (2009). Block compressed sensing of images using directional transforms. In Proceedings of IEEE international conference on image processing (ICIP), pp. 3021–3024. Nam, S., Davies, M. E., Elad, M., & Gribonval, R. (2013). The cosparse analysis model and algorithms. Applied and Computational Harmonic Analysis, 34(1), 30–56. Niu, S., Gao, Y., Bian, Z., Huang, J., Chen, W., Gaohang, Yu., Liang, Z., & Ma, J. (2014). Sparse-View X-Ray CT reconstruction via total generalized variation regularization. Physics in Medicine & Biology, 59(12), 2997. Parikh, N., Boyd, S., et al. (2014). Proximal algorithms. Foundations and in Optimization, 1(3), 127–239. Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., & Antiga, L. et al. (2019). PyTorch: An imperative style, high-performance deep learning library. Proceedings of Neural Information Processing Systems (NeurIPS), 32. Pelt, D. M., Batenburg, K. J., & Sethian, J. A. (2018). Improving tomographic reconstruction from limited data using mixed-scale dense convolutional neural networks. Journal of Imaging, 4(11), 128. Radon, J. (1986). On the determination of functions from their integral values along certain manifolds. IEEE Transactions on Medical Imaging, 5(4), 170–176. Ravishankar, S., Ye, J. C., & Fessler, J. A. (2019). Image reconstruction: From sparsity to data-adaptive methods and machine learning. Proceedings of the IEEE, 108(1), 86–109. Ren, C., He, X., Wang, C., & Zhao, Zhibo. (2021). Adaptive consistency prior based deep network for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 8596–8606. Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of international conference on medical image computing and computer-assisted intervention (MICCAI), pp. 234–241. Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1874–1883. Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019a). Image compressed sensing using convolutional neural network. IEEE Transactions on Image Processing, 29, 375–388. Shi, W., Jiang, F., Liu, S., & Zhao, D. (2019b). Scalable convolutional neural network for image compressed sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 12290–12299. Song, J., Chen, B., & Zhang, J. (2021). Memory-augmented deep unfolding network for compressive sensing. In Proceedings of ACM international conference on multimedia (ACM MM), pp. 4249–4258. Song, J., Chen, B., & Zhang, J. (2023a). Deep memory-augmented proximal unrolling network for compressive sensing. International Journal of Computer Vision, 1–20. Song, J., Chen, B., & Zhang, J. (2023b). Dynamic path-controllable deep unfolding network for compressive sensing. IEEE Transactions on Image Processing, 32, 2202–2214. Sun, J., Li, H., Zongben, X., et al. (2016). Deep ADMM-Net for compressive sensing MRI. Proceedings of Neural Information Processing Systems (NeurIPS), 29, 10–18. Sun, Y., Chen, J., Liu, Q., Liu, B., & Guo, G. (2020). Dual-path attention network for compressed sensing image reconstruction. IEEE Transactions on Image Processing, 29, 9482–9495. Szczykutowicz, T. P., & Chen, G.-H. (2010). Dual energy CT using slow kVp switching acquisition and prior image constrained compressed sensing. Physics in Medicine & Biology, 55(21), 6411. Tian, C., Yong, X., Li, Z., Zuo, W., Fei, L., & Liu, H. (2020). Attention-guided CNN for image denoising. Neural Networks, 124, 117–129. Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research,9(11). Wang, H., Zhang, T., Yu, M., Sun, J., Ye, W., Wang, C., & Zhang, S. (2020). Stacking networks dynamically for image restoration based on the plug-and-play framework. In Proceedings of European conference on computer vision (ECCV), pp. 446–462. Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. Wu, Z., Zhang, J., & Mou, C. (2021). Dense deep unfolding network with 3D-CNN prior for snapshot compressive imaging. In Proceedings of IEEE international conference on computer vision (ICCV), pp. 4892–4901. Xiang, J., Dong, Y., & Yang, Y. (2021). FISTA-Net: Learning a fast iterative shrinkage thresholding network for inverse problems in imaging. IEEE Transactions on Medical Imaging, 40(5), 1329–1339. Yang, J., Wright, J., Huang, T., & Ma, Y. (2008). Image super-resolution as sparse representation of raw image patches. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. You, D., Xie, J., & Zhang, J. (2021a). ISTA-Net\(^{++}\): Flexible deep unfolding network for compressive sensing. In Proceedings of IEEE international conference on multimedia and expo (ICME), pp. 1–6. You, D., Zhang, J., Xie, J., Chen, B., & Ma, S. (2021b). COAST: Controllable arbitrary-sampling network for compressive sensing. IEEE Transactions on Image Processing, 30, 6066–6080. Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021). Multi-stage progressive image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 14821–14831. Zhang, J., & Ghanem, B. (2018). ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 1828–1837. Zhang, J., Zhao, C., Zhao, D., & Gao, W. (2014a). Image compressive sensing recovery using adaptively learned sparsifying basis via L0 minimization. Signal Processing, 103, 114-126. Zhang, J., Zhao, D., & Gao, W. (2014b). Group-based sparse representation for image restoration. IEEE Transactions on Image Processing, 23(8), 3336–3351. Zhang, J., Zhao, C., & Gao, W. (2020a). Optimization-inspired compact deep compressive sensing. IEEE Journal of Selected Topics in Signal Processing, 14(4), 765–774. Zhang, J., Chen, B., Xiong, R., & Zhang, Y. (2023). Physics-inspired compressive sensing: Beyond deep unrolling. IEEE Signal Processing Magazine, 40(1), 58–72. Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017a). Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 26(7), 3142–3155. Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017b). Learning deep CNN denoiser prior for image restoration. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3929–3938. Zhang, K., Van Gool, L., & Timofte, R.(2020b). Deep unfolding network for image super-resolution. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 3217–3226. Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2021). Plug-and-play image restoration with deep denoiser prior. IEEE Transactions on Pattern Analysis and Machine Intelligence. Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018). Image super-resolution using very deep residual channel attention networks. In Proceedings of European conference on computer vision (ECCV), pp. 286–301. Zhang, Z., Liu, Y., Liu, J., Wen, F., & Zhu, C. (2020c). AMP-Net: Denoising-based deep unfolding for compressive image sensing. IEEE Transactions on Image Processing, 30, 1487–1500. Zhao, C., Ma, S., Zhang, J., Xiong, R., & Gao, W. (2016). Video compressive sensing reconstruction via reweighted residual sparsity. IEEE Transactions on Circuits and Systems for Video Technology, 27(6), 1182–1195. Zhao, C., Zhang, J., Ma, S., Fan, X., Zhang, Y., & Gao, W. (2016). Reducing image compression artifacts by structural sparse representation and quantization constraint prior. IEEE Transactions on Circuits and Systems for Video Technology, 27(10), 2057–2071. Zheng, H., Yong, H., & Zhang, L. (2021). Deep convolutional dictionary learning for image denoising. In Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp. 630–641.