VEViD: Vision Enhancement via Virtual diffraction and coherent Detection
Tóm tắt
The history of computing started with analog computers consisting of physical devices performing specialized functions such as predicting the position of astronomical bodies and the trajectory of cannon balls. In modern times, this idea has been extended, for example, to ultrafast nonlinear optics serving as a surrogate analog computer to probe the behavior of complex phenomena such as rogue waves. Here we discuss a new paradigm where physical phenomena coded as an algorithm perform computational imaging tasks. Specifically, diffraction followed by coherent detection becomes an image enhancement tool. Vision Enhancement via Virtual diffraction and coherent Detection (VEViD) reimagines a digital image as a spatially varying metaphoric “lightfield” and then subjects the field to the physical processes akin to diffraction and coherent detection. The term “Virtual” captures the deviation from the physical world. The light field is pixelated and the propagation imparts a phase with dependence on frequency which is different from the monotonically-increasing behavior of physical diffraction. Temporal frequencies exist in three bands corresponding to the RGB color channels of a digital image. The phase of the output, not the intensity, represents the output image. VEViD is a high-performance low-light-level and color enhancement tool that emerges from this paradigm. The algorithm is extremely fast, interpretable, and reduces to a compact and intuitively-appealing mathematical expression. We demonstrate image enhancement of 4k video at over 200 frames per second and show the utility of this physical algorithm in improving the accuracy of object detection in low-light conditions by neural networks. The application of VEViD to color enhancement is also demonstrated.
Tài liệu tham khảo
https://en.wikipedia.org/wiki/Antikythera_mechanism
Donald Routledge Hill, Mechanical Engineering in the Medieval Near East, Scientific American, 1991, pp. 64–69 (cf. Donald Routledge Hill, Mechanical Engineering)
D.R. Solli, B. Jalali, Analog optical computing. Nat. Photonics 9(11), 704–706 (2015)
D.R. Solli, C. Ropers, P. Koonath, B. Jalali, Optical rogue waves. Nature 450(7172), 1054–1057 (2007)
J.M. Dudley, G. Genty, A. Mussot, A. Chabchoub, F. Dias, Rogue waves and analogies in optics and oceanography. Nat. Rev. Phys. 1(11), 675–689 (2019)
E.H. Land, J.J. McCann, Lightness and Retinex theory. J. Opt. Soc. Amer. 61(1), 1–11 (1971)
E.H. Land, The retinex theory of color vision. Sci. Am. 237, 108–129 (1977)
M. Li et al., Structure-revealing low-light image enhancement via robust retinex model. IEEE Trans. Image Process. 27(6), 2828–2841 (2018)
X. Guo, Y. Li, H. Ling, ‘LIME: low-light image enhancement via illumination map estimation.’ IEEE Trans. Image Process. 26(2), 982–993 (2017)
L. Li, S. Sun, C. Xia, Survey of histogram equalization technology. Comput. Syst. Appl. 23(3), 1–8 (2014). (in Chinese)
G. Yadav, S. Maheshwari, A. Agarwal, Contrast limited adaptive histogram equalization based enhancement for real time video system, 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2014, pp. 2392–2397, https://doi.org/10.1109/ICACCI.2014.6968381.
K.G. Lore, A. Akintayo, S. Sarkar, LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognit 61, 650–662 (2017)
Lv F, Lu F, Wu J, Lim C. (2022). MBLLEN: Low-light Image/Video Enhancement Using CNNs.
M. Zhu, P. Pan, W. Chen, Y. Yang, EEMEFN: low-light image enhancement via edge-enhanced multi-exposure fusion network. Proc. AAAI Conf. Artif. Intell. 34(07), 13106–13113 (2020). https://doi.org/10.1609/aaai.v34i07.7013
K. Lu, L. Zhang, TBEFN: a two-branch exposure-fusion network for low-light image enhancement. IEEE Trans. Multimedia 23, 4093–4105 (2021). https://doi.org/10.1109/TMM.2020.3037526
Y. Jiang, X. Gong, D. Liu, Y. Cheng, C. Fang, X. Shen, J. Yang, P. Zhou, Z. Wang, Enlightengan: deep light enhancement without paired supervision. IEEE Trans. Image Process. 30, 2340–2349 (2021)
C. Guo, C. Li, J. Guo, C. Loy, J. Hou, S. Kwong, R. Cong (2020). Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) (pp. 1780–1789)
M. Asghari, B. Jalali, Edge detection in digital images using dispersive phase stretch transform. Int. J. Biomed. Imaging 2015, 687819 (2015)
M. Suthar, B. Jalali, "Phase-stretch adaptive gradient-field extractor (PAGE)," in Coding Theory, 2020.
C. Neil. Thompson et al. (MIT) 2020 https://arxiv.org/pdf/2007.05558.pdf
J Redmon, A. Farhadi (2018). YOLOv3: An Incremental Improvement. ArXiv, abs/1804.02767.
C. Li, C. Guo, L.H. Han, J. Jiang, M.M. Cheng, J. Gu, C.C. Loy, Low-light image and video enhancement using deep learning: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 01, 1–1 (2021)
B. Jalali, Device Physics & Modeling, In: InP HBTs: Growth, Processing, and Applications, Jalali, B., Pearton, S.J. (Eds.), (0-89006-724-4). PP. 229 - 263 (December 1994) Artech House
W. Yang, Y. Yuan, W. Ren, J. Liu, W. Scheirer, Z. Wang, T. Zhang et al., Advancing image understanding in poor visibility environments: a collective benchmark study. IEEE Trans. Image Process. 29, 5737–5752 (2020)
X. Jia, et al. LLVIP: A visible-infrared paired dataset for low-light vision. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
K. Ma et al., perceptual quality assessment for multi-exposure image fusion. IEEE Trans. Image Process. 24(11), 3345–3356 (2015)
https://www.pexels.com/video/neo-classical-building-in-city-at-night-9935075/.
https://photographylife.com/underexposure-and-overexposure-in-photography
https://sites.google.com/site/vonikakis/datasets/tm-died
https://pjreddie.com/media/files/yolov3.weights.
P.T. DeVore, B.W. Buckley, M.H. Asghari, D.R. Solli, B. Jalali, Coherent time-stretch transform for near-field spectroscopy. IEEE Photonics J. 6(2), 1–7 (2014)
D.R. Solli, S. Gupta, B. Jalali, Optical phase recovery in the dispersive Fourier transform. Appl. Phys. Lett. 95(23), 231108 (2009)
K. Goda, D.R. Solli, K.K. Tsia, B. Jalali, Theory of amplified dispersive Fourier transformation. Phys. Rev. A 80(4), 043821 (2009)