Saliency-based image correction for colorblind patients

Springer Science and Business Media LLC - 2020
Jinjiang Li1,2, Xiaomei Feng2,1, Hui Fan2,1
1Co-innovation Center of Shandong Colleges and Universities, Future Intelligent Computing, Yantai, China
2School of Computer Science and Technology, Shandong Technology and Business University, Yantai, China

Tóm tắt

AbstractImproper functioning, or lack, of human cone cells leads to vision defects, making it impossible for affected persons to distinguish certain colors. Colorblind persons have color perception, but their ability to capture color information differs from that of normal people: colorblind and normal people perceive the same image differently. It is necessary to devise solutions to help persons with color blindness understand images and distinguish different colors. Most research on this subject is aimed at adjusting insensitive colors, enabling colorblind persons to better capture color information, but ignores the attention paid by colorblind persons to the salient areas of images. The areas of the image seen as salient by normal people generally differ from those seen by the colorblind. To provide the same saliency for colorblind persons and normal people, we propose a saliency-based image correction algorithm for color blindness. Adjusted colors in the adjusted image are harmonious and realistic, and the method is practical. Our experimental results show that this method effectively improves images, enabling the colorblind to see the same salient areas as normal people.

Từ khóa


Tài liệu tham khảo

Ohata, F.; Kudo, H.; Matsumoto, T.; Takeuchi, Y.; Ohnishi, N. Image transform based on the distribution of representative colors for color deficient. IEEJ Transactions on Electronics, Information and Systems Vol. 130, No. 12, 2176–2177, 2010.

Meguro, M.; Taguchi, A. A color conversion method for realizing barrier free of color defective vision. IEEJ Transactions on Electronics, Information and Systems Vol. 131, No. 2, 482–483, 2011.

Yanagida, T.; Okajima, K.; Mimura, H. Color scheme adjustment by fuzzy constraint satisfaction for color vision deficiencies. Color Research & Application Vol. 40, No. 5, 446–464, 2015.

Shen, W. Y.; Mao, X. Y.; Hu, X. H.; Wong, T. T. Seamless visual sharing with color vision deficiencies. ACM Transactions on Graphics Vol. 35, No. 4, Article No. 70, 2016.

Tanuwidjaja, E.; Huynh, D.; Koa, K.; Nguyen, C.; Shao, C.; Torbett, P.; Emmenegger, C.; Weibel, N. Chroma: A wearable augmented-reality solution for color blindness. In: Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing, 799–810, 2014.

Melillo, P.; Riccio, D.; di Perna, L.; Sanniti di Baja, G.; de Nino, M.; Rossi, S.; Testa, F.; Simonelli, F.; Frucci, M. Wearable improved vision system for color vision deficiency correction. IEEE Journal of Translational Engineering in Health and Medicine Vol. 5, 1–7, 2017.

Weale, R. Defective colour vision: Fundamentals, diagnosis and management. British Journal of Ophthalmology Vol. 70, No. 2, 159, 1986.

Rosenstock, H. B.; Swick, D. A. Color discrimination for the color blind. Aerospace Medicine Vol. 45, No. 10, 1194, 1974.

Kessler, J. What can be done for the color blind? Annals of Ophthalmology Vol. 9, No. 4, 431–433, 1977.

Subbian, V.; Ratcliff, J.; Meunier, J.; Korfhagen, J.; Beyette, F.; Shaw, G. Integration of new technology for research in the emergency department: Feasibility of deploying a robotic assessment tool for mild traumatic brain injury evaluation. IEEE Journal of Translational Engineering in Health and Medicine Vol. 3, Article No. 3200109, 2015.

Nakayama, K. Assist device in color discrimination using Heilmeier type guest-host liquid crystal for red-green color vision defect. Electronics and Communications in Japan Vol. 102, No. 8, 17–24, 2019.

Hunt, R. W. G. Colour standards and calculations. In: The Reproduction of Colour. Kriss, M. A.; Hunt, R. John Wiley & Sons, Ltd, 92–125, 2005.

Nathans, J.; Thomas, D.; Hogness, D. Molecular genetics of human color vision: The genes encoding blue, green, and red pigments. Science Vol. 232, No. 4747, 193–202, 1986.

Wong, B. Points of view: Color blindness. Nature Methods Vol. 8, No. 6, 441, 2011.

Scoles, D.; Sulai, Y. N.; Dubra, A. In vivo dark-field imaging of the retinal pigment epithelium cell mosaic. Biomedical Optics Express Vol. 4, No. 9, 1710, 2013.

Brettel, H.; Vi´enot, F.; Mollon, J. D. Computerized simulation of color appearance for dichromats. Journal of the Optical Society of America A Vol. 14, No. 10, 2647, 1997.

Meyer, G. W.; Greenberg, D. P. Color-defective vision and computer graphics displays. IEEE Computer Graphics and Applications Vol. 8, No. 5, 28–40, 1988.

MacHado, G. M.; Oliveira, M. M.; Fernandes, L. A physiologically-based model for simulation of color vision deficiency. IEEE Transactions on Visualization and Computer Graphics Vol. 15, No. 6, 1291–1298, 2009.

Chen, C. S.; Wu, S. Y.; Huang, J. B. Enhancing color representation for the color vision impaired. In: Proceedings of ECCV Workshop on Computer Vision Applications for the Visually Impaired, 2008.

Okajima, K.; Kanbe, S. A real-time color simulation of dichromats. Technical Report of the IEICE, 107: 107110, 2007.

Flatla, D. R.; Gutwin, C. Individual models of color differentiation to improve interpretability of information visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2563–2572, 2010.

Kuhn, G. R.; Oliveira, M. M.; Fernandes, L. An efficient naturalness-preserving image-recoloring method for dichromats. IEEE Transactions on Visualization and Computer Graphics Vol. 14, No. 6, 1747–1754, 2008.

Jiang, H. Z.; Wang, J. D.; Yuan, Z. J.; Wu, Y.; Zheng, N. N.; Li, S. P. Salient object detection: A discriminative regional feature integration approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2083–2090, 2013.

Peng, H. W.; Li, B.; Ling, H. B.; Hu, W. M.; Xiong, W. H.; Maybank, S. J. Salient object detection via structured matrix decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 4, 818–832, 2017.

Cong, R. M.; Lei, J. J.; Fu, H. Z.; Huang, Q. M.; Cao, X. C.; Hou, C. P. Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation. IEEE Transactions on Image Processing Vol. 27, No. 2, 568–579, 2018.

Cheng, M. M.; Mitra, N. J.; Huang, X. L.; Torr, P. H. S.; Hu, S. M. Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 3, 569–582, 2015.

Wang, W. G.; Shen, J. B.; Shao, L.; Porikli, F. Correspondence driven saliency transfer. IEEE Transactions on Image Processing Vol. 25, No. 11, 5025–5034, 2016.

Borji, A.; Cheng, M. M.; Jiang, H. Z.; Li, J. Salient object detection: A benchmark. IEEE Transactions on Image Processing Vol. 24, No. 12, 5706–5722, 2015.

Wang, L. J.; Lu, H. C.; Ruan, X.; Yang, M. H. Deep networks for saliency detection via local estimation and global search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3183–3192, 2015.

Zhao, R.; Ouyang, W. L.; Li, H. S.; Wang, X. G. Saliency detection by multi-context deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1265–1274, 2015.

Han, J. W.; Zhang, D. W.; Hu, X. T.; Guo, L.; Ren, J. C.; Wu, F. Background prior-based salient object detection via deep reconstruction residual. IEEE Transactions on Circuits and Systems for Video Technology Vol. 25, No. 8, 1309–1321, 2015.

Huang, X.; Shen, C. Y.; Boix, X.; Zhao, Q. SALICON: Reducing the semantic gap in saliency prediction by adapting deep neural networks. In: Proceedings of the IEEE International Conference on Computer Vision, 262–270, 2015.

Chen, T. S.; Lin, L.; Liu, L. B.; Luo, X. N.; Li, X. L. DISC: Deep image saliency computing via progressive representation learning. IEEE Transactions on Neural Networks and Learning Systems Vol. 27, No. 6, 1135–1149, 2016.

Zhang, J. M.; Sclaroff, S.; Lin, Z.; Shen, X. H.; Price, B.; Mech, R. Unconstrained salient object detection via proposal subset optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5733–5742, 2016.

Li, G. B.; Xie, Y.; Lin, L.; Yu, Y. Z. Instance-level salient object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2386–2395, 2017.

Wang, W. G.; Shen, J. B.; Xie, J. W.; Cheng, M. M.; Ling, H. B.; Borji, A. Revisiting video saliency prediction in the deep learning era. IEEE Transactions on Pattern Analysis and Machine Intelligence DOI: 10.1109/TPAMI.2019.2924417, 2019.

Wang, W. G.; Shen, J. B.; Yang, R. G.; Porikli, F. Saliency-aware video object segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 40, No. 1, 20–33, 2018.

Wei, Y. C.; Feng, J. S.; Liang, X. D.; Cheng, M. M.; Zhao, Y.; Yan, S. C. Object region mining with adversarial erasing: A simple classification to semantic segmentation approach. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1568–1576, 2017.

Wei, Y. C.; Liang, X. D.; Chen, Y. P.; Shen, X. H.; Cheng, M. M.; Feng, J. S.; Zhao, Y.; Yan, S. STC: A simple to complex framework for weakly-supervised semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 39, No. 11, 2314–2320, 2017.

Wang, W. G.; Shen, J. B.; Ling, H. B. A deep network solution for attention and aesthetics aware photo cropping. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 41, No. 7, 1531–1544, 2019.

Sun, J.; Ling, H. B. Scale and object aware image retargeting for thumbnail browsing. In: Proceedings of the International Conference on Computer Vision, 1511–1518, 2011.

Zhou, L.; Yang, Z. H.; Zhou, Z. T.; Hu, D. W. Salient region detection using diffusion process on a two-layer sparse graph. IEEE Transactions on Image Processing Vol. 26, No. 12, 5882–5894, 2017.

Mateescu, V. A.; Bajic, I. V. Visual attention retargeting. IEEE MultiMedia Vol. 23, No. 1, 82–91, 2016.

Nguyen, T. V.; Ni, B.; Liu, H.; Xia, W.; Luo, J.; Kankanhalli, M.; Yan, S. Image re-attentionizing. IEEE Transactions on Multimedia Vol. 15, No. 8, 1910–1919, 2013.

Mateescu, V. A.; Baji´c, I. V. Attention retargeting by color manipulation in images. In: Proceedings of the 1st International Workshop on Perception Inspired Video Processing, 15–20, 2014.

Mendez, E.; Feiner, S.; Schmalstieg, D. Focus and context in mixed reality by modulating first order salient features. In: Smart Graphics. Lecture Notes in Computer Science, Vol. 6133. Taylor, R.; Boulanger, P.; Kru¨ger, A.; Olivier, P. Eds. Springer Berlin Heidelberg, 232–243, 2010.

Lu, S. P.; Dauphin, G.; Lafruit, G.; Munteanu, A. Color retargeting: Interactive time-varying color image composition from time-lapse sequences. Computational Visual Media Vol. 1, No. 4, 321–330, 2015.

Fried, O.; Shechtman, E.; Goldman, D. B.; Finkelstein, A. Finding distractors in images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1703–1712, 2015.

Su, S. L.; Durand, F.; Agrawala, M. De-emphasis of distracting image regions using texture power maps. In: Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, 164, 2005.

Li, J. J.; Li, G. H.; Fan, H. Image dehazing using residual-based deep CNN. IEEE Access Vol. 6, 26831–26842, 2018.

Mechrez, R.; Shechtman, E.; Zelnik-Manor, L. Saliency driven image manipulation. Machine Vision and Applications Vol. 30, No. 2, 189–202, 2019.

Huang, J. B.; Chen, C. S.; Jen, T. C.; Wang, S. J. Image recolorization for the colorblind. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, 1161–1164, 2009.

Lin, H. Y.; Chen, L. Q.; Wang, M. L. Improving discrimination in color vision deficiency by image re-coloring. Sensors Vol. 19, No. 10, 2250, 2019.

Sundaram, N.; Brox, T.; Keutzer, K. Dense point trajectories by GPU-accelerated large displacement optical flow. In: Computer Vision–ECCV 2010. Lecture Notes in Computer Science, Vol. 6311. Daniilidis, K.; Maragos, P.; Paragios, N. Eds. Springer Berlin Heidelberg, 438–451, 2010.

Yang, C.; Zhang, L. H.; Lu, H. C.; Ruan, X.; Yang, M. H. Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3166–3173, 2013.

Zhang, S. Y.; Liang, R. Z.; Wang, M. ShadowGAN: Shadow synthesis for virtual objects with conditional adversarial networks. Computational Visual Media Vol. 5, No. 1, 105–115, 2019.

Thông tin
Thông tin xuất bản

Springer Science and Business Media LLC

Thông tin tác giả