Hiển thị thông tin dựa trên điều chế tâm lý-đồ họa: Giới thiệu và tổng quan

Springer Science and Business Media LLC - Tập 15 - Trang 1-18 - 2021
Ning Liu1, Zhongpai Gao2, Jia Wang3, Guangtao Zhai3
1Cooperative Medianet Innovation Center, Shanghai Jiao Tong University, Shanghai, China
2MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai, China
3Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China

Tóm tắt

Ngành công nghiệp và các cơ sở nghiên cứu học thuật đã nỗ lực lớn trong việc cải thiện tốc độ làm tươi và độ phân giải của các thiết bị hiển thị để đáp ứng nhu cầu ngày càng tăng của người tiêu dùng về chất lượng hình ảnh tốt hơn. Kết quả là, nhiều màn hình hiện đại có độ phân giải không gian và thời gian vượt xa khả năng nhận discern của hệ thống thị giác con người. Điều này dẫn đến khả năng sử dụng những dư thừa mật độ hiển thị-mắt này cho các ứng dụng sáng tạo. Điều chế tâm lý-thời gian/không gian (TPVM/SPVM) đã được đề xuất nhằm khai thác những dư thừa này để tạo ra nhiều cảm nhận thị giác cho các người xem khác nhau hoặc để truyền tải dữ liệu không phải hình ảnh đến các thiết bị tính toán mà không ảnh hưởng đến việc xem bình thường. Bài báo này xem xét công nghệ STPVM từ cả góc độ khái niệm và thuật toán, với các ứng dụng tiêu biểu trong hiển thị đa chiều, hiển thị với giao tiếp ánh sáng khả kiến, v.v. Một số hướng nghiên cứu tương lai khả thi cũng được xác định.

Từ khóa

#Điều chế tâm lý-đồ họa #công nghệ hiển thị #thông tin không gian-thời gian #giao tiếp ánh sáng khả kiến #hiển thị đa góc nhìn

Tài liệu tham khảo

Wu X, Zhai G. Temporal psychovisual modulation: a new paradigm of information display [exploratory DSP]. IEEE Signal Processing Magazine, 2013, 30(1): 136–141 Kelly D H. Spatio-temporal frequency characteristics of color-vision mechanisms. Journal of the Optical Society America, 1974, 64(7): 983–990 Varner D, Jameson D, Hurvich L M. Temporal sensitivities related to color theory. Journal of the Optical Society of America A, 1984, 1(5): 474–481 Instruments T. DLP discovery 4100 development kit. See Ti Website, 2015 Corporation N. NVIDIA 3D Vision. See Nvidia Website, 2014 Ko H, Paik J, Zalewski G. Stereoscopic screen sharing method and apparatus. 2010. US Patent App. 12/503,029 Corporation N. Pixel density display listing. See Pixensity Website, 2018 Karnik A, Martinez Plasencia D, Mayol-Cuevas W, Subramanian S. PiVOT: personalized view-overlays for tabletops. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology. 2012, 271–280 Wetzstein G, Lanman D, Hirsch M, Raskar R. Tensor displays: compressive light field synthesis using multilayer displays with directional backlighting. ACM Transactions on Graphics, 2012, 31(4): 80 Ye G, State A, Fuchs H. A practical multi-viewer tabletop autostereoscopic display. In: Proceedings of IEEE International Symposium on Mixed and Augmented Reality. 2010, 147–156 Karnik A, Mayol-Cuevas W, Subramanian S. MUSTARD: a multi user see through AR display. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2012, 2541–2550 Nashel A, Fuchs H. Random hole display: a non-uniform barrier autostereoscopic display. In: Proceedings of 3DTV Conference: The True Vision — Capture, Transmission and Display of 3D Video. 2009, 1–4 Lanman D, Wetzstein G, Hirsch M, Heidrich W, Raskar R. Polarization fields: dynamic light field display using multi-layer LCDs. In: Proceedings of the SIGGRAPH Asia Conference. 2011, 1–10 Zhai G, Wu X. Multiuser collaborative viewport via temporal psychovisual modulation [applications corner]. IEEE Signal Processing Magazine, 2014, 31(5): 144–149 Wu X, Zhai G. Backward compatible stereoscopic displays via temporal psychovisual modulation. In: Proceedings of SIGGRAPH Asia Emerging Technologies. 2012 Jiao L, Shu X, Wu X. LED backlight adjustment for backward-compatible stereoscopic display. IEEE Signal Processing Letters, 2013, 20(12): 1203–1206 Ma R, Au O C, Wan P, Xu L, Sun W, Hu W. Improved temporal psychovisual modulation for backward-compatible stereoscopic display. In: Proceedings of IEEE Global Conference on Signal and Information Processing. 2014, 1034–1038 Chen Y, Zhai G, Zhou J, Wan Z, Tang L. Global quality of assessment and optimization for the backward-compatible stereoscopic display system. In: Proceedings of IEEE International Conference on Image Processing. 2017, 191–195 Fujimura W, Koide Y, Songer R, Hayakawa T, Shirai A, Yanaka K. 2x3D: real time shader for simultaneous 2D/3D hybrid theater. In: Proceedings of SIGGRAPH Asia Emerging Technologies. 2012, 1–2 Scher S, Liu J, Vaish R, Gunawardane P, Davis J. 3D+2DTV: 3D displays with no ghosting for viewers without glasses. ACM Transactions on Graphics, 2013, 32(3): 21 Gao Z, Zhai G, Min X. Information security display system based on temporal psychovisual modulation. In: Proceedings of IEEE International Symposium on Circuits and Systems. 2014, 449–452 Hu C, Zhai G, Gao Z, Min X. Information security display system based on spatial psychovisual modulation. In: Proceedings of IEEE International Conference on Multimedia and Expo. 2014, 1–4 Chen Y, Liu N, Zhai G, Gao Z, Gu K. Information security display system on android device. In: Proceedings of IEEE Region 10 Conference. 2016, 1634–1637 Li X, Zhai G, Wang J, Gu K. Portable information security display system via spatial psychovisual modulation. In: Proceedings of IEEE Visual Communications and Image Processing. 2017, 1–4 Hu C, Zhai G, Gao Z, Min X. Simultaneous dual-subtitles exhibition via spatial psychovisual modulation. In: Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. 2014, 1–4 Hu C, Zhai G, Gao Z, Min X. Simultaneous triple subtitles exhibition via temporal psychovisual modulation. In: Proceedings of the 9th IEEE Conference on Industrial Electronics and Applications. 2014, 944–947 Sun W, Zhai G, Gao Z, Chen T, Zhu Y, Wang Z. Dual-view oracle bone script recognition system via temporal-spatial psychovisual modulation. In: Proceedings of IEEE Conference on Multimedia Information Processing and Retrieval. 2020, 193–198 Gao Z, Zhai G, Hu C, Min X. Dual-view medical image visualization based on spatial-temporal psychovisual modulation. In: Proceedings of IEEE International Conference on Image Processing. 2014 Fang W, Zhai G, Yang X, Liu J, Chen Y. An eye-friendly dual-view projection system using temporal psychovisual modulation. In: Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. 2017, 1–5 Zhai G, Wu X. Defeating camcorder piracy by temporal psychovisual modulation. Journal of Display Technology, 2014, 10(9): 754–757 Gao Z, Zhai G, Wu X, Min X, Zhi C. DLP based anti-piracy display system. In: Proceedings of IEEE Conference on Visual Communications and Image Processing. 2014, 145–148 Chen Y, Zhai G, Gao Z, Gu K, Zhang W, Hu M, Liu J. Movie piracy tracking using temporal psychovisual modulation. In: Proceedings of IEEE International Symposium on Broadband Multimedia Systems and Broadcasting. 2017, 1–4 Gao Z, Zhai G, Hu C. The invisible QR code. In: Proceedings of the ACM International Conference on Multimedia. 2015, 675–678 Lu X, You B, Lin P Y. Augmented reality via temporal psycho-visual modulation. In: Proceedings of IEEE International Conference on Multimedia Expo Workshops. 2016, 1–4 Chen Q, Chen Y. Polarization based invisible barcode display. In: Proceedings of International Forum on Digital TV and Wireless Multimedia Communication. 2018, 67–77 Shi S, Chen L, Hu W, Gruteser M. Reading between lines: high-rate, non-intrusive visual codes within regular videos via implicitcode. In: Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 2015, 157–168 Wu X, Shu X. Combining information display and visible light wireless communication. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. 2015, 1657–1661 Shu X, Wu X. Frame untangling for unobtrusive display-camera visible light communication. In: Proceedings of the 24th ACM International Conference on Multimedia. 2016, 650–654 Liu K, Wu X, Shu X. On display-camera synchronization for visible light communication. In: Proceedings of Visual Communications and Image Processing. 2015, 1–4 Hu C, Zhai G, Gao Z. Visible light communication via temporal psychovisual modulation. In: Proceedings of the 23rd ACM International Conference on Multimedia. 2015, 785–788 Fang W, Zhai G, Yang X. A flash light system for individuals with visual impairment based on TPVM. In: Proceedings of International Conference on Cloud Computing and Big Data. 2016, 362–366 Zhang Y, Zhai G, Liu J, Weng X, Chen Y. ‘window of visibility’ inspired security lighting system. In: Proceedings of International Conference on Systems, Signals and Image Processing. 2017, 1–5 Gao Z, Zhai G, Zhou J. Factorization algorithms for temporal psychovisual modulation display. IEEE Transactions on Multimedia, 2016, 18(4): 614–626 Feng J, Huo X, Song L, Yang X, Zhang W. Evaluation of different algorithms of nonnegative matrix factorization in temporal psychovisual modulation. IEEE Transactions on Circuits and Systems for Video Technology, 2014, 24(4): 553–565 Wang L, Zhai G. Constrained nmf for multiple exhibition on a single display. In: Proceedings of Picture Coding Symposium. 2015, 292–296 Gao Z, Zhai G, Gu X, Zhou J. Adapting hierarchical ALS algorithms for temporal psychovisual modulation. In: Proceedings of IEEE International Symposium on Circuits and Systems. 2015, 2756–2759 Gao Z, Zhai G, Wang J. Spatially-weighted nonnegative matrix factorization with application to temporal psychovisual modulation. Digital Signal Processing, 2017, 67: 123–130 Kim J, Park H. Fast nonnegative matrix factorization: an active-set-like method and comparisons. SIAM Journal on Scientific Computing, 2011, 33(6): 3261–3281 Lee D D, Seung H S. Learning the parts of objects by non-negative matrix factorization. Nature, 1999, 401(6755): 788–791 Gonzalez E F, Zhang Y. Accelerating the Lee-Seung algorithm for nonnegative matrix factorization. Department of Computational and Applied Mathematics, Rice University, Houston, TX, Technical Report, TR-05-02, 2005 Berry M W, Browne M, Langville A N, Pauca V P, Plemmons R J. Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analysis, 2007, 52(1): 155–173 Kim J, Park H. Toward faster nonnegative matrix factorization: a new algorithm and comparisons. In: Proceedings of IEEE International Conference on Data Mining. 2008, 353–362 Kim H, Park H. Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM Journal on Matrix Analysis and Applications, 2008, 30(2): 713–730 Cichocki A, Anh-Huy P. Fast local algorithms for large scale nonnegative matrix and tensor factorizations. IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, 2009, 92(3): 708–721 Gillis N, Glineur F. Accelerated multiplicative updates and hierarchical als algorithms for nonnegative matrix factorization. Neural Computation, 2012, 24(4): 1085–1105 Itti L, Koch C, Niebur E. A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1998, 20(11): 1254–1259 Reinhard E, Ward G, Pattanaik S, Debevec P. High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2005 Li D, Gao Z, Zhang X P, Zhai G, Yang X. Generative adversarial networks for non-negative matrix factorization in temporal psycho-visual modulation. Digital Signal Processing, 2020, 100: 102681 Gao Z, Zhai G. Dual-view display based on spatial psychovisual modulation. IEEE Access, 2018, 6: 41356–41366 NÑsÑnen R E, Kukkonen H T, Rovamo J M. A window model for spatial integration in human pattern discrimination. Investigative Ophthalmology & Visual Science, 1995, 36(9): 1855–1862 Watamaniuk S N J, Sekuler R. Temporal and spatial integration in dynamic random-dot stimuli. Vision Research, 1992, 32(12): 2341–2347 Sun W, Gao Z, Zhai G, Zhang J, Wang Z, Zhu Y. An improved algorithm for real-time dual-view display. In: Proceedings of IEEE International Symposium on Circuits and Systems. 2020, 1–5 Zhai G, Min X. Perceptual image quality assessment: a survey. Science China Information Sciences, 2020, 63(11): 211301 Chen Y, Liu N, Zhai G, Gu K, Wang J, Gao Z, Zhu Y. Quality assessment for dual-view display system. In: Proceedings of Visual Communications and Image Processing. 2016, 1–4 Chen Y, Zhai G, Gu K, Zhang X, Lin W, Zhou J. Benchmarking screen content image quality evaluation in spatial psychovisual modulation display system. In: Proceedings of Pacific Rim Conference on Multimedia. 2018, 629–640 Scherzinger A L, Hendee W R. Basic principles of magnetic resonance imaging-an update. The Western Journal of Medicine, 1985, 143(6): 782–792 Bushong S C, Clarke G. Magnetic Resonance Imaging: Physical and Biological Principles. Elsevier Health Sciences, 2014 Lambooij M, IJsselsteijn W, Heynderickx I. Visual discomfort of 3d tv: assessment methods and modeling. Displays, 2011, 32(4): 209–218 Taylor S A. CCD and CMOS imaging array technologies: technology review. Xerox Research Centre Europe,Technical Report EPC-1998-106, 1998, 1–14 ISO/IEC. Information technology-automatic identification and data capture techniques-QR Code 2005 bar code symbology specification. see Iso.org Website, 2006 Song K, Liu N, Gao Z, Zhang J, Zhai G, Zhang X P. Deep restoration of invisible QR code from TPVM display. In: Proceedings of IEEE International Conference on Multimedia & Expo Workshops. 2020, 1–6 Siwek S E. The true cost of copyright industry piracy to the US economy. IPI Center for Technology Freedom, 2007 Dorning J. Intellectual Property Theft: A Threat to U.S. Workers, Industries, and Our Economy. DPE Research Department, 2014 NEWS B. The fact and fiction of camcorder piracy. see Bbc.co.uk Website, 2015 Byers S, Cranor L F, Cronin E, Korman D, McDaniel P. An analysis of security vulnerabilities in the movie production and distribution process. Telecommunications Policy, 2004, 28(78): 619–644 Haitsma J, Kalker T. A watermarking scheme for digital cinema. In: Proceedings of International Conference on Image Processing. 2001, 487–489 Nguyen P, Balter R, Montfort N, Baudry S. Registration methods for nonblind watermark detection in digital cinema applications. In: Proceedings of Security and Watermarking of Multimedia Contents V. 2003, 553–562 Lubin J, Bloom J A, Cheng H. Robust content-dependent high-fidelity watermark for tracking in digital cinema. In: Proceedings of Security and Watermarking of Multimedia Contents V. 2003, 536–545 Nakashima Y, Tachibana R, Babaguchi N. Watermarked movie soundtrack finds the position of the camcorder in a theater. IEEE Transactions on Multimedia, 2009, 11(3): 443–454 Davis J, Hsieh Y H, Lee H C. Humans perceive flicker artifacts at 500 hz. Scientific Reports, 2015, 5: 7861