Deep residual learning for denoising Monte Carlo renderings

Jakob, W.; Marschner, S. Manifold exploration: A Markov Chain Monte Carlo technique for rendering scenes with difficult specular transport. ACM Transactions on Graphics Vol. 31, No. 4 Article No. 58, 2012.

Hachisuka, T.; Pantaleoni, J.; Jensen, H. W. A path space extension for robust light transport simulation. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 191, 2012.

Georgiev, I.; Křivánek, J.; Davidovič, T.; Slusallek, P. Light transport simulation with vertex connection and merging. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 192, 2012.

Moon, B.; Carr, N.; Yoon, S. E. Adaptive rendering based on weighted local regression. ACM Transactions on Graphics Vol. 33, No. 5, Article No. 170, 2014.

Bitterli, B.; Rousselle, F.; Moon, B.; Iglesias-Guitián, J. A.; Adler, D.; Mitchell, K.; Jarosz, W.; NováK, J. Nonlinearly weighted first-order regression for denoising Monte Carlo renderings. Computer Graphics Forum Vol. 35, No. 4, 107–117, 2016.

Rousselle, F.; Manzi, M.; Zwicker, M. Robust denoising using feature and color information. Computer Graphics Forum Vol. 32, No. 7, 121–130, 2013.

Bako, S.; Vogels, T.; McWilliams, B.; Meyer, M.; NováK, J.; Harvill, A.; Sen, P.; DeRose, T.; Rousselle, F. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Transactions on Graphics Vol. 36, No. 4, 1–14, 2017.

Kalantari, N. K.; Bako, S.; Sen, P. A machine learning approach for filtering Monte Carlo noise. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 122, 2015.

He, K. M.; Zhang, X. Y.; Ren, S. Q.; Sun, J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778, 2016.

He, K. M.; Zhang, X. Y.; Ren, S. Q.; Sun, J. Identity mappings in deep residual networks. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol 9908. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 630–645, 2016.

Bitterli, B. Rendering resources 2016. Available at https://doi.org/benedikt-bitterli.me/ resources/.

Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.

Zwicker, M.; Jarosz, W.; Lehtinen, J.; Moon, B.; Ramamoorthi, R.; Rousselle, F.; Sen, P.; Soler, C.; Yoon, S.-E. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Computer Graphics Forum Vol. 34, No. 2, 667–681, 2015.

Goodfellow, I.; Bengio, Y.; Courville, A.; Bengio, Y. Deep Learning, Vol. 1. MIT Press, 2016.

LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature Vol. 521, No. 7553, 436–444, 2015.

Schmidhuber, J. Deep learning in neural networks: An overview. Neural Networks Vol. 61, 85–117, 2015.

McCool, M. D. Anisotropic diffusion for Monte Carlo noise reduction. ACM Transactions on Graphics Vol. 18, No. 2, 171–194, 1999.

Perona, P.; Malik, J. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 12, No. 7, 629–639, 1990.

Dammertz, H.; Sewtz, D.; Hanika, J.; Lensch, H. P. A. Edge-avoiding À-Trous wavelet transform for fast global illumination filtering. In: Proceedings of the Conference on High Performance Graphics, 67–75, 2010.

Fattal, R. Edge-avoiding wavelets and their applications. ACM Transactions on Graphics Vol. 28, No. 3, Article No. 22, 2009.

Sen, P.; Darabi, S. On filtering the noise from the random parameters in Monte Carlo rendering. ACM Transactions on Graphics Vol. 31, No. 3, Article No. 18, 2012.

Aurich, V.; Weule, J. Non-linear Gaussian filters performing edge preserving diffusion. In: Mustererkennung 1995. Informatik aktuell. Sagerer, G.; Posch, S.; Kummert, F. Eds Springer Berlin Heidelberg, 538–545 1995.

Tomasi, C.; Manduchi, R. Bilateral filtering for gray and color images. In: Proceedings of the IEEE International Conference on Computer Vision, 839–846, 1998.

Eisemann, E.; Durand, F. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics Vol. 23, No. 3, 673–678, 2004.

Li, T. M.; Wu, Y. T.; Chuang, Y. Y. SURE-based optimization for adaptive sampling and reconstruction. ACM Transactions on Graphics Vol. 31, No. 6, Article No. 194, 2012.

Stein, C. M. Estimation of the mean of a multivariate normal distribution. The Annals of Statistics Vol. 9, No. 6, 1135–1151, 1981.

Van de Ville, D.; Kocher, M. SURE-based non-local means. IEEE Signal Processing Letters Vol. 16, No. 11, 973–976, 2009.

Buades, A.; Coll, B.; Morel, J. M. Nonlocal image and movie denoising. International Journal of Computer Vision Vol. 76, No. 2, 123–139, 2008.

Moon, B.; Iglesias-Guitian, J. A.; Yoon, S. E.; Mitchell, K. Adaptive rendering with linear predictions. ACM Transactions on Graphics Vol. 34, No. 4, Article No. 121, 2015.

Krizhevsky, A.; Sutskever, I.; Hinton, G. E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems 25, 1097–1105, 2012.

Szegedy, C.; Liu, W.; Jia, Y. Q.; Sermanet, P.; Reed, S.; Anguelov, D.; Erhan, D.; Vanhoucke, V.; Rabinovich, A. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1–9, 2015.

He, K. M.; Zhang, X. Y.; Ren, S. Q.; Sun, J. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In: Proceedings of the IEEE International Conference on Computer Vision, 1026–1034, 2015.

Xie, J.; Xu, L.; Chen, E. Image denoising and inpainting with deep neural networks. In: Proceedings of the Advances in Neural Information Processing Systems 25, 341–349, 2012.

Iizuka, S.; Simo-Serra, E.; Ishikawa, H. Globally and locally consistent image completion. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 107, 2017.

Xu, L.; Ren, J. S.; Liu, C.; Jia, J. Deep convolutional neural network for image deconvolution. In: Proceedings of the Advances in Neural Information Processing Systems 27, 1790–1798, 2014.

Shi, W. Z.; Caballero, J.; Huszar, F.; Totz, J.; Aitken, A. P.; Bishop, R.; Rueckert, D.; Wang, Z. Real-time single image and video super-resolution using an efficient subpixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1874–1883, 2016.

Ledig, C.; Theis, L.; Huszar, F.; Caballero, J.; Cunningham, A.; Acosta, A.; Aitken, A.; Tejani, A.; Totz, J.; Wang, Z.; Shi, W. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 105–114, 2017.

Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.

Glorot, X.; Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, 249–256, 2010.

Nah, S.; Kim, T. H.; Lee, K. M. Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 257–265, 2017.

Xu, R. F.; Pattanaik, S. N. A novel Monte Carlo noise reduction operator. IEEE Computer Graphics and Applications Vol. 25, No. 2, 31–35, 2005.

Shirley, P.; Timo, A. L.; Cohen, J.; Enderton, E.; Laine, S.; Luebke, D.; McGuire, M. A local image reconstruction algorithm for stochastic rendering. In: Proceedings of the Symposium on Interactive 3D Graphics and Games, 9–14, 2011.

Bauszat, P.; Eisemann, M.; Magnor, M. Guided image filtering for interactive high-quality global illumination. Computer Graphics Forum Vol. 30, No. 4, 1361–1368, 2011.

Rousselle, F.; Knaus, C.; Zwicker, M. Adaptive sampling and reconstruction using greedy error minimization. ACM Transactions on Graphics Vol. 30, No. 6, Article No. 159, 2011.

Lim, B.; Son, S.; Kim, H.; Nah, S.; Lee, K. M. Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 1132–1140, 2017.

Mao, X.; Shen, C.; Yang, Y.-B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Proceedings of the Advances in Neural Information Processing Systems 29, 2802–2810, 2016.

Zeiler, M. D.; Fergus, R. Visualizing and understanding convolutional networks. In: Computer Vision - ECCV 2014. Lecture Notes in Computer Science, Vol. 8689. Fleet, D.; Pajdla, T.; Schiele, B.; Tuytelaars, T. Eds. Springer Cham, 818–833, 2014.

Yu, D.; Eversole, A.; Seltzer, M. L.; Yao, K.; Huang, Z.; Guenter, B.; Kuchaiev, O.; Zhang, Y.; Seide, F.; Wang, H. et al. An introduction to computational networks and the computational network toolkit. Microsoft Technical Report MSR-TR-2014-112. 2014.

CNTK. Microsoft cognitive toolkit. 2018.

Gritz, L. Openimageio software. 2008.

Van der Walt, S.; Colbert, S. C.; Varoquaux, G. The NumPy array: A structure for efficient numerical computation. Computing in Science & Engineering Vol. 13, No. 2, 22–30, 2011.

Kingma, D. P.; Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.

Zhao, H.; Gallo, O.; Frosio, I.; Kautz, J. Loss functions for image restoration with neural networks. IEEE Transactions on Computational Imaging Vol. 3, No. 1, 47–57, 2017.

Chaitanya, C. R. A.; Kaplanyan, A. S.; Schied, C.; Salvi, M.; Lefohn, A.; Nowrouzezahrai, D.; Aila, T. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics Vol. 36, No. 4, Article No. 98, 2017.

Johnson, J.; Alahi, A.; Li, F. F. Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol. 9906. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 694–711, 2016.

Srivastava, R. K.; Greff, K.; Schmidhuber J. Highway networks. arXiv preprint arXiv:1505.00387, 2015.

Huang, G.; Sun, Y.; Liu, Z.; Sedra, D.; Weinberger, K. Q. Deep networks with stochastic depth. In: Computer Vision - ECCV 2016. Lecture Notes in Computer Science, Vol 9908. Leibe, B.; Matas, J.; Sebe, N.; Welling, M. Eds. Springer Cham, 646–661, 2016.

Zagoruyko, S.; Komodakis, N. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016.

Wang, Z.; Bovik, A. C; Sheikh, H. R.; Simoncelli, E. P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing Vol. 13, No. 4, 600–612, 2004.

Briggs, W. L.; McCormick, S. F. A Multigrid Tutorial, Vol. 72. Siam, 2000.

Zimmer, H.; Rousselle, F.; Jakob, W.; Wang, O.; Adler, D.; Jarosz, W.; Sorkine-Hornung, O.; Sorkine-Hornung, A. Path-space motion estimation and decomposition for robust animation filtering. Computer Graphics Forum Vol. 34, No. 4, 131–142, 2015.