Overcoming catastrophic forgetting in neural networks

10.1007/s11023-007-9079-x

10.1016/S1364-6613(99)01294-2

M McCloskey, NJ Cohen, Catastrophic interference in connectionist networks: The sequential learning problem. The Psychology of Learning and Motivation, ed GH Bower (Academic, New York) Vol 24, 109–165 (1989).

10.1037/0033-295X.102.3.419

10.1016/j.tics.2016.05.004

10.1037/0033-295X.97.2.285

A Krizhevsky, I Sutskever, GE Hinton, Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25, eds F Pereira, CJC Burges, L Bottou, KQ Weinberger (Curran Assoc, Red Hook, NY), pp. 1097–1105 (2012).

10.1038/nature14539

AA Rusu Policy distillation. arXiv:1511.06295. (2015).

E Parisotto JL Ba R Salakhutdinov Actor-mimic: Deep multitask and transfer reinforcement learning. arXiv:1511.06342. (2015).

10.1038/nature14251

10.1038/nature15257

10.1038/nature08577

10.1126/science.1249098

10.1016/j.neuron.2005.02.001

MK Benna S Fusi Computational principles of biological memory. arXiv:1507.07580. (2015).

R Hecht-Nielsen, Theory of the backpropagating network. Neural Netw (Suppl 1), pp. 445–448 (1988).

10.1016/S0893-6080(05)80037-1

10.1162/neco.1992.4.3.448

R Pascanu Y Bengio Revisiting natural gradient for deep networks. arXiv:1301.3584. (2013).

E Eskin, AJ Smola, S Vishwanathan, Laplace propagation. Advances in Neural Information Processing Systems 16, eds S Thrun, LK Saul, PB Schoelkopf (MIT Press, Cambridge, MA), pp. 441–448 (2004).

Y LeCun C Cortes CJ Burges The MNIST database of handwritten digits. Available at yann.lecun.com/exdb/mnist/. Accessed March 3 2017. (1998).

RK Srivastava, J Masci, S Kazerounian, F Gomez, J Schmidhuber, Compete to compute. Advances in Neural Information Processing Systems 26, eds CJC Burges, L Bottou, M Welling, Z Ghahramani, KQ Weinberg (Curran Assoc, Red Hook, NY) Vol 26, 2310–2318 (2013).

IJ Goodfellow M Mirza D Xiao A Courville Y Bengio An empirical investigation of catastrophic forgeting in gradient-based neural networks. arXiv:1312.6211. (2015).

10.1038/nature14236

10.1613/jair.3912

10.1007/978-1-4615-5529-2_11

AA Rusu Progressive neural networks. arXiv:1606.04671. (2016).

10.1037/a0030852

10.1162/089976606775093909

10.1038/nature12742

10.1146/annurev.neuro.24.1.167

10.1162/089976602753712972

K Milan The forget-me-not process. Advances in Neural Information Processing Systems 29 eds DD Lee M Sugiyama UV Luxburg I Guyon R Garnett (Curran Assoc Red Hook NY 2016).

10.1162/08997660260028700

PL Ruvolo, E Eaton, ELLA: An efficient lifelong learning algorithm. JMLR Workshop Conf Proc 28, 507–515 (2013).

C Blundell, J Cornebise, K Kavukcuoglu, D Wierstra, Weight uncertainty in neural networks. JMLR Workshop Conf Proc 37, 1613–1622 (2015).

10.1017/CBO9780511623257

L Aitchison PE Latham Synaptic sampling: A connection between PSP variability and uncertainty explains neurophysiological observations. arXiv:1505.04544. (2015).

10.1038/nn.2640

10.1371/journal.pcbi.1000248

H van Hasselt, A Guez, D Silver, Deep reinforcement learning with double q-learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, eds D Schuurmans, M Wellman (AAAI Press, Palo Alto, CA), pp. 2094–2100 (2016).

10.1109/DCC.2012.39

10.1016/0047-259X(82)90077-X