Neural Networks and the Bias/Variance Dilemma

Neural Computation - Tập 4 Số 1 - Trang 1-58 - 1992
Stuart Geman1, Elie Bienenstock2, René Doursat2
1Division of Applied Mathematics, Brown University - Providence, RI 02912 USA
2ESPCI, 10, rue Vauquelin, 75005 Paris, France

Tóm tắt

Feedforward neural networks trained by error backpropagation are examples of nonparametric regression estimators. We present a tutorial on nonparametric inference and its relation to neural networks, and we use the statistical viewpoint to highlight strengths and weaknesses of neural models. We illustrate the main points with some recognition experiments involving artificial data as well as handwritten numerals. In way of conclusion, we suggest that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Furthermore, we suggest that the fundamental challenges in neural modeling are about representation rather than learning per se. This last point is supported by additional experiments with handwritten numerals.

Từ khóa


Tài liệu tham khảo

10.1016/S0364-0213(85)80012-4

10.2307/2290581

10.1037/0033-295X.84.5.413

10.1016/S0734-189X(89)80014-3

10.1016/0893-6080(89)90014-2

Barron A. R., 1989, Florida, 280

10.1162/neco.1990.2.2.248

10.1162/neco.1989.1.1.151

Baum L. E., 1972, Inequalities, 3, 1

10.1007/BF00332918

10.1109/34.62605

10.2307/2288473

Burr D. J., 1980, IEEE Transact. Pattern Anal. Machine Intell. PAMI-3, 6, 708

10.1209/0295-5075/4/10/020

Chauvin Y., 1990, Neural Information Processing Systems 11, D. S. Touretzky, ed., 642

10.1038/337129a0

10.1007/BF02551274

Dempster A., 1976, J. R. Statist. SOC. B, 39, 1

Denker J., 1987, Complex Syst., 1, 877

10.1214/aos/1176349830

10.1214/aop/1176991978

10.2307/2289609

10.1109/T-C.1973.223602

10.1016/0010-0277(88)90031-5

10.1214/aos/1176347963

10.2307/2287576

10.1016/0893-6080(89)90003-8

10.1214/aos/1176345782

Goldman L., 1982, J. Med., 307, 588

10.1007/BF02591360

10.1109/34.58871

10.2307/2288922

10.1162/neco.1990.2.2.210

10.1162/neco.1990.2.3.355

10.1016/0893-6080(89)90020-8

10.1214/aos/1176349519

10.1109/72.80236

10.1162/neco.1989.1.4.541

10.1162/neco.1989.1.1.1

10.1007/BF01972448

10.1162/neco.1991.3.2.258

10.1016/0021-9991(85)90121-4

10.1126/science.247.4945.978

10.1038/323533a0

10.1162/neco.1990.2.3.374

10.2307/2289391

10.1017/S0140525X00052432

Stone M., 1974, J. R. Statist. SOC., 36, 111

10.2307/2288030

10.1109/TMI.1987.4307849

10.1007/BFb0098499

10.1214/aos/1176349743

10.1080/03610927508827223

10.1162/neco.1989.1.4.425

10.1016/0893-6080(90)90004-5

10.1016/0031-3203(73)90042-3

10.1162/neco.1990.2.1.1

Thông tin
Thông tin xuất bản

Neural Computation

Tập 4 Số 1

1-58

Thông tin tác giả