Advances in surrogate based modeling, feasibility analysis, and optimization: A review

Adi, 2016, An effective computation strategy for assessing operational flexibility of high-dimensional systems with complicated feasible regions, Chem. Eng. Sci., 147, 137, 10.1016/j.ces.2016.03.028

Akaike, 1974, A new look at the statistical model identification, IEEE Trans. Autom. Control, 19, 716, 10.1109/TAC.1974.1100705

Anthony, 1997

Bachoc, 2013, Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model misspecification, Comput. Stat. Data Anal., 66, 55, 10.1016/j.csda.2013.03.016

Balabanov, 1998, Multifidelity response surface model for HSCT wing bending material weight, Proceedings of 7th …, 1

Banerjee, 2008, Gaussian predictive process models for large spatial data sets, J. R. Stat. Soc. Ser. B: Stat. Methodol., 70, 825, 10.1111/j.1467-9868.2008.00663.x

Banerjee, 2010, Computationally efficient black-box modeling for feasibility analysis, Comput. Chem. Eng., 34, 1515, 10.1016/j.compchemeng.2010.02.016

Barton, 2006, Chapter 18 metamodel-based simulation optimization, Handbooks Oper. Res. Manag. Sci., 13, 535, 10.1016/S0927-0507(06)13018-2

Barton, 1992, Metamodels for simulation input-output relations, 289

Bertsimas, 2016, OR Forum—an algorithmic approach to linear regression, Oper. Res., 64, 2, 10.1287/opre.2015.1436

Bischl, 2012, Resampling methods for meta-model validation with recommendations for evolutionary computation, Evol. Comput., 20, 249, 10.1162/EVCO_a_00069

Bischl, 2016, mlr: Machine learning in r, J. Mach. Learn. Res., 17, 1

Björkman, 2000, Global optimization of costly nonconvex functions using radial basis functions, Optim. Eng., 1, 373, 10.1023/A:1011584207202

Bloch, 2003, Neural networks for process control and optimization: two industrial applications, ISA Trans., 42, 39, 10.1016/S0019-0578(07)60112-8

Bouhlel, 2015, 935

Boukouvala, 2016, ARGONAUT: AlgoRithms for Global Optimization of coNstrAined grey-box compUTational problems, Optim. Lett.

Boukouvala, 2012, Feasibility analysis of black-box processes using an adaptive sampling Kriging-based method, Comput. Chem. Eng., 36, 358, 10.1016/j.compchemeng.2011.06.005

Boukouvala, 2014, Derivative-free optimization for expensive constrained problems using a novel expected improvement objective function, AIChE J., 60, 2462, 10.1002/aic.14442

Boukouvalas, 2009, Learning heteroscedastic gaussian processes for complex datasets, Group, 44

Breiman, 1995, Better subset regression using the nonnegative garrote, Technometrics, 10.1080/00401706.1995.10484371

Cadima, 2004, Computational aspects of algorithms for variable selection in the context of principal components, Comput. Stat. Data Anal., 47, 225, 10.1016/j.csda.2003.11.001

Cadima, J., Cerdeira, J. O., Silva, P. D., & Minhoto, M., 2012. The subselect R package.

Candes, 2007, The Dantzig selector: statistical estimation when p is much larger than n, Ann. Stat., 35, 2313, 10.1214/009053606000001523

Chen, 2013, Analysis methods for computer experiments: how to assess and what counts?, Stat. Sci., 31, 40, 10.1214/15-STS531

Chen, 2014, Extended bayesian information criteria for model selection with large model spaces model selection extended Bayesian information criteria for with large model spaces, Biometrika, 95, 759, 10.1093/biomet/asn034

Chu, 2011, Penalized maximum likelihood estimation and variable selection in geostatistics, Ann. Stat., 39, 2607, 10.1214/11-AOS919

Clarke, 2005, Analysis of support vector regression for approximation of complex engineering analyses, J. Mech. Des., 127, 1077, 10.1115/1.1897403

Conn, 1997, Recent progress in unconstrained nonlinear optimization without derivatives, Math. Program., 79, 397, 10.1007/BF02614326

Cozad, 2014, Learning surrogate models for simulation-based optimization, AIChE J., 60, 10.1002/aic.14418

Cozad, 2015, A combined first-principles and data-driven approach to model building, Comput. Chem. Eng., 73, 116, 10.1016/j.compchemeng.2014.11.010

Cressie, 2008, Fixed rank kriging for very large spatial data sets, J. R. Stat. Soc. Ser. B (Stat. Methodol.), 70, 209, 10.1111/j.1467-9868.2007.00633.x

Crombecq, 2011, Efficient space-filling and non-collapsing sequential design strategies for simulation-based modeling, Eur. J. Oper. Res., 214, 683, 10.1016/j.ejor.2011.05.032

Dolan, 2002, Benchmarcking optimization software with performance profiles, Math. Programm., 91, 201, 10.1007/s101070100263

Eason, 2014, Adaptive sequential sampling for surrogate model generation with artificial neural networks, Comput. Chem. Eng., 68, 220, 10.1016/j.compchemeng.2014.05.021

Efron, 2004, Least angle regression, The Ann. Stat., 32, 407, 10.1214/009053604000000067

Fan, 2001, Variable selection via nonconcave penalized, J. Am. Stat. Assoc., 96, 1348, 10.1198/016214501753382273

Fan, 2008, Sure independence screening for ultrahigh dimensional feature space, J. R. Stat. Soc. Ser. B: Stat. Methodol., 70, 849, 10.1111/j.1467-9868.2008.00674.x

Fernandes, 2006, Optimization of fischer-tropsch synthesis using neural networks, Chem. Eng. Technol., 29, 449, 10.1002/ceat.200500310

Floudas, 2001, Global optimization in design under uncertainty: feasibility test and flexibility index problems, Ind. Eng. Chem. Res., 40, 4267, 10.1021/ie001014g

Forrester, 2009, Recent advances in surrogate-based optimization, Progr. Aerospace Sci., 45, 50, 10.1016/j.paerosci.2008.11.001

Foster, 1994, The risk inflation criterion for multiple regression, The Ann. Stat., 22, 1947, 10.1214/aos/1176325766

Furnival, 1974, Regressions by leaps and bounds, Technometrics, 16, 499, 10.1080/00401706.1974.10489231

Furrer, 2006, Covariance tapering for interpolation of large spatial datasets, J. Comput. Graph. Stat., 15, 502, 10.1198/106186006X132178

GLOBAL Library. http://www.gamsworld.org/global/globallib.htm.

Garud, 2017, Smart sampling algorithm for surrogate model development, Comput. Chem. Eng., 96, 103, 10.1016/j.compchemeng.2016.10.006

Goel, 2007, Ensemble of surrogates, Struct. Multidiscip. Optim., 33, 199, 10.1007/s00158-006-0051-9

Gorissen, 2010, A surrogate modeling and adaptive sampling toolbox for computer based design, J. Mach. Learn. Res., 11, 2051

Goyal, 2002, Determination of operability limits using simplicial approximation, AIChE J., 48, 2902, 10.1002/aic.690481217

Grossmann, 2014, Evolution of concepts and models for quantifying resiliency and flexibility of chemical processes, Comput. Chem. Eng., 70, 22, 10.1016/j.compchemeng.2013.12.013

Gunn, 1998

Gutmann, 2001, A radial basis function method for global optimization, J. Global Optim., 19, 201, 10.1023/A:1011255519438

Haftka, 2016, Parallel surrogate-assisted global optimization with expensive functions? A survey, Struct. Multidiscip. Optim., 54, 3, 10.1007/s00158-016-1432-3

Hannan, 1979, The determination of the order of an autoregression, J. R. Stat. Soc. Ser. B (Methodol.), 41, 190, 10.1111/j.2517-6161.1979.tb01072.x

Hastie, 2009, The elements of statistical learning, Elements, 1, 337

Henao, 2011, Surrogate-based superstructure optimization framework, AIChE J., 57, 1216, 10.1002/aic.12341

Holmström, 2008, An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization, Optim. Eng., 9, 311, 10.1007/s11081-008-9037-3

Hooke, 1960, 212

Huang, 2007, Optimal geostatistical model selection, J. Am. Stat. Assoc., 102, 1009, 10.1198/016214507000000491

Hurvich, 1989, Regression and time series model selection in small samples, Biometrika, 76, 297, 10.1093/biomet/76.2.297

Jia, 2009, Predictive modeling for pharmaceutical processes using Kriging and response surface, J. Pharm. Innov., 4, 174, 10.1007/s12247-009-9070-6

Jin, 2001, Comparative studies of metamodelling techniques under multiple modelling criteria, Struct. Multidiscip. Optim., 23, 1, 10.1007/s00158-001-0160-4

Jones, 1993, Lipschitzian optimization without the Lipschitz constant, J. Optim. Theory Appl., 79, 157, 10.1007/BF00941892

Jones, 1998, Efficient global optimization of expensive black-box functions, J. Global Optim., 13, 455, 10.1023/A:1008306431147

Joseph, 2008, Blind kriging: a new method for developing metamodels, J. Mech. Des., 130, 1, 10.1115/1.2829873

Kaufman, 2008, Covariance tapering for likelihood-based estimation in large spatial data sets, J. Am. Stat. Assoc., 103, 1545, 10.1198/016214508000000959

Kersting, 2007, Most likely heteroscedastic gaussian process regression, 24th International Conference on Machine Learning (ICML 2007), 393

Konno, 2009, Choosing the best set of variables in regression analysis using integer programming, J. Global Optim., 44, 273, 10.1007/s10898-008-9323-9

Konno, 2009, Choosing the best set of variables in regression analysis using integer programming, J. Global Optim., 44, 273, 10.1007/s10898-008-9323-9

Krige, 1951

Liang, 2013, A resampling-based stochastic approximation method for analysis of large geostatistical data, J. Am. Stat. Assoc., 108, 325, 10.1080/01621459.2012.746061

Linhart, 1986

Liu, 2007

Lophaven, S., Nielsen, H., & Sondergaard, J., 2002. DACE: A MATLAB Kriging Toolbox. In Version 2.0, Tech. Rep. IMMTR-2002-12, Informatics and Mathematical Modelling.

Lophaven, S., Nielsen, H., & Søndergaard, J., 2002. Aspects of the matlab toolbox DACE. Technical Report, (IMM-REP-2002-13).

Müller, 2011, Mixture surrogate models based on Dempster-Shafer theory for global optimization problems, J. Global Optim., 51, 79, 10.1007/s10898-010-9620-y

Müller, 2015, CH4 parameter estimation in CLM4.5bgc using surrogate global optimization, Geosci. Model Dev., 8, 3285, 10.5194/gmd-8-3285-2015

Mallows, 1973, Some comments on CP, Technometrics, 15, 661

Matheron, 1963, Principles of geostatistics, Econ. Geol., 10.2113/gsecongeo.58.8.1246

McKay, 1979, Comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics, 21, 239

Meert, 1998, Intelligent modelling in the chemical process industry with neural networks: a case study, Comput. Chem. Eng., 22, S587, 10.1016/S0098-1354(98)00104-5

Misener, 2014, ANTIGONE: algorithms for coNTinuous/Integer global optimization of nonlinear equations, J. Global Optim., 59, 503, 10.1007/s10898-014-0166-2

Mujtaba, 2006, Neural network based modelling and control in batch reactor, Chem. Eng. Res. Des., 84, 635, 10.1205/cherd.05096

Nelder, 1965, A simplex method for function minimization, Comput. J., 7, 308, 10.1093/comjnl/7.4.308

Nippgen, 2016, Water resources research, Water Resour. Res., 1

Nychka, 2015, A multiresolution gaussian process model for the analysis of large spatial datasets, J. Comput. Graph. Stat., 24, 579, 10.1080/10618600.2014.914946

Oeuvray, 2009, Boosters: a derivative-free algorithm based on radial basis functions, Int. J. Model. Simul., 29, 26, 10.1080/02286203.2009.11442507

Palmer, 2002, Metamodeling approach to optimization of steady-state flowsheet simulations, Chem. Eng. Res. Des., 80, 760, 10.1205/026387602320776830

Park, Y. W., & Klabjan, D., 2013. Subset Selection for Multiple Linear Regression via Optimization, (i), 1–27.

Pedregosa, 2011, Scikit-learn: machine learning in {P}ython, J. Mach. Learn. Res., 12, 2825

Powell, 1994, A direct search optimization method that models the objective and constraint functions by linear interpolation, 51

Powell, 1994, The NEWUOA software for unconstrained optimization without derivatives, 255

Prebeg, 2014, Application of a surrogate modeling to the ship structural design, Ocean Eng., 84, 259, 10.1016/j.oceaneng.2014.03.032

Princeton Library. http://www.gamsworld.org/performance/princetonlib/princetonlib.htm.

Provost, 1999, Efficient progressive sampling, 23

Queipo, 2005, Surrogate-based analysis and optimization, Prog. Aerosp. Sci., 41, 1, 10.1016/j.paerosci.2005.02.001

Ranjan, 2011, A computationally stable approach to gaussian process interpolation of deterministic computer simulation data, Technometrics, 53, 366, 10.1198/TECH.2011.09141

Rasmussen, 2006

Rasmussen, 2006

Razavi, 2012, Review of surrogate modeling in water resources, Water Resour. Res., 48, 10.1029/2011WR011527

Regis, 2005, Constrained global optimization of expensive black box functions using radial basis functions, J. Global Optim., 31, 153, 10.1007/s10898-004-0570-0

Regis, 2007, Improved strategies for radial basis function methods for global optimization, J. Global Optim., 37, 113, 10.1007/s10898-006-9040-1

Regis, R. G., & Wild, S. M., 2015. CONORBIT: Constrained optimization by radial basis function interpolation in trust regions 1 CONORBIT: Constrained optimization by radial basis function interpolation in trust regions, (October).

Regis, 2016, Trust regions in Kriging-based optimization with expected improvement, Eng. Optim., 48, 1037, 10.1080/0305215X.2015.1082350

Riolo, 2010

Riolo, 2011

Rios, 2013, Derivative-free optimization: a review of algorithms and comparison of software implementations, J. Global Optim., 56, 1247, 10.1007/s10898-012-9951-y

Rogers, 2015, Feasibility and flexibility analysis of black-box processes part 2: Surrogate-based flexibility analysis, Chem. Eng. Sci., 137, 1005, 10.1016/j.ces.2015.06.026

Sacks, 1989, Design and analysis of computer experiments, Stat. Sci., 4, 409, 10.1214/ss/1177012413

Sang, 2012, A full scale approximation of covariance functions for large spatial data sets, J. R. Stat. Soc., 1988, 111, 10.1111/j.1467-9868.2011.01007.x

Schmidt, 2009, Distilling natural laws, Science, 324, 81, 10.1126/science.1165893

Schwarz, 1978, Estimating the dimension of a model, Ann. Stat., 6, 461, 10.1214/aos/1176344136

Seber, 2003

Simpson, 1997, On the use of statistics in design and the implications for deterministic computer experiments, Proceedings of DETC’97 1997 ASME Design Engineering Technical Conferences, 1

Smola, 2004, A tutorial on support vector regression, Stat. Comput., 14, 199, 10.1023/B:STCO.0000035301.49549.88

Snelson, E., & Ghahramani, Z., 2007. Local and global sparse Gaussian process approximations. In M. Meila & X. Shen (Eds.), Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS-07) (Vol. 2, pp. 524–531). Journal of Machine Learning Research − Proceedings Track. Retrieved from http://jmlr.csail.mit.edu/proceedings/papers/v2/snelson07a/snelson07a.pdf.

Sobol, 1967, on of points in a cube and the approximate evaluation of integrals, Zh. Vychisl. Mat. I Mat. Fiz., 7, 784

Straub, 1993, Design optimization of stochastic flexibility, Comput. Chem. Eng., 17, 339, 10.1016/0098-1354(93)80025-I

Swaney, 1985, An index for operational flexibility in chemical process design. Part I: Formulation and theory, AIChE J., 31, 621, 10.1002/aic.690310412

Tajbakhsh, S., Aybat, N., & Castillo, E. Del., 2014. Sparse Precision Matrix Selection for Fitting Gaussian Random Field Models to Large Data Sets. arXiv Preprint arXiv:1405.5576, 25(Ml), 1–18. http://doi.org/10.1007/s10107-014-0826-5.

Tawarmalani, 2005, A polyhedral branch-and-cut approach to global optimization, Math. Program., 103, 225, 10.1007/s10107-005-0581-8

Thacker, 2010, Algorithm 905: SHEPPACK: modified shepard algorithm for interpolation of scattered multivariate data, ACM Trans. Math. Softw., 37, 34, 10.1145/1824801.1824812

Toal, 2009, An adjoint for likelihood maximization, Proc. R. Society A Math. Phys. Eng. Sci., 465, 3267, 10.1098/rspa.2009.0096

Toal, 2011, The development of a hybridized particle swarm for kriging hyperparameter tuning, Eng. Optim., 43, 675, 10.1080/0305215X.2010.508524

Viana, 2009, Multiple surrogates: how cross-validation errors can help us to obtain the best predictor, Struct. Multidiscip. Optim., 39, 439, 10.1007/s00158-008-0338-0

Viana, 2013, Efficient global optimization algorithm assisted by multiple surrogate techniques, J. Global Optim., 56, 669, 10.1007/s10898-012-9892-5

Viana, F. A. C., 2010. SURROGATES Toolbox User’s Guide, version 2.1.

Wang, 2016, A novel feasibility analysis method for black-box processes using a radial basis function adaptive sampling approach, AIChE J.

Wang, 2007, Review of metamodeling techniques in support of engineering design optimization, J. Mech. Des., 129, 370, 10.1115/1.2429697

Wild, 2013, Global convergence of radial basis function trust-region algorithms for derivative-free optimization, SIAM Rev., 55, 349, 10.1137/120902434

Wild, 2008, ORBIT: optimization by radial basis function interpolation in trust-Regions, SIAM J. Sci. Comput., 30, 3197, 10.1137/070691814

Wilson, 2017, The ALAMO approach to machine learning, Comput. Chem. Eng., 10.1016/j.compchemeng.2017.02.010

Yang, 2005, Metamodeling development for vehicle frontal impact simulation, J. Mech. Des., 127, 1014, 10.1115/1.1906264

Yin, 2011, Kriging metamodel with modified nugget-effect: the heteroscedastic variance case, Comput. Ind. Eng., 61, 760, 10.1016/j.cie.2011.05.008

Zerpa, 2005, An optimization methodology of alkaline-surfactant-polymer flooding processes using field scale numerical simulation and multiple surrogates, J. Petrol. Sci. Eng., 47, 197, 10.1016/j.petrol.2005.03.002

Zhang, 2016, vol. 17

Zou, 2008

Zou, 2006, The adaptive lasso and its oracle properties, J. Am. Stat. Assoc., 101, 1418, 10.1198/016214506000000735