Evolutionary optimization of machine learning algorithm hyperparameters for strength prediction of high-performance concrete

Ahmad, A., Ostrowski, K. A., Maślak, M., Farooq, F., Mehmood, I., & Nafees, A. (2021). Comparative study of supervised machine learning algorithms for predicting the compressive strength of concrete at high temperature. Materials, 14, 4222. https://doi.org/10.3390/ma14154222 Aïtcin, P.-C. (1998). High performance concrete. CRC Press. https://doi.org/10.4324/9780203475034 Al Yamani, W. H., Ghunimat, D. M., & Bisharah, M. M. (2023). Modeling and predicting the sensitivity of high-performance concrete compressive strength using machine learning methods. Asian Journal of Civil Engineering. https://doi.org/10.1007/s42107-023-00614-4 Alqahtani, M., Gumaei, A., Mathkour, H., & Maher Ben Ismail, M. (2019). A genetic-based extreme gradient boosting model for detecting intrusions in wireless sensor networks. Sensors, 19, 4383. https://doi.org/10.3390/s19204383 Andonie, R. (2019). Hyperparameter optimization in learning systems. Journal of Membrane Computing, 1, 279–291. https://doi.org/10.1007/s41965-019-00023-0 Anyaoha, U., Zaji, A., & Liu, Z. (2020). Soft computing in estimating the compressive strength for high-performance concrete via concrete composition appraisal. Construction and Building Materials, 257, 119472. https://doi.org/10.1016/j.conbuildmat.2020.119472 Awad, M., & Khanna, R. (2015). Support vector regression. In M. Awad & R. Khanna (Eds.), Efficient learning machines: Theories, concepts, and applications for engineers and system designers (pp. 67–80). Apress. https://doi.org/10.1007/978-1-4302-5990-9_4 Bebis, G., & Georgiopoulos, M. (1994). Feed-forward neural networks. IEEE Potentials, 13, 27–31. https://doi.org/10.1109/45.329294 Bergstra, J., Bardenet, R., Bengio, Y., & Kégl, B. (2011). Algorithms for hyper-parameter optimization. Advances in Neural Information Processing Systems, 24. Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13. Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford University Press. Browne, M. W. (2000). Cross-Validation methods. Journal of Mathematical Psychology, 44, 108–132. https://doi.org/10.1006/jmps.1999.1279 Bui, D.-K., Nguyen, T., Chou, J.-S., Nguyen-Xuan, H., & Ngo, T. D. (2018). A modified firefly algorithm-artificial neural network expert system for predicting compressive and tensile strength of high-performance concrete. Construction and Building Materials, 180, 320–333. https://doi.org/10.1016/j.conbuildmat.2018.05.201 Chang, T., Chuang, F., & Lin, H. (1996). A mix proportioning methodology for high-performance concrete. Journal of the Chinese Institute of Engineers, 19, 645–655. https://doi.org/10.1080/02533839.1996.9677830 Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16. Association for Computing Machinery, New York, NY, USA, pp. 785–794. https://doi.org/10.1145/2939672.2939785 Chen, T., & He, T. (n.d.). XGBoost: eXtreme gradient boosting. Chou, J.-S., Chiu, C.-K., Farfoura, M., & Al-Taharwa, I. (2011). Optimizing the prediction accuracy of concrete compressive strength based on a comparison of data-mining techniques. Journal of Computing in Civil Engineering, 25, 242–253. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000088 Chou, J.-S., Chong, W. K., & Bui, D.-K. (2016). Nature-inspired metaheuristic regression system: Programming and implementation for civil engineering applications. Journal of Computing in Civil Engineering, 30, 04016007. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000561 Chou, J.-S., & Pham, A.-D. (2013). Enhanced artificial intelligence for ensemble approach to predicting high performance concrete compressive strength. Construction and Building Materials, 49, 554–563. https://doi.org/10.1016/j.conbuildmat.2013.08.078 Chou, J.-S., & Tsai, C.-F. (2012). Concrete compressive strength analysis using a combined classification and regression technique. Automation in Construction, 24, 52–60. https://doi.org/10.1016/j.autcon.2012.02.001 Chou, J.-S., Tsai, C.-F., Pham, A.-D., & Lu, Y.-H. (2014). Machine learning in concrete strength simulations: Multi-nation data analytics. Construction and Building Materials, 73, 771–780. https://doi.org/10.1016/j.conbuildmat.2014.09.054 Claesen, M., & De Moor, B. (2015). Hyperparameter search in machine learning. https://doi.org/10.48550/arXiv.1502.02127 Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273–297. https://doi.org/10.1007/BF00994018 Duan, J., Asteris, P. G., Nguyen, H., Bui, X.-N., & Moayedi, H. (2021). A novel artificial intelligence technique to predict compressive strength of recycled aggregate concrete using ICA-XGBoost model. Engineering with Computers, 37, 3329–3346. https://doi.org/10.1007/s00366-020-01003-0 Eiben, A. E., & Smith, J. E. (n.d.). Introduction to evolutionary computing. Engen, M., Hendriks, M. A. N., Köhler, J., Øverli, J. A., Åldstedt, E., Mørtsell, E., Sæter, Ø., & Vigre, R. (2018). Predictive strength of ready-mixed concrete: Exemplified using data from the Norwegian market. Structural Concrete, 19, 806–819. https://doi.org/10.1002/suco.201700950 Feng, D.-C., Liu, Z.-T., Wang, X.-D., Chen, Y., Chang, J.-Q., Wei, D.-F., & Jiang, Z.-M. (2020). Machine learning-based compressive strength prediction for concrete: An adaptive boosting approach. Construction and Building Materials, 230, 117000. https://doi.org/10.1016/j.conbuildmat.2019.117000 Ferreira, A. J., & Figueiredo, M. A. T. (2012). Boosting algorithms: A review of methods, theory, and applications. In C. Zhang & Y. Ma (Eds.), Ensemble machine learning: methods and applications (pp. 35–85). Springer US. https://doi.org/10.1007/978-1-4419-9326-7_2 Fushiki, T. (2011). Estimation of prediction error by using K-fold cross-validation. Statistics and Computing, 21, 137–146. https://doi.org/10.1007/s11222-009-9153-8 Geng, X., Moayedi, H., Pan, F., & Foong, L. K. (2021). Predicting the concrete compressive strength through MLP network hybridized with three evolutionary algorithms. Smart Structures and Systems, 28, 711–725. https://doi.org/10.12989/sss.2021.28.5.711 Ghunimat, D., Alzoubi, A. E., Alzboon, A., & Hanandeh, S. (2023). Prediction of concrete compressive strength with GGBFS and fly ash using multilayer perceptron algorithm, random forest regression and k-nearest neighbor regression. Asian Journal of Civil Engineering, 24, 169–177. https://doi.org/10.1007/s42107-022-00495-z Gumus, M., & Kiran, M. S. (2017). Crude oil price forecasting using XGBoost. In: 2017 International Conference on Computer Science and Engineering (UBMK). Presented at the 2017 International Conference on Computer Science and Engineering (UBMK), pp. 1100–1103. https://doi.org/10.1109/UBMK.2017.8093500 Hutter, F., Kotthoff, L., & Vanschoren, J. (Eds.). (2019). Automated machine learning: Methods, systems, challenges, the springer series on challenges in machine learning. Springer International Publishing. https://doi.org/10.1007/978-3-030-05318-5 Jansen, B. J. (1998). The graphical user interface. Jones, E., Oliphant, T., & Peterson, P. (2001). SciPy: Open source scientific tools for Python. Jothilakshmi, S., & Gudivada, V. N. (2016). Chapter 10—Large scale data enabled evolution of spoken language research and applications. In V. N. Gudivada, V. V. Raghavan, V. Govindaraju, & C. R. Rao (Eds.), Handbook of statistics, cognitive computing: theory and applications (pp. 301–340). Elsevier. https://doi.org/10.1016/bs.host.2016.07.005 Kaveh, A. (2017). Advances in metaheuristic algorithms for optimal design of structures. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-46173-1 Kaveh, A., & Khalegi, A. (1998). Prediction of strength for concrete specimens using artificial neural networks. Advances in Engineering Computational Technology, 165–171. Kaveh, A., Bakhshpoori, T., & Hamze-Ziabari, S. M. (2018). GMDH-based prediction of shear strength of FRP-RC beams with and without stirrups. Computers and Concrete, an International Journal, 22, 197–207. Kaveh, A., & Bondarabady, H. A. R. (2004). Wavefront reduction using graphs, neural networks and genetic algorithm. International Journal for Numerical Methods in Engineering, 60, 1803–1815. https://doi.org/10.1002/nme.1023 Kaveh, A., Gholipour, Y., & Rahami, H. (2008). Optimal design of transmission towers using genetic algorithm and neural networks. International Journal of Space Structures, 23, 1–19. https://doi.org/10.1260/026635108785342073 Kaveh, A., & Khavaninzadeh, N. (2023). Efficient training of two ANNs using four meta-heuristic algorithms for predicting the FRP strength. Structures, 52, 256–272. https://doi.org/10.1016/j.istruc.2023.03.178 Kaveh, A., & Servati, H. (2001). Design of double layer grids using backpropagation neural networks. Computers & Structures, 79, 1561–1568. https://doi.org/10.1016/S0045-7949(01)00034-7 Keshtegar, B., Nehdi, M. L., Kolahchi, R., Trung, N.-T., & Bagheri, M. (2022). Novel hybrid machine leaning model for predicting shear strength of reinforced concrete shear walls. Engineering with Computers, 38, 3915–3926. https://doi.org/10.1007/s00366-021-01302-0 Kosmatka, S. H., Panarese, W. C., & Kerkhoff, B. (2002). Design and control of concrete mixtures. Portland Cement Association Skokie. Kuhn, M., & Johnson, K. (2013). Applied predictive modeling. Springer. Laskar, A. I. (2011). Mix design of high-performance concrete. Materials Research, 14, 429–433. Le, T.-T., Asteris, P. G., & Lemonis, M. E. (2022). Prediction of axial load capacity of rectangular concrete-filled steel tube columns using machine learning techniques. Engineering with Computers, 38, 3283–3316. https://doi.org/10.1007/s00366-021-01461-0 Lim, C.-H., Yoon, Y.-S., & Kim, J.-H. (2004). Genetic algorithm in mix proportioning of high-performance concrete. Cement and Concrete Research, 34, 409–420. https://doi.org/10.1016/j.cemconres.2003.08.018 Love, B. C. (2002). Comparing supervised and unsupervised category learning. Psychonomic Bulletin & Review, 9, 829–835. https://doi.org/10.3758/BF03196342 Mirjalili, S. (2019). Genetic algorithm. In S. Mirjalili (Ed.), Evolutionary algorithms and neural networks: Theory and applications, studies in computational intelligence (pp. 43–55). Springer International Publishing. https://doi.org/10.1007/978-3-319-93025-1_4 Mitchell, M. (1996). An introduction to genetic algorithms, complex adaptive systems. A Bradford Book. Moore, A. D. (2018). Python GUI programming with Tkinter: Develop responsive and powerful GUI applications with Tkinter. Packt Publishing Ltd. Nguyen, H., Vu, T., Vo, T. P., & Thai, H.-T. (2021). Efficient machine learning models for prediction of concrete strengths. Construction and Building Materials, 266, 120950. https://doi.org/10.1016/j.conbuildmat.2020.120950 Oliphant, T. E. (2006). Guide to NumPy. Oluokun, F. A. (1994). Fly ash concrete mix design and the water-cement ratio law. Materials Journal, 91, 362–371. Parhi, S. K., & Patro, S. K. (2023). Prediction of compressive strength of geopolymer concrete using a hybrid ensemble of grey wolf optimized machine learning estimators. Journal of Building Engineering, 71, 106521. https://doi.org/10.1016/j.jobe.2023.106521 Parichatprecha, R., & Nimityongskul, P. (2009). Analysis of durability of high performance concrete using artificial neural networks. Construction and Building Materials, 23, 910–917. https://doi.org/10.1016/j.conbuildmat.2008.04.015 Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D. (n.d.). Scikit-learn: Machine learning in python. Machine Learning in Python. Probst, P., Boulesteix, A.-L., & Bischl, B. (2018.). Tunability: Importance of hyperparameters of machine learning algorithms. Rafiei, M. H., Khushefati, W. H., Demirboga, R., & Adeli, H. (2017). Novel approach for concrete mixture design using neural dynamics model and virtual lab concept. ACI Materials Journal, 114. Reback, J., McKinney, W., Jbrockmendel, Van Den Bossche, J., Augspurger, T., Cloud, P., Gfyoung, Sinhrks, Klein, A., Hawkins, S., Roeschke, M., Tratner, J., She, C., Ayd, W., Petersen, T., MomIsBestFriend, Garcia, M., Schendel, J., Hayden, A., Jancauskas, V., Battiston, P., Saxton, D., Seabold, S., Alimcmaster1, Chris-B1, H-Vetinari, Hoyer, S., Dong, K., Overmeire, W., & Winkel, M. (2020). pandas-dev/pandas: Pandas 1.0.5. Zenodo. https://doi.org/10.5281/zenodo.3898987 Refaeilzadeh, P., Tang, L., & Liu, H. (2009). Cross-validation. Encyclopedia of Database Systems, 5, 532–538. Reuter, U., Sultan, A., & Reischl, D. S. (2018). A comparative study of machine learning approaches for modeling concrete failure surfaces. Advances in Engineering Software, 116, 67–79. https://doi.org/10.1016/j.advengsoft.2017.11.006 Ruan, W., Shi, X., Hu, J., Hou, Y., Fan, M., Cao, R., & Wei, X. (2018). Modeling of malachite green removal from aqueous solutions by nanoscale zerovalent zinc using artificial neural network. Applied Sciences, 8, 3. https://doi.org/10.3390/app8010003 Salami, B. A., Olayiwola, T., Oyehan, T. A., & Raji, I. A. (2021). Data-driven model for ternary-blend concrete compressive strength prediction using machine learning approach. Construction and Building Materials, 301, 124152. https://doi.org/10.1016/j.conbuildmat.2021.124152 Sazli, M. H. (2006). A brief review of feed-forward neural networks. Communications Faculty of Sciences University of Ankara Series A2-A3 Physical Sciences and Engineering, 50, 0–0. https://doi.org/10.1501/commua1-2_0000000026 Schapire, R. E. (n.d.). A brief introduction to boosting. Shariati, M., Mafipour, M. S., Ghahremani, B., Azarhomayun, F., Ahmadi, M., Trung, N. T., & Shariati, A. (2022). A novel hybrid extreme learning machine–grey wolf optimizer (ELM-GWO) model to predict compressive strength of concrete with partial replacements for cement. Engineering with Computers, 38, 757–779. https://doi.org/10.1007/s00366-020-01081-0 Siddique, R., Aggarwal, P., & Aggarwal, Y. (2011). Prediction of compressive strength of self-compacting concrete containing bottom ash using artificial neural networks. Advances in Engineering Software, 42, 780–786. https://doi.org/10.1016/j.advengsoft.2011.05.016 Smola, A. J., & Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14, 199–222. https://doi.org/10.1023/B:STCO.0000035301.49549.88 Taud, H., & Mas, J. F. (2018). Multilayer perceptron (MLP). In M. T. Camacho Olmedo, M. Paegelow, J.-F. Mas, & F. Escobar (Eds.), Geomatic approaches for modeling land change scenarios, lecture notes in geoinformation and cartography (pp. 451–455). Springer International Publishing. https://doi.org/10.1007/978-3-319-60801-3_27 Thornton, C., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2013). Auto-WEKA: Combined selection and hyperparameter optimization of classification algorithms. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’13. Association for Computing Machinery, New York, NY, USA, pp. 847–855. https://doi.org/10.1145/2487575.2487629 Tipu, R. K., Panchal, V. R., & Pandya, K. S. (2023). Multi-objective optimized high-strength concrete mix design using a hybrid machine learning and metaheuristic algorithm. Asian Journal of Civil Engineering, 24, 849–867. https://doi.org/10.1007/s42107-022-00535-8 Vapnik, V. N. (1995). The nature of statistical learning. Theory. Vapnik, V., Golowich, S., & Smola, A. (1996). Support vector method for function approximation, regression estimation and signal processing. Advances in Neural Information Processing Systems, 9. Wong, T.-T., & Yeh, P.-Y. (2020). Reliable accuracy estimates from k-fold cross validation. IEEE Transactions on Knowledge and Data Engineering, 32, 1586–1594. https://doi.org/10.1109/TKDE.2019.2912815 Yang, L., & Shami, A. (2020). On hyperparameter optimization of machine learning algorithms: Theory and practice. Neurocomputing, 415, 295–316. https://doi.org/10.1016/j.neucom.2020.07.061 Yeh, I.-C. (2007). UCI machine learning repository: Concrete compressive strength data set. Yeh, I.-C., & Lien, L.-C. (2009). Knowledge discovery of concrete material using Genetic Operation Trees. Expert Systems with Applications, 36, 5807–5812. https://doi.org/10.1016/j.eswa.2008.07.004 Yokoyama, S., & Matsumoto, T. (2017). Development of an automatic detector of cracks in concrete using machine learning. In: Procedia Engineering, The 3rd International Conference on Sustainable Civil Engineering Structures and Construction Materials—Sustainable Structures for Future Generations, vol. 171, pp. 1250–1255. https://doi.org/10.1016/j.proeng.2017.01.418 Yu, X., Efe, M. O., & Kaynak, O. (2002). A general backpropagation algorithm for feedforward neural networks learning. IEEE Transactions on Neural Networks, 13, 251–254. https://doi.org/10.1109/72.977323 Zhang, J., Sun, Y., Li, G., Wang, Y., Sun, J., & Li, J. (2022). Machine-learning-assisted shear strength prediction of reinforced concrete beams with and without stirrups. Engineering with Computers, 38, 1293–1307. https://doi.org/10.1007/s00366-020-01076-x Zhao, S., Hu, F., Ding, X., Zhao, M., Li, C., & Pei, S. (2017). Dataset of tensile strength development of concrete with manufactured sand. Data in Brief, 11, 469–472. https://doi.org/10.1016/j.dib.2017.02.043 Zheng, D. X. M., Ng, S. T., & Kumaraswamy, M. M. (2004). Applying a genetic algorithm-based multiobjective approach for time-cost optimization. Journal of Construction Engineering and Management, 130, 168–176. https://doi.org/10.1061/(ASCE)0733-9364(2004)130:2(168)