XDataExplorer: A Three-Stage Comprehensive Self-Tuning Tool for Big Data Platforms

Big Data Research - Tập 29 - Trang 100329 - 2022
Qing Guo1,2, Yingying Xie2, Qiushi Li2, Yifan Zhu1
1School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China
2Bigdata department, Dawning Information Industry Co., Ltd., Beijing, 100193, China

Tài liệu tham khảo

Jagadish, 2014, Big data and its technical challenges, Commun. ACM, 57, 86, 10.1145/2611567 Cuzzocrea, 2014, Privacy and security of big data: current challenges and future research perspectives Jin, 2015, Significance and challenges of big data research, Big Data Res., 2, 59, 10.1016/j.bdr.2015.01.006 Cheng, 2014, Survey on big data system and analytic technology, J. Softw., 25, 1240 Wasi-ur- Rahman, 2018, MR-advisor: a comprehensive tuning, profiling, and prediction tool for MapReduce execution frameworks on HPC clusters, J. Parallel Distrib. Comput., 120, 237, 10.1016/j.jpdc.2017.11.004 Yeh, 2020, Big data platform configuration using machine learning, J. Inf. Sci. Eng., 36, 469 Gounaris, 2018, A methodology for spark parameter tuning, Big Data Res., 11, 22, 10.1016/j.bdr.2017.05.001 Xu, 2017, ECL-watch: a big data application performance tuning tool in the HPCC systems platform, 2941 White, 2014 Sammer, 2012 Qubole Karlupia Pattanshetti, 2020, Parameter tuning of big data platforms for performance optimization, J. Inf. Optim. Sci., 41, 403 Cai, 2019, mrMoulder: a recommendation-based adaptive parameter tuning approach for big data processing platform, Future Gener. Comput. Syst., 93, 570, 10.1016/j.future.2018.05.080 Cheng, 2017, Improving performance of heterogeneous MapReduce clusters with adaptive task tuning, IEEE Trans. Parallel Distrib. Syst., 28, 774, 10.1109/TPDS.2016.2594765 Hossain, 2017, A belief rule based expert system for datacenter PUE prediction under uncertainty, IEEE Trans. Sustain. Comput., 2, 140, 10.1109/TSUSC.2017.2697768 Joshi, 2012, Apache hadoop performance-tuning methodologies and best practices, 241 Yi, 2020, A barrier optimization framework for NUMA multi-core system, Concurr. Comput., 32, 10.1002/cpe.5527 Walek, 2020, A hybrid recommender system for recommending relevant movies using an expert system, Expert Syst. Appl., 158, 10.1016/j.eswa.2020.113452 Chen, 2020, Developing two heuristic algorithms with metaheuristic algorithms to improve solutions of optimization problems with soft and hard constraints: an application to nurse rostering problems, Appl. Soft Comput., 93, 10.1016/j.asoc.2020.106336 Ding, 2015, JellyFish: online performance tuning with adaptive configuration and elastic container in hadoop yarn, 831 Ye, 2018, A Kalman Filter Based Hill-Climbing Strategy for Application Server Configuration, 1524 Pirzadeh, 2017, A performance study of big data analytics platforms, 2911 Samadi, 2018, Performance comparison between Hadoop and Spark frameworks using HiBench benchmarks, Concurr. Comput., 30, 10.1002/cpe.4367 Nambiar, 2014, Introducing TPCx-HS: the first industry standard for benchmarking big data systems, 1