Communication optimization for RDMA-based science data transmission tools

Springer Science and Business Media LLC - Tập 72 - Trang 3312-3327 - 2015
Weizhe Zhang1, Meng Hao1, Zhiyong Xu1
1School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China

Tóm tắt

Big data has raised new challenges to data communication and transmission capacity. RDMA (Remote Direct Memory Access) Transport Protocol can reduce the communication delay of big data through kernel memory bypass technology and zero-copy technology. This paper introduced RDMA technology for the network performance optimization of BBCP (Babar Copy Program), a big data network file copy tool. Also, transmission queue-oriented Tetris Scheduler algorithm, I/O performance optimization and buffer memory optimization technique were introduced to realize RDMA-based file transfer tool, OBCP (Optimized Babar Copy Program). The experiment results verified that the optimization method could decrease the file transmission time to one-tenth of original BBCP and improve the performance by 50 % compared with related works.

Tài liệu tham khảo

Martin H, López P (2011) The world’s technological capacity to store, communicate, and compute information. Science 332(6025):60–65 Li T, Binghang Y, Luan Z (2010) Fast large file distribution in data grid. In: Intelligent Computing and Integrated Systems (ICISS), 2010 International Conference on. IEEE (2010) http://www.rdmaconsortium.org/ Rashti Mohammad J, Afsahi A (2009) A speculative and adaptive MPI Rendezvous protocol over RDMA-enabled interconnects. Int. J. Parallel Progr. 37(2):223–246 Pinkerton J (2002) The case for RDMA. RDMA Consortium, p 27 May 29 Shivam P, Wyckoff P, Panda D (2001) EMP: zero-copy OS-bypass NIC-driven gigabit ethernet message passing. In: Supercomputing, ACM/IEEE 2001 Conference. IEEE Hanushevsky A, Artem T, Les C (2001) Peer-to-peer computing for secure high performance data copying. In: Proc. of the 2001 Int. Conf. on Computing in High Energy and Nuclear Physics (CHEP 2001), Beijing Adam Smyk, Tudruj Marek (2006) RDMA control support for fine-grain parallel computations. J Syst Archit 52(2):117–128 Wang B, Chen L, Liu Y (2013) The research of performance optimization technology in the high-speed network. In: Software Engineering and Service Science (ICSESS), 2013 4th IEEE International Conference on. IEEE IBCP. https://github.com/UniqueStudio/ibcp Taub. https://campuscluster.illinois.edu/hardware/#taub Weizhe Z, Zhang Y, Kim T-H (2014) Detecting bad information in mobile wireless networks based on the wireless application protocol. Computing 96(9):855–874 Zhang W, Wang X, Lu B, Kim TH (2013) Secure encapsulation and publication of biological services in the cloud computing environment. BioMed Res Int 2013 Zhang W, He H, Zhang Q, Kim TH (2014) PhoneProtector: protecting user privacy on the android-based mobile platform. Int J Distrib Sens Netw 2014