Communication optimization for RDMA-based science data transmission tools
Tóm tắt
Big data has raised new challenges to data communication and transmission capacity. RDMA (Remote Direct Memory Access) Transport Protocol can reduce the communication delay of big data through kernel memory bypass technology and zero-copy technology. This paper introduced RDMA technology for the network performance optimization of BBCP (Babar Copy Program), a big data network file copy tool. Also, transmission queue-oriented Tetris Scheduler algorithm, I/O performance optimization and buffer memory optimization technique were introduced to realize RDMA-based file transfer tool, OBCP (Optimized Babar Copy Program). The experiment results verified that the optimization method could decrease the file transmission time to one-tenth of original BBCP and improve the performance by 50 % compared with related works.
Tài liệu tham khảo
Martin H, López P (2011) The world’s technological capacity to store, communicate, and compute information. Science 332(6025):60–65
Li T, Binghang Y, Luan Z (2010) Fast large file distribution in data grid. In: Intelligent Computing and Integrated Systems (ICISS), 2010 International Conference on. IEEE (2010)
http://www.rdmaconsortium.org/
Rashti Mohammad J, Afsahi A (2009) A speculative and adaptive MPI Rendezvous protocol over RDMA-enabled interconnects. Int. J. Parallel Progr. 37(2):223–246
Pinkerton J (2002) The case for RDMA. RDMA Consortium, p 27 May 29
Shivam P, Wyckoff P, Panda D (2001) EMP: zero-copy OS-bypass NIC-driven gigabit ethernet message passing. In: Supercomputing, ACM/IEEE 2001 Conference. IEEE
Hanushevsky A, Artem T, Les C (2001) Peer-to-peer computing for secure high performance data copying. In: Proc. of the 2001 Int. Conf. on Computing in High Energy and Nuclear Physics (CHEP 2001), Beijing
Adam Smyk, Tudruj Marek (2006) RDMA control support for fine-grain parallel computations. J Syst Archit 52(2):117–128
Wang B, Chen L, Liu Y (2013) The research of performance optimization technology in the high-speed network. In: Software Engineering and Service Science (ICSESS), 2013 4th IEEE International Conference on. IEEE
IBCP. https://github.com/UniqueStudio/ibcp
Taub. https://campuscluster.illinois.edu/hardware/#taub
Weizhe Z, Zhang Y, Kim T-H (2014) Detecting bad information in mobile wireless networks based on the wireless application protocol. Computing 96(9):855–874
Zhang W, Wang X, Lu B, Kim TH (2013) Secure encapsulation and publication of biological services in the cloud computing environment. BioMed Res Int 2013
Zhang W, He H, Zhang Q, Kim TH (2014) PhoneProtector: protecting user privacy on the android-based mobile platform. Int J Distrib Sens Netw 2014