RailwayDB: adaptive storage of interaction graphs
Tóm tắt
We are living in an ever more connected world, where data recording the interactions between people, software systems, and the physical world is becoming increasingly prevalent. These data often take the form of a temporally evolving graph, where entities are the vertices and the interactions between them are the edges. We call such graphs interaction graphs. Various domains, including telecommunications, transportation, and social media, depend on analytics performed on interaction graphs. The ability to efficiently support historical analysis over interaction graphs requires effective solutions for the problem of data layout on disk. This paper presents an adaptive disk layout called the railway layout for optimizing disk block storage for interaction graphs. The key idea is to divide blocks into one or more sub-blocks. Each sub-block contains the entire graph structure, but only a subset of the attributes. This improves query I/O, at the cost of increased storage overhead. We introduce optimal integer linear program (ILP) formulations for partitioning disk blocks into sub-blocks with overlapping and nonoverlapping attributes. Additionally, we present greedy heuristics that can scale better compared to the ILP alternatives, yet achieve close to optimal query I/O. We provide an implementation of the railway layout as part of RailwayDB—an open-source graph database we have developed. To demonstrate the benefits of the railway layout, we provide an extensive experimental evaluation, including model-based as well as empirical results comparing our approach to baseline alternatives.
Tài liệu tham khảo
Hoque, I., Gupta, I.: Disk layout techniques for online social network data. Internet Comput. 16(3), 24–36 (2012)
Steinhaus, R.: G-Store: a storage manager for graph data. Master’s thesis, University of Oxford (2011)
Gedik, B., Bordawekar, R.: Disk-based management of interaction graphs. IEEE Trans. Knowl. Data. Eng. (TKDE) 26(11), 650–665 (2014)
Alagiannis, I., Idreos, S., Ailamaki, A.: H2O: a hands-free adaptive store. In: ACM International Conference on Management of Data (SIGMOD), pp. 1103–1114 (2014)
Grund, M., Zeier, A., Krüger, J., Plattner, H., Madden, S.: HYRISE: a main memory hybrid storage engine. VLDB J. (VLDBJ) 4(2), 105–116 (2010)
Stonebraker, M., Abadi, D.J., Batkin, A., Chen, X., Cherniack, M., Ferreira, M., Lau, E., Lin, A., Madden, S., O’Neil, E., O’Neil, P., Rasin, A., Tran, N., Zdonik, S.: C-store: a column-oriented DBMS. In: International Conference on Very Large Data Bases (VLDB), pp. 553–564 (2005)
Hellerstein, J.M., Stonebraker, M., Hamilton, J.: Architecture of a database system. Found. Trends Databases 1(2), 141–259 (2007). doi:10.1561/1900000002
Abadi, D., Boncz, P., Harizopoulos, S., Idreos, S., Madden, S.: The design and implementation of modern column-oriented database systems. Found. Trends Databases 5(3), 197–280 (2013)
Wilson, C., Boe, B., Sala, A., Puttaswamy, K.P., Zhao, B.Y.: User interactions in social networks and their implications. In: European Conference on Computer Systems (EUROSYS), pp. 205–218 (2009)
Wilson, C., Sala, A., Puttaswamy, K.P.N., Zhao, B.Y.: Beyond social graphs: user interactions in online social networks and their implications. ACM Trans. Web (TWEB) 6(4), 17:1–17:31 (2012)
Ghemawat, S.: LevelDB - a fast and lightweight key/value database library by Google. Available at https://github.com/google/leveldb, retrieved March, 2015
Hadjieleftheriou, M., Hoel, E.G., Tsotras, V.J.: SaIL: a spatial index library for efficient application integration. GeoInformatica 9(4), 367–389 (2005)
Gurobi Optimization Inc.: The Gurobi Optimizer. Available at http://www.gurobi.com
Turkey Protests Spread from Istanbul to Ankara, Euronews. http://www.euronews.com/2013/05/31/turkey-protests-spread-from-istanbul-to-ankara/
Malewicz, G., Austern, M.H., Bik, A.J., Dehnert, J.C., Horn, I., Leiser, N., Czajkowski, G.: Pregel: a system for large-scale graph processing. In: ACM International Conference on Management of Data (SIGMOD), pp. 135–146 (2010)
Apache giraph. http://giraph.apache.org
Gonzalez, J.E., Low, Y., Gu, H., Bickson, D., Guestrin, C.: PowerGraph: distributed graph-parallel computation on natural graphs. In: USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 17–30 (2012)
Kyrola, A., Blelloch, G., Guestrin, C.: GraphChi: large-scale graph computation on just a PC. In: USENIX Symposium on Operating Systems Design and Implementation (OSDI), pp. 31–46 (2012)
Kang, U., Tsourakakis, C.E., Faloutsos, C.: PEGASUS: A peta-scale graph mining system implementation and observations. In: IEEE International Conference on Data Mining (ICDM), pp. 229–238 (2009)
Idreos, S., Kersten, M.L., Manegold, S.: Self-organizing tuple reconstruction in column-stores. In: ACM International Conference on Management of Data (SIGMOD), pp. 297–308 (2009)
Idreos, S., Kersten, M.L., Manegold, S.: Database cracking. In: Conference on Innovative Data Systems Research (CIDR), pp. 68–78 (2007)
Graefe, G., Kuno, H.: Self-selecting, self-tuning, incrementally optimized indexes. In: International Conference on Extending Database Technology (EDBT), pp. 371–381 (2010)
Graefe, G., Kuno, H.A.: Adaptive indexing for relational keys. In: Workshop on Self-Managing Database Systems (SMDB), pp. 69–74 (2010)
Idreos, S., Manegold, S., Kuno, H., Graefe, G.: Merging what’s cracked, cracking what’s merged: adaptive indexing in main-memory column-stores. In: International Conference on Very Large Data Bases (VLDB), pp. 586–597 (2014)
Wu, E., Madden, S.: Partitioning techniques for fine-grained indexing. In: IEEE International Conference on Data Engineering (ICDE), pp. 1127–1138 (2011)
Neo4j Graph Database. http://www.neo4j.org
Kang, U., Tong, H., Sun, J., Lin, C.Y., Faloutsos, C.: GBASE: A scalable and general graph management system. In: ACM International Conference on Knowledge Discovery and Data Mining (SIGKDD), pp. 1091–1099 (2011)
Khurana, U., Deshpande, A.: Efficient snapshot retrieval over historical graph data. In: IEEE International Conference on Data Engineering (ICDE), pp. 997–1008 (2013)
Bornea, M.A., Dolby, J., Kementsietsidis, A., Srinivas, K., Dantressangle, P., Udrea, O., Bhattacharjee, B.: Building an efficient rdf store over a relational database. In: ACM International Conference on Management of Data (SIGMOD), pp. 121–132 (2013)