ChronicleDB

ACM Transactions on Database Systems - Tập 44 Số 4 - Trang 1-45 - 2019
Marc Seidemann1, Nikolaus Glombiewski1, Michael Körber1, Bernhard Seeger1
1Philipps-University Marburg, Marburg, Germany

Tóm tắt

Reactive security monitoring, self-driving cars, the Internet of Things (IoT), and many other novel applications require systems for both writing events arriving at very high and fluctuating rates to persistent storage as well as supporting analytical ad hoc queries. As standard database systems are not capable of delivering the required write performance, log-based systems, key-value stores, and other write-optimized data stores have emerged recently. However, the drawbacks of these systems are a fair query performance and the lack of suitable instant recovery mechanisms in case of system failures. In this article, we present ChronicleDB, a novel database system with a storage layout tailored for high write performance under fluctuating data rates and powerful indexing capabilities to support a variety of queries. In addition, ChronicleDB offers low-cost fault tolerance and instant recovery within milliseconds. Unlike previous work, ChronicleDB is designed either as a serverless library to be tightly integrated in an application or as a standalone database server. Our results of an experimental evaluation with real and synthetic data reveal that ChronicleDB clearly outperforms competing systems with respect to both write and query performance.

Từ khóa


Tài liệu tham khảo

2011. BerlinMOD. Retrieved from: http://dna.fernuni-hagen.de/secondo/BerlinMOD/BerlinMOD.html. 2011. BerlinMOD. Retrieved from: http://dna.fernuni-hagen.de/secondo/BerlinMOD/BerlinMOD.html.

2014. DEBS Grand Challenge 2014 . Retrieved from http://debs.org/debs-2014-smart-homes/. 2014. DEBS Grand Challenge 2014. Retrieved from http://debs.org/debs-2014-smart-homes/.

2015. KairosDB. Retrieved from https://kairosdb.github.io/. 2015. KairosDB. Retrieved from https://kairosdb.github.io/.

2016. Apache Cassandra. Retrieved from http://cassandra.apache.org/. 2016. Apache Cassandra. Retrieved from http://cassandra.apache.org/.

2016. DEBS Grand Challenge 2013 . Retrieved December 10, 2017 from http://debs.org/debs-2013-grand-challenge-soccer-monitoring/. 2016. DEBS Grand Challenge 2013. Retrieved December 10, 2017 from http://debs.org/debs-2013-grand-challenge-soccer-monitoring/.

2017. Apache Hadoop. Retrieved from: http://hadoop.apache.org/. 2017. Apache Hadoop. Retrieved from: http://hadoop.apache.org/.

2017. Apache HBase. Retrieved from: http://hbase.apache.org/. 2017. Apache HBase. Retrieved from: http://hbase.apache.org/.

2017. InfluxDB. Retrieved from: https://github.com/influxdata/influxdb. 2017. InfluxDB. Retrieved from: https://github.com/influxdata/influxdb.

2017. LZ4 Compression. Retrieved from: https://github.com/lz4/lz4. 2017. LZ4 Compression. Retrieved from: https://github.com/lz4/lz4.

2017. OpenTSDB. Retrieved from: http://opentsdb.net/. 2017. OpenTSDB. Retrieved from: http://opentsdb.net/.

2017. PostgreSQL. Retrieved from: http://www.postgresql.org/. 2017. PostgreSQL. Retrieved from: http://www.postgresql.org/.

2017. SafeCast. Retrieved from: http://blog.safecast.org/data/. 2017. SafeCast. Retrieved from: http://blog.safecast.org/data/.

Anastassia Ailamaki , David J. DeWitt , Mark D. Hill , and Marios Skounakis . 2001 . Weaving relations for cache performance . In Proceedings of the VLDB. 169--180 . Anastassia Ailamaki, David J. DeWitt, Mark D. Hill, and Marios Skounakis. 2001. Weaving relations for cache performance. In Proceedings of the VLDB. 169--180.

Roger S. Barga , Jonathan Goldstein , Mohamed H. Ali , and Mingsheng Hong . 2007 . Consistent streaming through time: A vision for event stream processing . In Proceedings of the CIDR. 363--374 . Roger S. Barga, Jonathan Goldstein, Mohamed H. Ali, and Mingsheng Hong. 2007. Consistent streaming through time: A vision for event stream processing. In Proceedings of the CIDR. 363--374.

10.1145/2675743.2771829

10.1145/362686.362692

Zhao Cao , Shimin Chen , Feifei Li , Min Wang , and Xiaoyang Sean Wang . 2013 . LogKV: Exploiting key-value stores for log processing . In Proceedings of the CIDR 2013. Zhao Cao, Shimin Chen, Feifei Li, Min Wang, and Xiaoyang Sean Wang. 2013. LogKV: Exploiting key-value stores for log processing. In Proceedings of the CIDR 2013.

Paris Carbone , Asterios Katsifodimos , Stephan Ewen , Volker Markl , Seif Haridi , and Kostas Tzoumas . 2015 . Apache Flink™: Stream and batch processing in a single engine . IEEE Data Eng. Bull. 38 , 4 (2015), 28 -- 38 . Paris Carbone, Asterios Katsifodimos, Stephan Ewen, Volker Markl, Seif Haridi, and Kostas Tzoumas. 2015. Apache Flink™: Stream and batch processing in a single engine. IEEE Data Eng. Bull. 38, 4 (2015), 28--38.

10.1145/2983323.2983807

10.1145/248603.248616

Alan Demers , Johannes Gehrke , Mingsheng Hong , Biswanath Panda , Mirek Riedewald , Varun Sharma , and Walker White . 2007 . Cayuga: A general purpose event monitoring system . In Proceedings of the CIDR. 412--422 . Alan Demers, Johannes Gehrke, Mingsheng Hong, Biswanath Panda, Mirek Riedewald, Varun Sharma, and Walker White. 2007. Cayuga: A general purpose event monitoring system. In Proceedings of the CIDR. 412--422.

10.1007/978-3-642-28534-9_16

10.1145/1559845.1559934

10.1007/s13222-013-0133-y

Theodore Johnson and Vladislav Shkapenyuk . 2015 . Data stream warehousing in tidalrace . In Proceedings of the CIDR. Theodore Johnson and Vladislav Shkapenyuk. 2015. Data stream warehousing in tidalrace. In Proceedings of the CIDR.

10.1145/1058150.1058158

Jin Li , David Maier , Kristin Tufte , Vassilis Papadimos , and Peter A . Tucker . 2005 . Semantics and evaluation techniques for window aggregates in data streams. In Proceedings of the SIGMOD. 311--322. Jin Li, David Maier, Kristin Tufte, Vassilis Papadimos, and Peter A. Tucker. 2005. Semantics and evaluation techniques for window aggregates in data streams. In Proceedings of the SIGMOD. 311--322.

10.1109/ICDE.2009.95

Charles Loboz , Slawek Smyl , and Suman Nath . 2010 . DataGarage: Warehousing massive performance data on commodity servers . PVLDB 3 , 1 -- 2 (2010), 1447--1458. Charles Loboz, Slawek Smyl, and Suman Nath. 2010. DataGarage: Warehousing massive performance data on commodity servers. PVLDB 3, 1--2 (2010), 1447--1458.

Yuan Mei and Samuel Madden . 2009 . ZStream: A cost-based query processor for adaptively detecting composite events categories and subject descriptors . In Proceedings of the SIGMOD. 193--206 . Yuan Mei and Samuel Madden. 2009. ZStream: A cost-based query processor for adaptively detecting composite events categories and subject descriptors. In Proceedings of the SIGMOD. 193--206.

Guido Moerkotte . 1998 . Small materialized aggregates: A lightweight index structure for data warehousing . In Proceedings of the VLDB. Morgan Kaufmann Publishers Inc. , San Francisco, CA, 476--487. Guido Moerkotte. 1998. Small materialized aggregates: A lightweight index structure for data warehousing. In Proceedings of the VLDB. Morgan Kaufmann Publishers Inc., San Francisco, CA, 476--487.

10.1007/s007780050004

10.1007/s002360050048

10.14778/2824032.2824078

Jun Rao and Kenneth A. Ross . 2000. Making B+- trees cache conscious in main memory . In Proceedings of the SIGMOD. 475--486 . Jun Rao and Kenneth A. Ross. 2000. Making B+- trees cache conscious in main memory. In Proceedings of the SIGMOD. 475--486.

Marc Seidemann and Bernhard Seeger . 2017 . ChronicleDB: A high-performance event store . In Proceedings of the EDBT. 144--155 . Marc Seidemann and Bernhard Seeger. 2017. ChronicleDB: A high-performance event store. In Proceedings of the EDBT. 144--155.

10.14778/2752939.2752940

10.1145/3093742.3093925

10.1109/TKDE.2003.1198390

Fabio Valdés and Ralf Hartmut Güting . 2014. Index-supported pattern matching on symbolic trajectories . In Proceedings of the SIGSPATIAL 2014 . ACM Press , New York , New York, 53--62. Fabio Valdés and Ralf Hartmut Güting. 2014. Index-supported pattern matching on symbolic trajectories. In Proceedings of the SIGSPATIAL 2014. ACM Press, New York, New York, 53--62.

10.14778/2336664.2336673

10.14778/2732286.2732290

10.1145/1142473.1142520

10.1007/s00778-003-0107-z

10.1145/2588555.2593671

Thông tin
Thông tin xuất bản

ACM Transactions on Database Systems

Tập 44 Số 4

1-45

Thông tin tác giả