2013-09-26 14:31:11 +00:00
|
|
|
chronosdb
|
|
|
|
|
=========
|
|
|
|
|
|
|
|
|
|
Scalable datastore for metrics, events, and real-time analytics
|
2013-09-27 16:37:19 +00:00
|
|
|
|
|
|
|
|
Requirements
|
|
|
|
|
------------
|
|
|
|
|
|
|
|
|
|
* horizontal scalable
|
|
|
|
|
* http interface
|
|
|
|
|
* udp interface (low priority)
|
|
|
|
|
* persistent
|
|
|
|
|
* metadata for time series
|
|
|
|
|
* perform functions quickly (count, unique, sum, etc.)
|
|
|
|
|
* group by time intervals (e.g. count ticks every 5 minutes)
|
|
|
|
|
* joining multiple time series to generate new timeseries
|
|
|
|
|
* dynamic schema
|
|
|
|
|
* filter/query language (sql subset) with where clauses
|
|
|
|
|
* support multiple databases with read/write api key
|
|
|
|
|
* single time series should scale horizontally (no hot spots)
|
|
|
|
|
* dynamic cluster changes and data balancing
|
|
|
|
|
* pubsub layer
|
|
|
|
|
* continuous queries (keep connection open and return new points as they arrive)
|
|
|
|
|
* Delete ranges of points from any number of timeseries (that should reflect in disk space usage)
|
|
|
|
|
* querying should support one or more timeseries (possibly with regex to match on)
|
2013-09-30 15:03:05 +00:00
|
|
|
|
2013-09-30 19:10:53 +00:00
|
|
|
New Requirements
|
|
|
|
|
----------------
|
|
|
|
|
* Easy to backup and restore
|
|
|
|
|
* Large time range queries with one column ?
|
|
|
|
|
* Optimize for HDD access ?
|
|
|
|
|
* What are the common use cases that we should optimize for ?
|
|
|
|
|
|
2013-09-30 15:03:05 +00:00
|
|
|
Modules
|
|
|
|
|
-------
|
|
|
|
|
|
|
|
|
|
|
2013-09-30 19:17:50 +00:00
|
|
|
+--------------------+ +--------------------+
|
|
|
|
|
| | | |
|
|
|
|
|
| WebConsole/docs | | Http API |
|
|
|
|
|
| | | |
|
|
|
|
|
+------------------+-+ +-+------------------+
|
|
|
|
|
| |
|
|
|
|
|
| |
|
|
|
|
|
+-----+-------+-----------+
|
|
|
|
|
| |
|
|
|
|
|
| Lang. Bindings |
|
|
|
|
|
| |
|
|
|
|
|
+-----------------+ |
|
|
|
|
|
| | |
|
|
|
|
|
| Query Engine | |
|
|
|
|
|
| | |
|
|
|
|
|
+-----------------+-------+
|
|
|
|
|
| |
|
|
|
|
|
| Processing Engine |
|
|
|
|
|
| |
|
|
|
|
|
+-------------------------+
|
|
|
|
|
| |
|
|
|
|
|
+----+ Coordinator (consensus) +-----+
|
|
|
|
|
| | | |
|
|
|
|
|
| +-------------------------+ |
|
2013-09-30 15:03:05 +00:00
|
|
|
| |
|
|
|
|
|
| |
|
|
|
|
|
+--------+-----------+ +-------+------------+
|
|
|
|
|
| | | |
|
|
|
|
|
| Storage Engine | | Storage Engine |
|
|
|
|
|
| | | |
|
|
|
|
|
+--------+-----------+ +-------+------------+
|
2013-10-03 22:07:59 +00:00
|
|
|
|
|
|
|
|
Concensus Notes
|
|
|
|
|
---------------
|
|
|
|
|
|
|
|
|
|
Two state machines:
|
|
|
|
|
* 1 for the entire cluster of which machines are taking which portions of the ring
|
|
|
|
|
* 1 for each portion of the ring to replicate the operations
|
|
|
|
|
sequence number per ring location? that's the concensus, if they don't agree then request a replay from the last known sequence number
|
