075ba0d8d1
refactor: remove delorean_table_schema crate and fold it into data_types ( #408 )
2020-11-05 06:17:20 -05:00
7d25dc8487
fix: Remove unused arrow dependency in delorean_ingest
...
This wasn't really causing any problems, just confusion, because the old
arrow and its deps were in the Cargo.lock.
2020-11-04 15:34:34 -05:00
bf0c58698e
refactor: rename delorean_data_types crate to data_type ( #407 )
...
* refactor: rename delorean_data_types crate to data_type - #401
* fix: fmt
2020-11-04 12:33:41 -05:00
9f36914351
chore: Upgrade version of Arrow / DataFusion (3 of 3) + update code for new interfaces ( #395 )
2020-11-02 11:20:44 -05:00
1e966b5153
feat: implement API for storing the server configuration in object storage
...
This adds basic API calls for persisting and loading the server configuratioon of database rules and host groups to and from object storage. It stores all the data in a single JSON file.
2020-10-26 13:43:43 -06:00
ef501871bb
feat: remove partition_store ( #387 )
2020-10-26 14:39:38 -04:00
4e1e8dbf79
chore: Upgrade version of Arrow/DataFusion (2 of 3) ( #391 )
...
* chore: Upgrade version of Arrow/DataFusion (2 of 3)
* fix: Fixup error type usage and use async stream interface
* fix: post merge fixups
2020-10-26 13:49:16 -04:00
88b9f43110
chore: Upgrade version of Arrow/DataFusion (1 of 3) ( #390 )
...
* chore: Upgrade version of Arrow/DataFusion
* fix: update code for deps
2020-10-26 11:46:02 -04:00
1004854403
refactor: remove uneeded dependencies, switch to tracing from log ( #388 )
2020-10-26 06:15:47 -04:00
0ef76db208
feat: implement series_query for write buffer database, tests for same ( #360 )
...
* feat: implement series_query for write buffer database, tests for same
* fix: fixup comments
* fix: sort field columns too
2020-10-15 17:23:14 -04:00
262a988207
Merge pull request #357 from influxdata/pd-cluster_replicate
...
chore: refactor cluster to use in memory write buffer
2020-10-14 09:43:02 -04:00
9a345e226c
chore: refactor cluster to use in memory write buffer
...
This refactors cluster to use the in memory write buffer. It removes the injected DatabaseStore as it is no longer needed.
2020-10-14 08:36:49 -04:00
6091963d50
test: skip NaN test for now
2020-10-14 13:21:15 +01:00
74ed1904c9
feat: fixed encoding for non-null numerics
2020-10-14 13:18:42 +01:00
206df6a325
feat: implement data fusion execution and conversion to series sets ( #353 )
2020-10-13 16:53:00 -04:00
a80eb0fed3
feat: Store replicated writes
...
This commit refactors the flatbuffers data types from the wal to a new crate where they can be used by storage, write buffer, and cluster. It also refactors cluster to move the configuration types out to the data types crate so they can be used across storage and elsewhere.
Finally, it adds a new method to store replicated writes on a database in the database trait and implements it.
2020-10-11 15:45:08 -04:00
996f8905b6
feat: Implement partition templates and key generation
...
This commit implements partition templates as a struct that can be serialized and deserialzed. It is comprised of parts that can include the table name, a column name and its value, a formatted time, or a string column and regex captures of its value.
2020-10-10 11:32:17 -04:00
cceeebb317
Merge pull request #342 from influxdata/pd-cluster-updates
...
feat: Update cluster with replication and subscriptions
2020-10-09 07:41:32 -04:00
2b8c04f2b4
chore: Update arrow (again) to pick up latest changes to datafusion ( #345 )
2020-10-09 07:17:02 -04:00
a72e608810
feat: enable simd in arrow ( #343 )
2020-10-08 11:21:22 -04:00
05dcbd7236
feat: Update cluster with replication and subscriptions
...
This updates cluster so that the concept of replication and subscriptions for handling queries are separated. It also adds flatbuffer structure that can be used as a common format for replication.
2020-10-08 08:40:13 -04:00
bc5378c7fe
chore: Update arrow to latest version ( #335 )
...
* chore: Update arrow to latest version
* fix: Updates needed by new version of datafusion
2020-10-02 14:46:07 -04:00
ff29610e44
refactor: Switch back to https://github.com/apache/arrow ( #333 )
2020-10-01 16:57:12 -04:00
2b98da593b
feat: write_database support for predicates ( #326 )
...
* feat: write_database support for predicates
* fix: temporarily pull in arrow fork to pick up fix for ARROW-10136
* fix: Update mutex usage based on PR feedback
* fix: more mutex polish and use OptionExt
* fix: update comments
* fix: rust-fu the table lookup
* fix: update docs
* fix: more idomatic rust types
* fix: better usage of reference types
2020-10-01 14:34:53 -04:00
a2287acb7c
Merge pull request #330 from influxdata/er/feat/segment-store-shell
...
feat: Segment Store shell
2020-10-01 14:01:45 +01:00
bd6b0db691
refactor: address PR feedback
2020-10-01 13:13:32 +01:00
fdc86fd186
feat: add some initial framework for clustering ( #329 )
2020-09-30 14:41:42 -04:00
8a14896487
chore: update version of datafusion ( #324 )
...
* chore: update version of datafusion
* chore: Update interfaces to be async
2020-09-30 08:02:15 -04:00
2470bdb975
feat: segment store shell
2020-09-30 11:25:59 +01:00
da5c74d3c6
feat: storage interface plans + executor ( #318 )
...
* feat: storage interface plans + executor
* refactor: less `expect`
* fix: use more idomatic rust From
2020-09-28 11:41:10 -04:00
0236522dfa
feat: Send panic information to tracing events ( #313 )
...
* feat: Send panic information to tracing events
* fix: PR Review improvements
* fix: PR comments
* fix: Apply suggestions from code review
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
* fix: more fixes
* fix: clarify /cleanup drop more
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2020-09-25 14:55:58 -04:00
ec1aaa3a47
chore: update dependencies
2020-09-25 17:22:48 +01:00
9eee0c2852
refactor: make clippy happy
2020-09-25 10:12:46 +01:00
c42d2dcd79
refactor: rebase with delorean_arrow
2020-09-25 10:12:46 +01:00
d0f3cae9b3
feat: add tag values schema API
2020-09-25 10:12:46 +01:00
47b2f7940b
refactor: spike on arrow encoding
2020-09-25 10:12:46 +01:00
e5f9c7c574
refactor: add encoding trait
2020-09-25 10:12:46 +01:00
54e9d38589
chore: update the refs to github
2020-09-25 10:12:46 +01:00
41899203d9
refactor: implement a prototype datafusion integration layer demonstration
2020-09-25 10:12:46 +01:00
820277a529
feat: load segments from parquet
2020-09-25 10:12:46 +01:00
acfef35a0e
feat: load segments from parquet
2020-09-25 10:12:46 +01:00
7f815099d0
feat: Read from parquet rather than arrow
2020-09-25 10:12:46 +01:00
a5a8667a42
feat: group by sorting
2020-09-25 10:12:46 +01:00
231f429a56
feat: sort group by measurement
2020-09-25 10:12:46 +01:00
2387b7c849
feat: add support for group by aggregate
2020-09-25 10:12:46 +01:00
aba02cb731
feat: basic store
2020-09-25 10:12:46 +01:00
77f58efca7
chore: update Arrow/Parquet/DataFusion versions, consolidate references into new crate ( #309 )
...
* chore: consolidate all arrow/parquet/datafusion dependencies
* chore: update datafusion version
2020-09-24 08:46:54 -04:00
498478c066
refactor: rename delorean_storage_interface to delorean_storage ( #308 )
2020-09-22 17:18:53 -04:00
d0f2902c8d
feat: implement tag_keys and measurement_tag_keys ( #307 )
...
* feat: implement tag_keys and measurement_tag_keys
* fix: fix timestamp bound evaluation
2020-09-22 16:42:45 -04:00
648d42568d
feat: Add a benchmark for restoring the WAL
2020-09-18 16:45:01 -04:00
Andrew Lamb
Carol (Nichols || Goulding)
Paul Dix
Paul Dix
Edd Robinson
alamb
Jake Goulding