Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
6,778 Commits
228 Branches
453 Tags
343 MiB
Commit Graph

6778 Commits (e6ec8ef5f345a9e3b91a414da154e256b6d35fe3)

Author SHA1 Message Date
Andrew Lamb 55daf95b48
refactor: Consolidate more query_test DbSetups (#3607) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 15:16:56 +00:00
Andrew Lamb 5d5310351b
refactor: Move query_test readme to standard location (#3608) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 15:06:13 +00:00
Dom Dwyer 4744c5804e refactor: remove Dashmap 
Swap Dashmap for a regular RwLock<HashMap<..,>> due to soundness issues:

    https://rustsec.org/advisories/RUSTSEC-2022-0002
 2022-02-02 14:04:53 +00:00
Dom Dwyer 39d489d9e7 refactor: enable schema validation 
Adds the SchemaValidator to the DML handler stack - this adds it into
the request path in router2.
 2022-02-02 14:04:14 +00:00
Dom Dwyer 6598023726 feat: cache NamespaceSchema in validator 
Adds an in-memory cache of table schemas to the SchemaValidator DML
handler.

The cache pulls from the global catalog when observing a column for the
first time, and pushes the column type to set it for subsequent requests
if it does not exist (this pull & push is done by atomically by the
catalog in an "upsert" call).

The in-memory cache is sharded by namespace, with each shard guarded by
an individual lock to minimise contention between readers (the expected
average case) and writers (only when adding new columns/tables).

Relies on the catalog to serialise new column creation and validate
parallel creation requests.
 2022-02-02 13:04:53 +00:00
Dom Dwyer c81f207298 feat: schema validation 
Implements a write schema validation DML handler, denying requests that
conflict with the schema within the global catalog. Additive schema
changes are accepted, incrementally updating the global catalog schema.

Deletes are passed through unchanged and unvalidated.
 2022-02-02 13:04:53 +00:00
Dom Dwyer f9f9beac36 refactor: get_schema_by_name no Option 
Fetching a NamespaceSchema always succeeds and never returns None.
 2022-02-02 13:04:53 +00:00
Marko Mikulicic 7b2b5b7ebd
chore: Use ubuntu mirror in test_perf (#3609) 2022-02-02 12:37:44 +00:00
kodiakhq[bot] dc1718421a
Merge pull request #3606 from influxdata/er/feat/rpc_cli 
feat: add skeleton `storage` CLI command
 2022-02-02 12:06:47 +00:00
Edd Robinson 5441682207 feat: add support for parsing predicate 2022-02-02 11:02:33 +00:00
Edd Robinson 08901c13cd feat: support parsing timerange 2022-02-02 11:02:33 +00:00
Edd Robinson a424d1c912 feat: shell command read_filter 2022-02-02 11:02:33 +00:00
Edd Robinson c1f5994660
refactor: move sql parsing -> RPC predicate into own crate (#3604) 
* chore: create crate

* refactor: move module to new crate
 2022-02-02 10:41:57 +00:00
Marco Neumann 9567acd621
feat: expose all relevant configs for rskafka write buffers (#3599) 
* feat: expose all relevant configs for rskafka write buffers

* refactor: `CreationConfig` => `TopicCreationConfig`
 2022-02-02 09:35:54 +00:00
kodiakhq[bot] 6b3851242e
Merge pull request #3602 from influxdata/alamb/null_not_null 
fix: handle IS NULL / IS NOT NULL during rpc rewrite
 2022-02-01 22:52:04 +00:00
Andrew Lamb b03c900174 fix: handle IS NULL / IS NOT NULL during rpc rewrite 2022-02-01 13:48:58 -05:00
kodiakhq[bot] 323d873a37
Merge pull request #3600 from influxdata/alamb/case_fold 
feat: Fold past CASE blocks / translate to boolean CASE
 2022-02-01 18:11:55 +00:00
Andrew Lamb 0570662da2 feat: Fold past CASE blocks / translate to boolean CASE 2022-02-01 12:24:33 -05:00
kodiakhq[bot] 46d43b865f
Merge pull request #3597 from influxdata/er/refactor/read_buffer_clippy 
refactor: introduce to_string clippy lints in read buffer
 2022-02-01 16:36:03 +00:00
Edd Robinson 37ad778603
Merge branch 'main' into er/refactor/read_buffer_clippy 2022-02-01 16:25:12 +00:00
Marco Neumann 36a7d9b8f3
feat: flush interface for write buffer producers (#3595) 
Closes #3504.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-01 15:16:23 +00:00
Edd Robinson 04a85515d3
Merge branch 'main' into er/refactor/read_buffer_clippy 2022-02-01 15:11:00 +00:00
Edd Robinson 04a59f825f refactor: fix clippy lints 2022-02-01 14:56:49 +00:00
Edd Robinson fa546047fb
refactor: update dep with API change (#3596) 2022-02-01 14:53:10 +00:00
Edd Robinson 2f3cc046f6 chore: enable to_string restrictive lints 2022-02-01 14:47:30 +00:00
kodiakhq[bot] 42fddf5fc4
Merge pull request #3594 from influxdata/er/feat/parse_sql_rpc 
feat: add support for parsing sql expressions into RPCNode
 2022-02-01 14:05:12 +00:00
Edd Robinson 443fd00c1b
Merge branch 'main' into er/feat/parse_sql_rpc 2022-02-01 13:55:08 +00:00
Edd Robinson 1f0f6dad45 refactor: address PR feedback 2022-02-01 13:54:43 +00:00
Marco Neumann 59a2c74352
refactor: reusable ingester/router2 CLI pieces (#3590) 
* refactor: use a single CLI parser for ingester/router2 WB

* refactor: reusable catalog DSN CLI handling

We are going to need DSN handling for the router as well as for the some
admin tools.

* fix: DNS -> DSN
 2022-02-01 12:57:58 +00:00
Edd Robinson 506363dd1c refactor: expose API to generate RPC predicate 2022-02-01 12:47:37 +00:00
Marco Neumann 0a2cb36ddf
fix: workspace-hack (#3593) 2022-02-01 12:32:58 +00:00
Edd Robinson 0b9315bfaa test: check _measurement and _field 2022-02-01 12:21:35 +00:00
Edd Robinson 4db2865558 feat: add support for precedence 2022-02-01 12:14:20 +00:00
Marco Neumann 22778a3a80
chore: upgrade rskafka and parking_lot (#3592) 2022-02-01 11:50:42 +00:00
Edd Robinson 2f4c559a5a feat: add support for ::field and ::tag 2022-02-01 11:06:51 +00:00
Edd Robinson 175732c3ca feat: basic sqlparser -> RPCNode 2022-02-01 10:26:03 +00:00
Marco Neumann b326b62b44
feat: buffer writes when writing to RSKafka (#3520) 2022-02-01 10:07:52 +00:00
kodiakhq[bot] bf6691e56f
Merge pull request #3544 from influxdata/cn/persist 
feat: Ingester persistence
 2022-01-31 19:01:21 +00:00
kodiakhq[bot] 8bef2c105c
Merge branch 'main' into cn/persist 2022-01-31 18:50:45 +00:00
Andrew Lamb 7b96a37165
chore: Update datafusion (#3586) 
* chore: update DataFusion to f849968057ddddccc9aa19915ef3ea56bf14d80d

* fix: reduce overhead of creating physical expressions

* chore: use MemTrackingMetrics

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-31 18:15:28 +00:00
Carol (Nichols || Goulding) 4006dc14b3
fix: Correct typo in function name 2022-01-31 10:48:30 -05:00
Carol (Nichols || Goulding) 749989a937
refactor: Simplify type, eliminating empty vec creation 
If there aren't any record batches, there isn't any metadata, and vice
versa. Make this relationship clearer by putting the Option around both
the vec of recordbatches and the metadata.
 2022-01-31 10:48:30 -05:00
Carol (Nichols || Goulding) 093d5acfd4
fix: Unify temporary multiple definitions of IoxMetadata 2022-01-31 10:48:29 -05:00
Carol (Nichols || Goulding) 8f81ce5501
refactor: Share parquet_file::storage code between new and old metadata 2022-01-31 10:36:33 -05:00
Carol (Nichols || Goulding) bf89162fa5
refactor: Move IoxMetadata to parquet_file 2022-01-31 10:36:33 -05:00
Carol (Nichols || Goulding) 0f72a881ef
refactor: Rename Rust struct parquet_file::IoxMetadata to be IoxMetadataOld 2022-01-31 10:36:33 -05:00
Carol (Nichols || Goulding) 1b298bb5bd
refactor: Alias the old proto definitions to make clearer the new ones coming in 2022-01-31 10:36:33 -05:00
Carol (Nichols || Goulding) dd9620da0c
feat: Create a new proto definition for the new design's IoxMetadata 2022-01-31 10:36:32 -05:00
Carol (Nichols || Goulding) 81647f253c
feat: Use IoxMetadata and a list of RecordBatches 2022-01-31 10:36:32 -05:00
Carol (Nichols || Goulding) fef968f75c
fix: Remove catalog insertion; will be handled elsewhere 2022-01-31 10:36:32 -05:00