Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
7,145 Commits
188 Branches
453 Tags
378 MiB
Commit Graph

285 Commits (30d1c77d363e5d0b730800aa113f3defbed6e9c6)

Author SHA1 Message Date
Marco Neumann 77f6153f72
refactor: remove `QueryDatabase::chunk_summaries` (#3977) 
- This is not used by the query engine at all.
- The query engine should not care about ALL chunks but only about the
  chunks it gets via `QueryDatabase::chunks` (which includes a table
  name and a predicate).
- All other users of that API are NOT really query-related.
 2022-03-08 11:34:26 +00:00
dependabot[bot] 48908054d1
chore(deps): Bump once_cell from 1.9.0 to 1.10.0 (#3955) 
* chore(deps): Bump once_cell from 1.9.0 to 1.10.0

Bumps [once_cell](https://github.com/matklad/once_cell) from 1.9.0 to 1.10.0.
- [Release notes](https://github.com/matklad/once_cell/releases)
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.9.0...v1.10.0)

---
updated-dependencies:
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
 2022-03-07 08:54:20 +00:00
Andrew Lamb 7b436d37cd
fix: correctly set nulls in string columns (#3939) 
* fix: correctly set nulls in string columns

* fix: test

* test: fix

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-03-04 20:15:44 +00:00
Andrew Lamb 07fdbe7c6b
fix: apply `_field` restrictions that do not match any fields (#3903) 
* fix: apply field restriction correctly

* refactor: Use Vec::retain
 2022-03-03 15:04:05 +00:00
Edd Robinson 3d047073b9
feat: add tracing down to the chunk level (#3804) 
* refactor: wire exectution context to Deduplicator

* feat: example trace to chunk read_filter

* refactor: make execution context required

* refactor: expose metadata API

* refactor: more span context for chunk read_filter

* refactor: fix build

* refactor: push context into result stream

* refactor: make executor optional
 2022-03-02 19:08:22 +00:00
Marco Neumann 33851be3a5
chore: upgrade Rust to 1.59 (#3875) 
Mostly a few new clippy crates around `flat_map`, `and_then`, and
"underscore locks" (!!!):
https://rust-lang.github.io/rust-clippy/master/index.html#let_underscore_lock

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-28 15:14:19 +00:00
Raphael Taylor-Davies 2a842fbb1a
feat: correctly sort data and store in catalog metadata (#3864) 
* feat: respect sort order in ChunkTableProvider (#3214)

feat: persist sort order in catalog (#3845)

refactor: owned SortKey (#3845)

* fix: size tests

* refactor: immutable SortKey

* test: test sort order restart (#3845)

* chore: explicit None for sort key

* chore: test cleanup

* fix: handling of sort keys containing fields

* chore: remove unused selected_sort_key

* chore: more docs

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-25 17:56:27 +00:00
dependabot[bot] 3b7d31c88a
chore(deps): Bump arrow from 9.0.2 to 9.1.0 (#3826) 
Bumps [arrow](https://github.com/apache/arrow-rs) from 9.0.2 to 9.1.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/9.0.2...9.1.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-02-23 09:25:46 +00:00
dependabot[bot] ad3868ed7c
chore(deps): Bump tokio from 1.16.1 to 1.17.0 (#3814) 
* chore(deps): Bump tokio from 1.16.1 to 1.17.0

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.16.1 to 1.17.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.16.1...tokio-1.17.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* build: update workspace-hack

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dom Dwyer <dom@itsallbroken.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-22 16:27:43 +00:00
Raphael Taylor-Davies 69480eaa5c
feat: remove remaining table-granularity metrics (#3808) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-22 15:02:57 +00:00
Andrew Lamb 9588b43a90
fix: Make errors in rewriting return `Error` rather than a `panic` (#3767) 
* test: add test for predicate errors

* fix: Return errors properly rather than panic

* fix: handle errors in influxrpc planner

* fix: appease clippy

* fix: tests

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-17 15:39:14 +00:00
Andrew Lamb a30803e692
chore: Update datafusion, update `arrow`/`parquet`/`arrow-flight` to 9.0 (#3733) 
* chore: Update datafusion

* chore: Update arrow

* fix: missing updates

* chore: Update cargo.lock

* fix: update for smaller parquet size

* fix: update test for smaller parquet files

* test: ensure parquet_file tests write multiple row groups

* fix: update callsite

* fix: Update for tests

* fix: harkari

* fix: use IoxObjectStore::existing

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-15 12:10:24 +00:00
Andrew Lamb d9f331ba2a
chore: update datafusion, stop repartitioning so aggressively (#3633) 
* chore: update datafusion

* fix: Update to use new datafusion api

* chore: update expected plans

* fix: support zero output partitions

* fix: update test

* fix: Update for new DataFusion API

* fix: newly added system table

* fix: update cargo lock
 2022-02-09 19:53:41 +00:00
Andrew Lamb 85004831a3
refactor: extract predicate sql tests out of rust (#3683) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-09 11:24:34 +00:00
Andrew Lamb 49edfebdf6
refactor: Complete move of test scenarios into library.rs (#3682) 
Co-authored-by: Edd Robinson <me@edd.io>
 2022-02-09 10:46:00 +00:00
Raphael Taylor-Davies c18ad4ac97
feat: special case max timestamp range for table_names and field_columns (#3642) 2022-02-08 16:09:36 +00:00
Andrew Lamb 8d7865496d
refactor: remove out of date comments (#3653) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-07 15:14:34 +00:00
Carol (Nichols || Goulding) 2e30483f1f
refactor: Remove predicate module from predicate crate (#3648) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-07 14:54:07 +00:00
Andrew Lamb 52b2f2e606
refactor: Move more test scenarios into library.rs (#3639) 2022-02-04 10:53:57 +00:00
Andrew Lamb a5045de02c
fix: remove incorrect comment (#3622) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-03 11:06:29 +00:00
kodiakhq[bot] a2ed6a1b75
Merge branch 'main' into combine-non-overlapping-chunks 2022-02-02 20:47:51 +00:00
Andrew Lamb 766bf9826c
refactor: remove unecessary #allow lints (#3615) 
* refactor: remove unecessary #allow lints

* fix: clippy

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 20:23:08 +00:00
Andrew Lamb c4a234e83c
feat: Allow sql test runner to compare sorted output (#3618) 
* refactor: Add Query struct

* feat: Implement sorted checking

* refactor: port some sql tests over

* fix: fmt

* fix: Apply suggestions from code review

Co-authored-by: Edd Robinson <me@edd.io>

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 19:59:52 +00:00
Raphael Taylor-Davies 8a8de19fb5 feat: combine non-overlapping chunks without deletes 2022-02-02 16:40:30 +00:00
Andrew Lamb 55daf95b48
refactor: Consolidate more query_test DbSetups (#3607) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 15:16:56 +00:00
Andrew Lamb 5d5310351b
refactor: Move query_test readme to standard location (#3608) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-02-02 15:06:13 +00:00
Andrew Lamb 62e8b41509
refactor: Start consolidating test scenarios in query_tests (#3580) 
* refactor: Start consolidating test scenarios in query_tests

* fix: reference

* fix: fmt
 2022-01-28 21:12:57 +00:00
Raphael Taylor-Davies 21c1824a7a
refactor: remove table_names from Predicate (#3545) 
* refactor: remove table_names from Predicate

* chore: fix benchmarks

* chore: review feedback

Co-authored-by: Edd Robinson <me@edd.io>

* chore: review feedback

* chore: replace Default::default with InfluxRpcPredicate::default()

Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-27 14:44:49 +00:00
Andrew Lamb 5488c257d1
chore: Update datafusion, upgrade to arrow/parqet/arrow-flight 8.0.0 (#3517) 
* chore: Update datafusion

* chore: update to arrow 8

* fix: update to use new DataFusion APIs

* fix: update case for sortedness

* fix: cargo hakari
 2022-01-27 13:33:27 +00:00
Raphael Taylor-Davies 54ae5de9bf
feat: chunk pruning metrics (#3516) 
Co-authored-by: Edd Robinson <me@edd.io>
 2022-01-25 11:11:50 +00:00
Andrew Lamb 9615feacb3
fix(InfluxQL): Support RegEx with escape sequences not supported by Rust regex (#3502) 
* fix(InfluxQL): Translate unsupported meta characters

* fix: remove debugging

* fix: clippy sacrifice

* docs: Add additional background and rationale for rewriting

* fix: doc link
 2022-01-21 14:40:10 +00:00
Andrew Lamb 9c19cd6cc4
fix: clamp start/end of TimestampRange to min/max valid timestamp values (#3487) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-20 16:08:00 +00:00
Andrew Lamb f0d50f447a
fix: Special case tag_keys with max timestamp range (#3485) 
* fix: Special case tag_keys with max timestamp range

* docs: comment

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-20 14:14:34 +00:00
Andrew Lamb 9b6e626626
chore: Update datafusion (and get fix for influxql test failure) (#3484) 
* test: add tests for comparing dictionary arrays

* chore: update datafusion deps

* refactor: Update code for DataFusion API changes

* fix: update test

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-20 14:01:47 +00:00
Marco Neumann 168afb63ad feat: add `size` methods to DML-related types 
This will be helpful when we want to batch DML operations in memory
(e.g. when using RSKafka).

This also ensures that `MBChunk` accounts for the column names that
are stored within `MutableBatch`.
 2022-01-18 13:52:31 +01:00
Andrew Lamb dd23056efd
chore: update datafusion, arrow, prost, tonic, pbjson, etc (#3455) 
* chore: update datafusion, arrow, prost, tonic, etc

* fix: update pprof as well

* chore: update hakari

* fix: update pbjson

* chore: update heappy

* fix: hakari

* fix: workaround https://github.com/influxdata/influxdb_iox/issues/3458

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-13 17:07:15 +00:00
Andrew Lamb cdf5c21cd4
fix: Fix max timestamp value comparison in chunk metadata (#3453) 
* fix: Fix max timestamp value comparison in chunk metadata

* refactor: rename contains to overlaps

Co-authored-by: Edd Robinson <me@edd.io>
 2022-01-13 16:58:30 +00:00
Andrew Lamb a750da43f1
test: Reproducer for #3428 (#3429) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-13 11:44:39 +00:00
Marco Neumann f3f6f335a9
chore: upgrade to snafu 0.7 (#3440) 2022-01-11 19:22:36 +00:00
Andrew Lamb 336ffd1966
refactor: Remove `Result` in QueryDatabase trait (none of the functions can fail) (#3422) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-01-06 22:03:08 +00:00
Andrew Lamb a93ae739a9
feat: Add table_name to Partition API (#3421) 2022-01-06 16:38:39 +00:00
Carol (Nichols || Goulding) f9174c483b
refactor: Extract server::db into its own crate (#3417) 
* refactor: Extract JobRegistry from the server crate

Both the server crate and a db crate that I'm about to extract depend on
JobRegistry, so to avoid making circular dependencies, extract the
JobRegistry to its own crate.

* refactor: Move db out of server into its own crate

Fixes #2821.
 2021-12-23 22:01:17 +00:00
Andrew Lamb 527885f7f8
chore: Update datafusion (#3413) 
* chore: Update datafusion and update code to handle timezone aware timestamps

* fix: cargo hakari

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2021-12-23 14:52:12 +00:00
Nga Tran 43e3e76de2 chore: Merge branch 'main' into ntran/compact_os_sql_tests 2021-12-08 12:07:02 -05:00
Nga Tran b93b7e9304 refactor: address review comments 2021-12-08 11:50:09 -05:00
Andrew Lamb 218042784f
feat: add `system.queries `system table (#3328) 
* feat: Add query log and system table

* docs: Add docstrings for test normalization
 2021-12-08 16:26:24 +00:00
Andrew Lamb 61dd7e0ba0
chore: clean up `all_chunks_dropped.sql` (#3337) 2021-12-08 14:51:56 +00:00
Nga Tran a13c3120a6 test: sql tests and checkpoint test for compact OS chunks 2021-12-07 17:32:48 -05:00
Carol (Nichols || Goulding) 16d8ae5e04
fix: Match tokio features to what's actually in use in each crate 
Some crates listed features they don't use; other crates ware relying on
feature flags enabled by something else. I tested these changes by
disabling the workspace hack crate and testing each crate.
 2021-12-06 09:37:16 -05:00
Carol (Nichols || Goulding) 02c297e850
fix: Always specify the parking_lot feature of tokio to get potential perf boost 2021-12-06 09:37:15 -05:00