Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
49,707 Commits
376 Branches
476 Tags
377 MiB
Commit Graph

49707 Commits (fix/docker-python-deps-earlier)

Author SHA1 Message Date
kodiakhq[bot] d38882c746
Merge branch 'main' into cn/cleanups 2023-08-11 13:37:44 +00:00
Carol (Nichols || Goulding) 74685de39b
fix: Set partition column ranges with an empty column range if cached range isn't available 2023-08-11 09:10:01 -04:00
Carol (Nichols || Goulding) 7939b92bb5
fix: Correct spelling in method name 2023-08-11 09:10:01 -04:00
Carol (Nichols || Goulding) e868b3cc13
fix: Don't prune ingester chunks because they've already been pruned 2023-08-11 09:10:01 -04:00
Carol (Nichols || Goulding) 3ad533201a
fix: Don't filter out partitions not in the partition cache 2023-08-11 09:10:01 -04:00
Marco Neumann b11f999806
feat: gRPC interface for ingester->querier v2 (#8443) 
* feat: gRPC interface for ingester->querier v2

Note that the actual "to/from bytes" conversion for the schema and
batches will be implemented in #8347.

Closes #8346.

* fix: typo

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

---------

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-11 09:52:05 +00:00
dependabot[bot] bb1a698cdf
chore(deps): Bump rustix from 0.38.7 to 0.38.8 (#8471) 
Bumps [rustix](https://github.com/bytecodealliance/rustix) from 0.38.7 to 0.38.8.
- [Release notes](https://github.com/bytecodealliance/rustix/releases)
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.38.7...v0.38.8)

---
updated-dependencies:
- dependency-name: rustix
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-11 09:26:32 +00:00
Nga Tran da92a5c9e1
feat: fill catalog `sort_key_ids` for partitions with coming data (#8462) 
* feat: fill catalog sort_key_ids for partition with coming data

* test: sort_key_ids has empty array for newly create partition

* test: name of non-existing column

* chore: add comments to ask Andrew about the code

* chore: make comments clearer

* chore: fix a comment to avoid failure in doc

* chore: add comment for the panic if column name of sort key not found

* fix: during import files the partition has to be created with empty sort key first. Then after its files are created, the partition will be uodated with sort key

* chore: remove no longer needed comments after the bug in build_catalog test is fixed

* chore: address review comments

* refactor: Use ColumnSet type

* chore: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* chore: fix a clippy

---------

Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
 2023-08-10 18:12:40 +00:00
Marco Neumann 9358ec74db
refactor: remove `Predicate` usage from `QueryNamespace` (#8468) 
For #8097.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-10 16:32:55 +00:00
Marco Neumann 3779bda65a
refactor: remove dead code in `predicate` (#8469) 
The pushdown check is a leftover from the old imperative planning code.
This is now handled by
`iox_query::physical_optimizer::predicate_pushdown`.
 2023-08-10 16:03:36 +00:00
Andrew Lamb 232eee059f
chore: Update DataFusion (#8460) 
* chore: Update DataFusion

* chore: update for API changes

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-10 14:54:52 +00:00
Marco Neumann 71e6b66476
refactor: replace `Predicate` w/ `&[Expr]` in querier internals (#8465) 
* refactor: replace `Predicate` w/ `&[Expr]` in querier internals

First step towards #8097. This replaces most internal usages of
`Predicate` with the more appropriate `&[Expr]` within the querier code.

This is also triggered by #8443 because the new ingester protocol shall
not use `Predicate` anymore.

Note that the querier still uses `Predicate` for a few interfaces. These
will be fixed later:

- the current ingester RPC version
- chunk pruning
- `QuerierNamespace::chunks`

* fix: docs
 2023-08-10 13:00:43 +00:00
dependabot[bot] 3675043585
chore(deps): Bump tokio from 1.29.1 to 1.30.0 (#8464) 
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.29.1 to 1.30.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.29.1...tokio-1.30.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-10 07:50:18 +00:00
Marco Neumann f1fbcca23c
fix: make concurrent index creations idempotent (#8450) 
* fix: make concurrent index creations idempotent

`CREATE INDEX CONCURRENTLY` may fail and leave an invalid index behind.
See:
<https://www.postgresql.org/docs/current/sql-createindex.html#SQL-CREATEINDEX-CONCURRENTLY>

Invalid indexes will NOT be used for queries, however they is also useless and
technically the migration wasn't successful. Since #8407 we detect this
situation. #8394 allows us to fix these migrations. #8434 makes sure
that we don't mess this up (esp. that the "other checksum" pragmas are
correct).

For #7897.

* fix: multi-line processing in `check_linear_migrations.sh`

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-09 19:16:35 +00:00
Carol (Nichols || Goulding) 2c66fc9652
fix: Break up long lines/strings 2023-08-09 14:03:20 -04:00
Carol (Nichols || Goulding) c6d607991c
docs: Copyedit migration docs 
Fix some typos, reword or add some words for clarity, add space after
headings for scanability
 2023-08-09 14:02:53 -04:00
Carol (Nichols || Goulding) f847463108
fix: Remove duplicated 'not' in an error message 2023-08-09 13:34:36 -04:00
Carol (Nichols || Goulding) 21eb57953b
fix: Add missing word to an error message 2023-08-09 13:34:16 -04:00
Carol (Nichols || Goulding) 71428b3274
docs: Correct adjective form in a comment 2023-08-09 13:33:49 -04:00
Carol (Nichols || Goulding) 180e982c73
docs: Fix typo 2023-08-09 13:22:33 -04:00
Andrew Lamb d26850390f
refactor(flightsql): Use upstream `SqlInfoData` implementation (#8453) 
* refactor(flightsql): Use upstream implementation

* fix: naming

* fix: better API
 2023-08-09 17:06:43 +00:00
Marco Neumann 7a462754e4
fix: avoid tracing flood (#8458) 
* feat: `RemoveIfHandle::remove_if_and_get_with_status`

* fix: avoid tracing flood

Do not create a span for every partition that we get from the cache
system.

Ref https://github.com/influxdata/idpe/issues/17884.

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-09 14:10:41 +00:00
Andrew Lamb ad663842cb
chore: Update `datafusion` / `arrow` / `parquet` to `45.0.0` (#8452) 
* chore: Update datafusion / arrow / parquet to `45.0.0`

* chore: remove deprecated API

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-09 13:18:18 +00:00
Marco Neumann 530acf170c
test: more automatic tests for migrations (#8457) 
* test: test self-tests

* test: more tests for migrations
 2023-08-09 12:33:53 +00:00
kodiakhq[bot] dccac541a5
Merge pull request #8446 from influxdata/savage/disk-full-ingest-state 
chore(ingester): Add `IngestStateError::DiskFull`
 2023-08-09 08:35:48 +00:00
kodiakhq[bot] a67aa2d322
Merge branch 'main' into savage/disk-full-ingest-state 2023-08-09 08:30:17 +00:00
dependabot[bot] 0973f60d5c
chore(deps): Bump clap from 4.3.19 to 4.3.21 (#8456) 
Bumps [clap](https://github.com/clap-rs/clap) from 4.3.19 to 4.3.21.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.3.19...v4.3.21)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-09 08:27:09 +00:00
Andrew Lamb 1025a85e12
refactor(influxql): remove `copy_with_replacement` and use upstream API (#8451) 
* refactor(influxql): remove `copy_with_replacement` and use upstream version

* fix: clippy

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-08 20:22:29 +00:00
Andrew Lamb 25ab230898
refactor(flightsql): Use upstream implementation of XdbcTypeInfo builder, support filter by datatype (#8455) 
* refactor(flightsql): Use upstream implementation of XdbyTypeInfo builder

* fix: Update test

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-08 20:16:39 +00:00
Joe-Blount 3213396caf
feat: force early L1 compaction, avoid invariant violations (#8444) 
* feat: force early L1 compaction, avoid invariant violations

* chore: variable renaming

* chore: make SplitBasedFileClassifier honor file selection for SimulatedLeadingEdge
 2023-08-08 20:10:36 +00:00
Fraser Savage a3ab4d33da
refactor(router): Revert configurable health-check `ERROR_WINDOW` 
Configuring the `ERROR_WINDOW` of the router's on-path health check
did not provide a consistent improvement for low write volume clusters.
Now that the `NUM_PROBES` parameter is configurable, this can be
un-exposed to simplify configuration options and clean up boiler plate.
 2023-08-08 17:49:14 +01:00
Christopher M. Wolff 3d972561d5
feat: push sort through union (#8423) 
* feat: push sort through union

* test: add a few more tests based on review feedback

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-08 15:00:34 +00:00
Andrew Lamb 46bfa0badc
chore: Update DataFusion (#8447) 
* chore: Update DataFusion pin

* chore: Update for API changes
 2023-08-08 13:41:14 +00:00
dependabot[bot] 71b32fbdba
chore(deps): Bump cc from 1.0.81 to 1.0.82 (#8449) 
Bumps [cc](https://github.com/rust-lang/cc-rs) from 1.0.81 to 1.0.82.
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Commits](https://github.com/rust-lang/cc-rs/compare/1.0.81...1.0.82)

---
updated-dependencies:
- dependency-name: cc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-08 08:54:29 +00:00
Fraser Savage f590d00c74
test(ingester): Property test `IngestStateError` behaviours 
By converting some of the `IngestState` tests to property tests,
coverage of new state errors becomes less error prone.
 2023-08-07 17:38:23 +01:00
Fraser Savage 23d4b19b4f
chore(ingester): Add `IngestStateError::DiskFull` 
This commit adds a new error state for the ingester,
providing a state which can block new writes in the
circumstances where the disk is full or nearing
capacity.
 2023-08-07 17:38:22 +01:00
Marco Neumann 2156bc7e05
feat: static migrations checks (#8434) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-07 15:13:28 +00:00
Marco Neumann 8966cc88df
fix: `O(n^2)` delete handling in LRU (#8416) 
* test: add regression test

* feat: `AddressableHeap::iter`

* fix: `O(n^2)` delete handling in LRU

* refactor: address review comments

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-07 13:46:03 +00:00
Martin Hilton 8e2662befb
feat(influxql): minimal SHOW RETENTION POLICIES implementation (#8433) 
For backwards compatibility with version 1 clients support a minimal
implementation of SHOW RETENTION POLICIES. This advertises a single,
default, retention policy for any database. This is suffiecient for
compatibility with the grafana plugin and avoids the need for
cross-database catalog queries in the querier.

The values used for the "duration", "shardGroupDuration", and
"resourceN" are the same canned values used in the cloud 2
implementation.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-07 13:39:53 +00:00
dependabot[bot] db1429b37d
chore(deps): Bump serde from 1.0.181 to 1.0.183 (#8431) 
Bumps [serde](https://github.com/serde-rs/serde) from 1.0.181 to 1.0.183.
- [Release notes](https://github.com/serde-rs/serde/releases)
- [Commits](https://github.com/serde-rs/serde/compare/v1.0.181...v1.0.183)

---
updated-dependencies:
- dependency-name: serde
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-07 10:24:11 +00:00
dependabot[bot] 548a68a71a
chore(deps): Bump rustix from 0.38.6 to 0.38.7 (#8430) 
Bumps [rustix](https://github.com/bytecodealliance/rustix) from 0.38.6 to 0.38.7.
- [Release notes](https://github.com/bytecodealliance/rustix/releases)
- [Commits](https://github.com/bytecodealliance/rustix/compare/v0.38.6...v0.38.7)

---
updated-dependencies:
- dependency-name: rustix
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-07 10:13:59 +00:00
dependabot[bot] 0e478af470
chore(deps): Bump tempfile from 3.7.0 to 3.7.1 (#8429) 
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.7.0 to 3.7.1.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.7.0...v3.7.1)

---
updated-dependencies:
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-07 09:21:11 +00:00
dependabot[bot] b62e9bb0c8
chore(deps): Bump pin-project from 1.1.2 to 1.1.3 (#8427) 
Bumps [pin-project](https://github.com/taiki-e/pin-project) from 1.1.2 to 1.1.3.
- [Release notes](https://github.com/taiki-e/pin-project/releases)
- [Changelog](https://github.com/taiki-e/pin-project/blob/main/CHANGELOG.md)
- [Commits](https://github.com/taiki-e/pin-project/compare/v1.1.2...v1.1.3)

---
updated-dependencies:
- dependency-name: pin-project
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-07 09:09:23 +00:00
dependabot[bot] e69b2c40c6
chore(deps): Bump regex from 1.9.1 to 1.9.3 (#8426) 
Bumps [regex](https://github.com/rust-lang/regex) from 1.9.1 to 1.9.3.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.9.1...1.9.3)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-07 09:01:07 +00:00
dependabot[bot] ae6d416ad4
chore(deps): Bump filetime from 0.2.21 to 0.2.22 (#8425) 
Bumps [filetime](https://github.com/alexcrichton/filetime) from 0.2.21 to 0.2.22.
- [Commits](https://github.com/alexcrichton/filetime/compare/0.2.21...0.2.22)

---
updated-dependencies:
- dependency-name: filetime
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2023-08-07 08:18:54 +00:00
dependabot[bot] 758c42305d
chore(deps): Bump regex-automata from 0.3.4 to 0.3.6 (#8424) 
Bumps [regex-automata](https://github.com/rust-lang/regex) from 0.3.4 to 0.3.6.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/regex-automata-0.3.4...regex-automata-0.3.6)

---
updated-dependencies:
- dependency-name: regex-automata
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-07 08:05:01 +00:00
Marco Neumann 922a3cd292
feat: harden migration version checks (#8415) 
* refactor: clean up

* feat: harden migration version checks

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-07 07:59:24 +00:00
Andrew Lamb 36276b962e
fix: change max files per plan to something reasonable (#8418) 
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2023-08-04 15:14:57 +00:00
Marco Neumann 614b64d3e4
feat: retry failed migrations (#8414) 2023-08-04 11:15:05 +00:00
Marco Neumann 9803623543
feat: post-migration sanity checks (#8407) 2023-08-04 08:56:22 +00:00