Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
9,866 Commits
203 Branches
453 Tags
381 MiB
Commit Graph

9866 Commits (651b7a1ce64df19c730e1fd721a1dcf4ecf618cf)

Author SHA1 Message Date
Luke Bond bf1681f4fe
feat: compactor ignores max file count for first file (#6144) 
* feat: compactor ignores max file count for first file

* chore: typo in comment in compactor
 2022-11-16 11:21:28 +00:00
Marco Neumann 62851afc27
feat: add querier->ingester circuit breaker (#6147) 
* feat: add log ingester memory pressure persist

* feat: add querier->ingester circuit breaker

Closes #4608.

* docs: explain high-level circuit breaker

* docs: improve

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

* test: add additional test assertion

* refactor: upgrade info to warning log

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
 2022-11-16 10:50:33 +00:00
Dom 456314d99f
Merge pull request #6149 from influxdata/dependabot/cargo/clap-4.0.25 
chore(deps): Bump clap from 4.0.24 to 4.0.25
 2022-11-16 10:02:49 +00:00
dependabot[bot] dbe2744e04
chore(deps): Bump clap from 4.0.24 to 4.0.25 
Bumps [clap](https://github.com/clap-rs/clap) from 4.0.24 to 4.0.25.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.24...v4.0.25)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-11-16 01:07:58 +00:00
Andrew Lamb 448911794c
test: test coverage for sorting and merging in compactor (#6136) 
* test: test coverage for sorting and merging in compactor

* fix: Apply suggestions from code review (comments)

Co-authored-by: Marco Neumann <marco@crepererum.net>

* feat: use itertools to cover all permutations

Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-15 20:39:45 +00:00
Andrew Lamb 20f1ae1c8f
test: tests in the reorg planner and query tests for merging parquet files (#6137) 
* test: tests in the reorg planner and query tests for merging parquet files

* fix: use 20 files

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-15 20:29:44 +00:00
Dom 2a9d330ef3
Merge pull request #6145 from influxdata/dom/rpc-write 
feat: define router -> ingester write protocol
 2022-11-15 14:45:33 +00:00
Dom 685a0858dc
Merge branch 'main' into dom/rpc-write 2022-11-15 14:38:15 +00:00
Dom 5865f68852
Merge pull request #6138 from influxdata/cn/remove-namespace 
fix: Remove namespace string from router -> write buffer -> ingester communication
 2022-11-15 14:37:52 +00:00
Dom 5ed1d8bc79
Merge branch 'main' into cn/remove-namespace 2022-11-15 14:30:44 +00:00
Dom d2e2e0a3cd
Merge pull request #6141 from influxdata/dependabot/cargo/insta-1.21.1 
chore(deps): Bump insta from 1.21.0 to 1.21.1
 2022-11-15 14:28:31 +00:00
Dom Dwyer dcc7b10bcf
feat: define router -> ingester write protocol 
Specify a gRPC service and request/response message formats to push
writes directly from a router to an ingester.
 2022-11-15 15:00:38 +01:00
dependabot[bot] fa07a107a1
chore(deps): Bump insta from 1.21.0 to 1.21.1 
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.21.0 to 1.21.1.
- [Release notes](https://github.com/mitsuhiko/insta/releases)
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/commits/1.21.1)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-11-15 12:01:26 +00:00
dependabot[bot] 9db808f795
chore(deps): Bump similar from 2.2.0 to 2.2.1 (#6140) 
Bumps [similar](https://github.com/mitsuhiko/similar) from 2.2.0 to 2.2.1.
- [Release notes](https://github.com/mitsuhiko/similar/releases)
- [Changelog](https://github.com/mitsuhiko/similar/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/similar/compare/2.2.0...2.2.1)

---
updated-dependencies:
- dependency-name: similar
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-15 11:59:19 +00:00
dependabot[bot] 1bb19e2137
chore(deps): Bump clap from 4.0.23 to 4.0.24 (#6139) 
Bumps [clap](https://github.com/clap-rs/clap) from 4.0.23 to 4.0.24.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.23...v4.0.24)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-15 11:09:39 +00:00
dependabot[bot] e4de0635ad
chore(deps): Bump uuid from 1.2.1 to 1.2.2 (#6143) 
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.2.1...1.2.2)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2022-11-15 09:31:59 +00:00
Carol (Nichols || Goulding) a4b277594c
fix: Remove namespace argument from kafka test helper fn 2022-11-14 16:46:05 -05:00
Carol (Nichols || Goulding) 5854ff28e9
fix: Remove namespace argument from write test helper fn 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) c27d3a22d2
fix: Remove namespace argument from test helper function 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) 3943faf998
fix: Remove namespace from DmlWrite and DmlDelete constructors 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) f78195f7c7
fix: Remove namespace name field from DmlWrite and DmlDelete 
But leave the argument in their constructors for now.

Not all numbers in tests can be 42, Dom.
 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) c203e8295f
test: Keep track of namespaces by ID in ingester TestContext 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) e2d2e69076
fix: Remove db_name from encode_operation and FileBufferProducer 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) 8dbdab8754
fix: Remove db_name field from DeletePayload 
Doesn't seem to be used anywhere.
 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) f4b8fe7751
fix: Remove now-unused db_name argument to encode_write 2022-11-14 16:46:04 -05:00
Carol (Nichols || Goulding) 4d4cdebbda
fix: Remove database_name field from DatabaseBatch 
It was only being used in one error message.
 2022-11-14 16:46:03 -05:00
Carol (Nichols || Goulding) 2c4c578df3
fix: Remove namespace name from IoxHeaders in write messages 2022-11-14 16:46:03 -05:00
Carol (Nichols || Goulding) a5e5d911c2
docs: Remove outdated comments about safety; the temporary unsafe code is gone 2022-11-14 16:46:03 -05:00
Carol (Nichols || Goulding) 0d8e756e1b
fix: Update docs on the ingester-querier protocol. 
I believe this fixes #6049.
 2022-11-14 16:46:03 -05:00
dependabot[bot] c902f33178
chore(deps): Bump sqlparser from 0.26.0 to 0.27.0 (#6131) 
* chore(deps): Bump sqlparser from 0.26.0 to 0.27.0

Bumps [sqlparser](https://github.com/sqlparser-rs/sqlparser-rs) from 0.26.0 to 0.27.0.
- [Release notes](https://github.com/sqlparser-rs/sqlparser-rs/releases)
- [Changelog](https://github.com/sqlparser-rs/sqlparser-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sqlparser-rs/sqlparser-rs/compare/v0.26.0...v0.27.0)

---
updated-dependencies:
- dependency-name: sqlparser
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix: Ignore new modifiers field of the sqlparser Custom data type

See <https://github.com/sqlparser-rs/sqlparser-rs/pull/680>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-14 19:56:28 +00:00
kodiakhq[bot] e8b912e049
Merge pull request #6135 from influxdata/cn/document-json-ticket 
fix: Improve testing and documentation to clarify Go client behavior
 2022-11-14 19:05:09 +00:00
kodiakhq[bot] 1881fa9204
Merge branch 'main' into cn/document-json-ticket 2022-11-14 18:58:52 +00:00
kodiakhq[bot] d0b3ac1cef
Merge pull request #6132 from influxdata/dom/deferred-table-name 
perf(ingester): deferred table name discovery
 2022-11-14 18:35:35 +00:00
kodiakhq[bot] 6c1e9f04ef
Merge branch 'main' into dom/deferred-table-name 2022-11-14 18:22:46 +00:00
Carol (Nichols || Goulding) fd898cea2a
docs: Correct grammar and update outdated comment 2022-11-14 13:21:55 -05:00
Carol (Nichols || Goulding) b0cf10c9aa
test: Add a test for decoding JSON tickets to ReadInfo 
Enshrine in a test what the Go clients are currently sending to avoid
breakage in the future, hopefully.
 2022-11-14 12:55:03 -05:00
Carol (Nichols || Goulding) 4f081a8405
fix: Remove 'legacy' from JSON ticket format as it's still used by Go clients 2022-11-14 12:46:58 -05:00
dependabot[bot] a969754819
chore(deps): Bump chrono from 0.4.22 to 0.4.23 (#6129) 
* chore(deps): Bump chrono from 0.4.22 to 0.4.23

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.22 to 0.4.23.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.22...v0.4.23)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* refactor: chrono future compat

Integer->timstamp conversions should not silently panic.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
 2022-11-14 13:34:09 +00:00
Dom Dwyer 413b7c8f4a
refactor: use table name from catalog 
Changes the TableData within the ingester to utilise a TableNameResolver
to fetch the TableName via the catalog on demand / in the background,
instead of using the table name sent over the write.

This change causes the ingester to perform a catalog query in the
background (or on demand) to resolve the table name. This is a
pre-requisite for removing the table name from the write wire format.
 2022-11-14 11:32:22 +01:00
Dom Dwyer 0df6c7877c
refactor: indirect DeferredLoad<TableName> init 
Like the NamespaceNameProvider, this commit adds a TableNameProvider to
provide decoupled initialisation of a DeferredLoad<TableName> instead of
hard-coding in a catalog instance / query code, and plumbs it into
position to be used when initialising a TableName.
 2022-11-14 11:32:21 +01:00
Dom Dwyer 8dae6d3994
perf(ingester): address tables by ID only 
Changes the buffer tree to address TableData by their ID only (removing
support for addressing tables by their string names). This removes the
double reference book keeping / twin indexes and associated overhead.

As part of this change, the TableName is now wrapped in a DeferredLoad
in preparation for removal of the names in the DmlOperation wire format.

This commit also switches the map of TableData within the NamespaceData
(the parent node) to use the ArcMap for faster lookups and DRY
exactly-once initialisation.
 2022-11-14 11:27:19 +01:00
kodiakhq[bot] d2c515b069
Merge pull request #6122 from influxdata/dom/internalise-partition-resolver 
refactor: internalise PartitionProvider
 2022-11-14 09:57:20 +00:00
Dom Dwyer d8fc9ff258
test: fix testing deadlocks 
The MemCatalog suffers from deadlocks when attempting to obtain a second
ref to RepoCollection:

    https://github.com/influxdata/influxdb_iox/issues/3859
 2022-11-14 10:50:10 +01:00
Dom Dwyer 9e97866b48
refactor: internalise PartitionProvider 
Removes the need to leak the PartitionProvider outside of the ingester
crate.

This will allow the PartitionProvider to utilise a
DeferredLoad<TableName> without having to make the DeferredLoad and
TableName pub.
 2022-11-14 10:50:05 +01:00
Dom 96a06c060d
Merge pull request #6130 from influxdata/dependabot/cargo/clap-4.0.23 
chore(deps): Bump clap from 4.0.22 to 4.0.23
 2022-11-14 09:45:10 +00:00
dependabot[bot] f144231e8c
chore(deps): Bump clap from 4.0.22 to 4.0.23 
Bumps [clap](https://github.com/clap-rs/clap) from 4.0.22 to 4.0.23.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.0.22...v4.0.23)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2022-11-14 01:07:26 +00:00
kodiakhq[bot] 6fc1bb4e70
Merge pull request #6116 from influxdata/dependabot/cargo/hashbrown-0.13.1 
chore(deps): Bump hashbrown from 0.12.3 to 0.13.1
 2022-11-11 21:33:28 +00:00
kodiakhq[bot] 05d7d1495e
Merge branch 'main' into dependabot/cargo/hashbrown-0.13.1 2022-11-11 21:26:40 +00:00
kodiakhq[bot] e7ac4eb7fe
Merge pull request #6127 from influxdata/cn/database 
fix: Rename more instances of "database" to "namespace"
 2022-11-11 21:21:06 +00:00
Carol (Nichols || Goulding) 98621f560b
fix: Remove obsolete SERVER_MODE from the Dockerfile 2022-11-11 16:14:13 -05:00