Commit Graph

13797 Commits (18c45d39bb311d052f535ffc7885e3c13b4d1fec)

Author SHA1 Message Date
Fraser Savage 168a9e37fc
fix(ingester): Correctly compute min sequence number for WAL replay debug logging 2023-09-01 15:34:22 +01:00
Dom Dwyer 855e8bb34c
feat(gossip): define CompactionEvent protobuf
This commit adds the CompactionEvent protobuf type that'll be gossiped
between peers to provide a (compact) description of completed
compactions.
2023-09-01 15:16:34 +02:00
Dom 0f138dc2a5
Merge pull request #8645 from influxdata/dom/reuse-sort-key
refactor(persist): load sort key once
2023-09-01 14:11:08 +01:00
Dom Dwyer e5e12ebb1a
refactor: remove Result from infallible return
The compact_persisting_batch() call is infallible, but prior to this
commit, would return a Result with a () unit type as the error type.

This is misleading - it's never going to return an error, so call sites
checking for errors are misleading.
2023-09-01 14:20:06 +02:00
Dom Dwyer 8853296bef
refactor(persist): load sort key once
Passes the already-loaded sort key into update_catalog_sort_key()

This makes the function inputs clear, and avoids loading the same key
twice (which should be a no-op now, but not in the future!).
2023-09-01 14:15:11 +02:00
kodiakhq[bot] edf47e907e
Merge pull request #8644 from influxdata/savage/crc-reader-checksum-consumes-self
refactor(wal): Consume `CrcReader` when invoking `checksum()`
2023-09-01 11:24:50 +00:00
kodiakhq[bot] b27060ea23
Merge branch 'main' into savage/crc-reader-checksum-consumes-self 2023-09-01 11:19:33 +00:00
kodiakhq[bot] c93045bb60
Merge pull request #8634 from influxdata/savage/cli-table-list
feat(cli): Add `table list <DATABASE>` command
2023-09-01 11:17:27 +00:00
kodiakhq[bot] e23c6840cf
Merge branch 'main' into savage/cli-table-list 2023-09-01 11:12:05 +00:00
Dom fbe2723fba
Merge pull request #8643 from influxdata/crepererum/proto_check_iq_proto
chore: apply proto lints to ingester->query gRPC
2023-09-01 12:10:54 +01:00
Fraser Savage b969d07bc7
refactor(wal): Consume `CrcReader` when invoking `checksum()` 2023-09-01 12:06:10 +01:00
Marco Neumann 3498ded991
chore: apply proto lints to ingester->query gRPC
We should catch breaking changes for the ingester->querier protobufs as well.
2023-09-01 13:03:11 +02:00
Marco Neumann 12f2716180
feat: scaffolding for ingester->querier V2 client (#8632)
Adds basic structure for #8349. This will be filled in using separate
PRs for easier review.

The layer structure was chosen to simplify testing and allow composition
of features (like retries, circuit breaking, metrics, etc.). In contrast
to the V1 client (`querier::ingester`) a client here addresses exactly 1
ingester, not multiple (via an `addr` parameter). The tracking around
mutiple states in the V1 version is not really nice and overly
complicated.
2023-09-01 07:58:26 +00:00
dependabot[bot] b337ce91d6
chore(deps): Bump clap from 4.4.1 to 4.4.2 (#8636)
Bumps [clap](https://github.com/clap-rs/clap) from 4.4.1 to 4.4.2.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/v4.4.1...v4.4.2)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-01 07:52:28 +00:00
Marco Neumann 1f732595d1
feat: re-introduce sqlx pool "used" counter (#8624)
This was removed in #8336 because the tracking was broken. However
having an "almost right" metric is still helpful because it helps with
dashboarding (see code comment).

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-31 16:18:36 +00:00
Marco Neumann 1f3ee8bf91
refactor: prep work for #8349 (#8626)
* refactor: make projection masks unsigned

* fix: buffer alignment

* feat: more precise serialization error

* refactor: make `client_util` tower helper public

This can be used for #8349 to set tracing headers.

* fix: impl `Eq` for `TimestampMinMax`

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-31 16:12:50 +00:00
Fraser Savage 4ff3998c2c
docs(cli): Table create should have help text match arg for database 2023-08-31 16:47:11 +01:00
Fraser Savage 00873fac09
test(cli): Add e2e test for `table list` command 2023-08-31 16:47:10 +01:00
Fraser Savage 2a396c1b73
feat(cli): Add `table list <DATABASE>` command
This commit hooks the influxdb_iox_client and CLI up to the new
`GetTables` gRPC endpoint on the `TableService`, allowing users to
query a list of tables within a database and consequently see any
custom partitioning schemes.
2023-08-31 16:47:09 +01:00
Marco Neumann 5af22cefe6
fix: add assertion migration for #8616 (#8623)
We don't think that there are any dirty clusters left, but better be
safe than sorry.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-31 15:07:18 +00:00
Dom 083b516b9d
Merge pull request #8630 from influxdata/dom/merkle-unsigned-port
refactor(proto): unsigned port numbers
2023-08-31 15:29:01 +01:00
Dom 2531cbbad2
Merge branch 'main' into dom/merkle-unsigned-port 2023-08-31 15:10:06 +01:00
kodiakhq[bot] ae4c3e1c2a
Merge pull request #8629 from influxdata/savage/add-get-tables-request-rpc
feat(table): `GetTables` gRPC endpoint implementation
2023-08-31 14:00:31 +00:00
Dom Dwyer 3a2e70157e
refactor(proto): unsigned port numbers
I was trying to remember what the notation for integer types was in
proto - -1 is not a port number...
2023-08-31 15:57:41 +02:00
kodiakhq[bot] b2222bb827
Merge branch 'main' into savage/add-get-tables-request-rpc 2023-08-31 13:55:12 +00:00
Marco Neumann 0b2c883440
fix: clippy (#8625)
I think by default we don't check all tests, but this one annoyed me
because my IDE complained about it.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-31 12:32:27 +00:00
Fraser Savage f110aee262
feat(table): GetTables gRPC endpoint implementation 2023-08-31 13:19:11 +01:00
Fraser Savage 8d47317345
chore(table): Add missing assertion for None partition template 2023-08-31 13:19:10 +01:00
Fraser Savage 1007989cd8
refactor(proto): Impl From<Table> to proto Table 2023-08-31 13:19:09 +01:00
kodiakhq[bot] 5b48b4983b
Merge pull request #8627 from influxdata/savage/define-get-tables-request-rpc
feat(proto): Define GetTables RPC for TableService
2023-08-31 11:54:27 +00:00
kodiakhq[bot] 0c21079884
Merge branch 'main' into savage/define-get-tables-request-rpc 2023-08-31 11:49:12 +00:00
Fraser Savage 683b513e40
feat(proto): Define GetTables RPC for TableService 2023-08-31 12:35:34 +01:00
Dom eb6a4a9dc3
Merge pull request #8611 from influxdata/dom/ingester-gossip-persist
feat(ingester): gossip new parquet file persistence
2023-08-31 11:26:04 +01:00
Dom 45079c0bc1
Merge branch 'main' into dom/ingester-gossip-persist 2023-08-31 11:03:11 +01:00
dependabot[bot] f631b13fb0
chore(deps): Bump chrono from 0.4.27 to 0.4.28 (#8622)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.27 to 0.4.28.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.27...v0.4.28)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-31 09:49:55 +00:00
dependabot[bot] 6c770004af
chore(deps): Bump memchr from 2.6.1 to 2.6.2 (#8621)
Bumps [memchr](https://github.com/BurntSushi/memchr) from 2.6.1 to 2.6.2.
- [Commits](https://github.com/BurntSushi/memchr/compare/2.6.1...2.6.2)

---
updated-dependencies:
- dependency-name: memchr
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-08-31 09:30:55 +00:00
dependabot[bot] 4ce11fd9f2
chore(deps): Bump chrono from 0.4.26 to 0.4.27 (#8607)
* chore(deps): Bump chrono from 0.4.26 to 0.4.27

Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.26 to 0.4.27.
- [Release notes](https://github.com/chronotope/chrono/releases)
- [Changelog](https://github.com/chronotope/chrono/blob/main/CHANGELOG.md)
- [Commits](https://github.com/chronotope/chrono/compare/v0.4.26...v0.4.27)

---
updated-dependencies:
- dependency-name: chrono
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* chore: Run cargo hakari tasks

* fix: Update deprecated chrono methods to their now-recommended versions

`chrono::DateTime::<Tz>::from_utc` has been deprecated and it's now
recommended to use `chrono::DateTime::from_naive_utc_and_offset`
instead.

<https://github.com/chronotope/chrono/pull/1175>

Note that the `Timestamp` type in `influxdb_influxql_parser` is an alias
for `chrono::DateTime<FixedOffset>`.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@gmail.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-08-31 09:18:25 +00:00
wiedld a4567a80e6
chore: more cleanup of compactor panics (#8620) 2023-08-30 14:41:42 -07:00
Dom 44348cdbbb
Merge pull request #8612 from influxdata/dom/merkle-move-mst
refactor: move MST et al into module
2023-08-30 18:11:05 +01:00
Dom bb93cbd3aa
Merge branch 'main' into dom/merkle-move-mst 2023-08-30 17:03:51 +01:00
kodiakhq[bot] 0e5bc6dbb8
Merge pull request #8614 from influxdata/savage/cli-should-display-partition-template
feat(cli): Queries for namespaces and tables should return their `PartitionTemplate`
2023-08-30 15:34:36 +00:00
kodiakhq[bot] c46b754529
Merge branch 'main' into savage/cli-should-display-partition-template 2023-08-30 15:29:07 +00:00
Fraser Savage c0fdd97212
test(e2e): Add minimal coverage that CLI reports custom partition templates 2023-08-30 16:27:03 +01:00
Dom 2f8edd316f
Merge pull request #8617 from influxdata/dom/sync-gossip
feat(gossip): schema sync proto message & topic
2023-08-30 16:10:24 +01:00
Dom a5a3996e1c
Merge branch 'main' into dom/sync-gossip 2023-08-30 16:01:51 +01:00
Dom Dwyer c2d50c0a95
feat(gossip): schema cache convergence topic
Adds a new topic to be used for schema cache convergence messages.
2023-08-30 16:58:44 +02:00
Fraser Savage 1c80c853b4
feat(table): Return PartitionTemplate in table create proto response
If tables can be created with a custom partition template (or the namespace
has a custom partition template) then this should be exposed to the user
interacting with it through the CLI too!
2023-08-30 15:53:54 +01:00
Fraser Savage e602f067ad
feat(namespace): Return namespace custom partition templates to API
This exposes the custom partitionining scheme of a namespace (if any)
in the response to namespace create and list calls.
2023-08-30 15:53:43 +01:00
Dom Dwyer 8751720f1f
feat(gossip): sync message proto
Defines the message one peer will send to another to solicit a
consistency check.
2023-08-30 16:44:26 +02:00
Dom be3c8220f5
Merge pull request #8610 from influxdata/dom/gossip-ident
feat(gossip): expose peer identity & socket address
2023-08-30 15:42:03 +01:00