e6f2a105e5
feat: Improved InfluxQL error messages ( #5632 )
...
* chore: Drive by to improve tests and coverage
* chore: Make Error generic, so we can change it
* chore: change visibility
pub(crate) is superfluous, as we are yet to specify
which APIs are public outside the crate in lib.rs
* chore: Introduce crate IResult type
In preparation of adding custom error type
* feat: Initial implementation of custom error type
* chore: Add module docs
* chore: Rename IResult → ParseResult; syntax and expect errors
* chore: ParserResult and error refactoring
* chore: Drive by simplification
* feat: Add custom errors to string parsing
* feat: Added public API to parse a set of statements
* chore: Errors are dyn Display to convey their intent
Errors from the parser are only displayable messages.
* chore: Separate SHOW for improved error handling
By moving SHOW to a separate parser, we can display clearer error
messages when consuming SHOW followed by an unexpected token.
* chore: Docs and cleanup
* chore: Add tests and a specific `ParseError` type
The fields are intentionally not public yet, as we would like clients
of the package to display the message only.
* chore: PR feedback to improve the `ORDER BY` error message
2022-09-15 00:19:03 +00:00
a5aa871ff8
Merge pull request #5639 from influxdata/cn/always-get-extra-info
...
refactor: Move fetching of table columns, extra partition info into the method
2022-09-14 17:08:57 +00:00
08e2523295
Merge branch 'main' into cn/always-get-extra-info
2022-09-14 17:01:59 +00:00
44e12aa512
feat: add needed budget and memory budget into the message for us to diagnose and increase our memory budget as needed ( #5640 )
2022-09-14 16:06:19 +00:00
e16306d21c
refactor: Move fetching of extra partition info into the method because it's always needed
2022-09-14 11:14:17 -04:00
8b273c2a7d
docs: Add comments about how to see debug logs via `cargo test` ( #5627 )
...
* docs: Add documentation about how to see debug logs via `cargo test`
* fix: Update test_helpers/src/lib.rs
Co-authored-by: Marco Neumann <marco@crepererum.net>
* fix: Update test_helpers/src/lib.rs
Co-authored-by: Marco Neumann <marco@crepererum.net>
* fix: Update test_helpers/src/lib.rs
* fix: fmt
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-14 14:16:46 +00:00
b52865e018
feat: garbage collector now cleans up old parquet files ( #5588 )
...
* feat: garbage collector now cleans up old parquet files
* chore: clarifying comment in GC
* chore: typos in GC
* chore: typos in GC
* fix: cmdline arg in GC test needs updating after refactor
* fix: use select! on shutdown rx in GC
* fix: recalc cutoff in GD each loop
* chore: add delete_old that returns IDs only, for GC
* chore: use duration in GC args instead of usize days
* chore: GC lister runs forever w/ sleep; tests updated accordingly
* docs: fix link in GC comments to automatic link
* chore: test for delete_old_ids_only; refactor mem impl thereof
* chore: make GC test less flakey
* chore: make GC test less flakey
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-14 14:09:28 +00:00
7e1f013346
chore(deps): Bump itertools from 0.10.3 to 0.10.4 ( #5631 )
...
Bumps [itertools](https://github.com/rust-itertools/itertools ) from 0.10.3 to 0.10.4.
- [Release notes](https://github.com/rust-itertools/itertools/releases )
- [Changelog](https://github.com/rust-itertools/itertools/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-itertools/itertools/compare/v0.10.3...v0.10.4 )
---
updated-dependencies:
- dependency-name: itertools
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-14 14:02:14 +00:00
2332e5de10
refactor: slightly increase querier namespace cache TTLs ( #5635 )
...
This should lower catalog load and eliminate a few costly cache misses.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-14 13:54:51 +00:00
1353a429d7
chore(deps): Bump tokio from 1.21.0 to 1.21.1 ( #5630 )
...
* chore(deps): Bump tokio from 1.21.0 to 1.21.1
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.21.0 to 1.21.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.21.0...tokio-1.21.1 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore: Run cargo hakari tasks
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-09-14 13:22:03 +00:00
b4a25fdb0e
chore(deps): Bump thiserror from 1.0.34 to 1.0.35 ( #5629 )
...
Bumps [thiserror](https://github.com/dtolnay/thiserror ) from 1.0.34 to 1.0.35.
- [Release notes](https://github.com/dtolnay/thiserror/releases )
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.34...1.0.35 )
---
updated-dependencies:
- dependency-name: thiserror
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-14 12:54:12 +00:00
9ed931271a
Merge pull request #5634 from influxdata/dom/split-data
...
refactor(ingester): split data.rs into submodules
2022-09-14 13:44:01 +01:00
ee8cdb48af
style(ingester): fmt imports & long strings
...
Rewrite the imports to be a consistent order; std, external, crate and
merge all crate-level imports into one use statement.
2022-09-14 14:20:19 +02:00
074722eb3e
refactor(ingester): split data.rs into modules
...
Breaks the gigantic data.rs file into sub-modules for Shard, Namespace,
Table, Partition, and finally the actual data buffer used to store
writes.
2022-09-14 14:20:19 +02:00
45d795055a
feat: Support calling influxql/flux selector aggregates from IOx SQL ( #5628 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-14 10:37:17 +00:00
674670e17f
Merge pull request #5622 from influxdata/cn/infallible-estimated-bytes
...
fix: Use `ColumnType` as an enum in more places; make `estimate_arrow_bytes_for_file` infallible
2022-09-14 01:07:05 +00:00
85641efa6f
Merge branch 'main' into cn/infallible-estimated-bytes
2022-09-14 01:00:10 +00:00
51dac55652
Merge pull request #5567 from influxdata/chore/parquetfile-size-trigger
...
feat: parquetfile size trigger
2022-09-13 16:39:57 +01:00
ee3f172d45
chore: renamed DB migration for billing trigger
2022-09-13 16:29:14 +01:00
c8b545134e
chore: add index to speed up billing_summary upsert
2022-09-13 16:22:44 +01:00
feae712881
fix: parquet_file billing trigger respects to_delete
2022-09-13 16:22:44 +01:00
80661a5d1c
chore: clippy
2022-09-13 16:22:44 +01:00
10acaf4567
chore: added test for parquet file delete trigger
2022-09-13 16:22:44 +01:00
cc93b2c275
chore: add catalog trigger for billing
2022-09-13 16:22:44 +01:00
f21cb43624
feat: add a few more buckets for the histograms ( #5621 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-13 13:52:23 +00:00
9d63edb299
chore: remove unecessary batch check ( #5620 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-13 12:41:59 +00:00
f86d3e31da
chore: Update datafusion + object_store ( #5619 )
...
* chore: Update datafusion pin
* chore: update object_store to 0.5.0
* chore: Run cargo hakari tasks
Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-13 12:34:54 +00:00
9e21c1c06b
chore(deps): Bump clap from 3.2.20 to 3.2.21 ( #5623 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 3.2.20 to 3.2.21.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/v3.2.21/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v3.2.20...v3.2.21 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-13 07:23:26 +00:00
e4a66f69c7
refactor: dedicated query path for querier tombstone materialization ( #5618 )
...
I would like to remove de-dup from the querier<>ingester query path in #5602 ,
but the tombstone application and parquet write path can still do the full work.
2022-09-13 07:16:38 +00:00
51918657c3
chore: update axum ( #5625 )
...
This fixes <https://rustsec.org/advisories/RUSTSEC-2022-0055 >.
2022-09-13 07:05:12 +00:00
d971980fd3
fix: Box a source error to please clippy
2022-09-12 17:38:40 -04:00
8a594621bc
fix: Converting a Column to a ColumnSchema can now be infallible too
2022-09-12 17:35:52 -04:00
224e3cec10
fix: Use ColumnType in errors rather than strings
2022-09-12 17:35:27 -04:00
c3937308f4
fix: Make estimate_arrow_bytes_for_file infallible
2022-09-12 16:50:25 -04:00
20e6d26aa9
refactor: Have sqlx decode ColumnTypes in the catalog
2022-09-12 16:50:25 -04:00
1fd31ee3bf
chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0 ( #5591 )
...
* chore: Update datafusion / `arrow` / `arrow-flight` / `parquet` to version 22.0.0
* fix: enable dynamic comparison flag
* chore: derive Eq for clippy
* chore: update explain plans
* chore: Update sizes for ReadBuffer encoding
* chore: update more tests
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-09-12 17:45:03 +00:00
65ba752544
Merge pull request #5604 from influxdata/cn/404
...
fix: Don't try to parse /api/v2/query response as JSON
2022-09-12 17:37:11 +00:00
0213a12db6
Merge branch 'main' into cn/404
2022-09-12 17:24:39 +00:00
ed5276369e
Merge pull request #5386 from influxdata/cn/brrr
...
feat: full cold compaction
2022-09-12 17:24:21 +00:00
f7a1937f96
fix: Rename influxdb2_client query to query_raw
2022-09-12 13:14:21 -04:00
0d313a4579
fix: Don't try to parse /api/v2/query response as JSON
...
It's just plain text afaict.
2022-09-12 13:14:20 -04:00
e7a3f15ecf
test: Remove outdated description
2022-09-12 13:13:30 -04:00
8981cbbd84
test: Reduce time from 18 to 9 hours
2022-09-12 13:13:29 -04:00
2ceb779c28
test: Correct a comment that I missed in the 24 hr -> 8 hr switch
2022-09-12 13:13:29 -04:00
baec40a313
test: Correct and expand assertions and descriptions
2022-09-12 13:13:29 -04:00
aba9759268
test: Correct expectations with skipped compactions
2022-09-12 13:13:29 -04:00
2aef7c7936
feat: Temporarily disable cold full compaction
2022-09-12 13:13:29 -04:00
3cea67fe36
test: Compaction of 1 file now upgrades the level, so the ID doesn't change
2022-09-12 13:13:29 -04:00
743b67f0e9
fix: Re-enable full cold compaction, in serial for now
2022-09-12 13:13:29 -04:00
6e1b06c435
fix: Work with Arc of PartitionCompactionCandidateWithInfo
2022-09-12 13:13:29 -04:00
Stuart Carnie
kodiakhq[bot]
Nga Tran
Carol (Nichols || Goulding)
Andrew Lamb
Luke Bond
dependabot[bot]
Marco Neumann
Dom