Commit Graph

12273 Commits (b299937d180cf6964053d9bab27f83dde8ea34b8)

Author SHA1 Message Date
Stuart Carnie b299937d18
chore: Resolve tags in GROUP BY clause
Any `VarRef`s in the `Select` where the `data_type` field is `None`
indicates there is no field or tag found in the schema of any children
in the `FROM` clause
2023-05-17 14:55:10 +10:00
Stuart Carnie 9cecb3504c
chore: Remove redundant call to `to_lowercase`
Function call names are normalised to lowercase by the parser.
2023-05-17 14:32:57 +10:00
Stuart Carnie c506ee5946
chore: Add test to validate `as_str` returns unquoted identifier 2023-05-17 14:31:01 +10:00
Stuart Carnie c9f8151302
feat: Resolve data types for `VarRef` nodes in the `WHERE` condition 2023-05-17 12:31:37 +10:00
Stuart Carnie 76aa1ac3d6
feat: Refactor node used to represent a tag in a `GROUP BY`
* Use `VarRef` for `Tag`, to ensure a consistent representation of a
  column reference across `GROUP BY`, `SELECT` projection and `WHERE`
  clause.
* Rename `tags` to `tag_names`
* `tags` now returns `VarRef`
2023-05-17 12:29:02 +10:00
Stuart Carnie 09835e5a98
feat: Implement Hash 2023-05-17 11:20:07 +10:00
Stuart Carnie 5c931fa92c
feat: Add explicit lifetimes 2023-05-17 11:17:57 +10:00
kodiakhq[bot] 95d9553afd
Merge pull request #7773 from influxdata/sgc/issue/6879_subquery_01
feat: Initial support for subqueries
2023-05-16 23:32:27 +00:00
Stuart Carnie 90328e5d0f
chore: PR feedback 2023-05-17 09:24:55 +10:00
Stuart Carnie d7ab96c879
Merge branch 'main' into sgc/issue/6879_subquery_01 2023-05-17 07:20:08 +10:00
wiedld a4ad4fe69e
fix(4895): handle measurement missing, null bytes, and `=` in measurement names (#7759)
* test: add tests for the desired contract for parsing measurements from line protocol
* fix: restrict null chars in measurement
* chore: make an explicit Measurement type
* refactor: have iox lp parser match influxdb contract, for acceptance of eq in measurements
* test: create end_to_end test to confirm same write-then-read behavior with `=` in measurements, is the same as influxdb
2023-05-16 10:48:39 -07:00
wiedld 007460c0a6
Merge pull request #7690 from influxdata/chore/router-metrics-for-auth-v2
chore: authz metric to measure duration of ioxauth rpc call
2023-05-16 10:40:39 -07:00
wiedld 506bd80f6f
Merge branch 'main' into chore/router-metrics-for-auth-v2 2023-05-16 10:24:42 -07:00
Andrew Lamb c077dec0e0
refactor: rename `import` to `iox_import_export` (#7799)
* refactor: rename `import` crate to `import_export`

* chore: Consolidate tsm import/export

---------

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-16 17:10:17 +00:00
Martin Hilton c9cd1fdc44
chore: add a go_package option to the authz proto file (#7802)
This is to fix a downstream service that builds a go package from
these definitions.
2023-05-16 16:51:51 +00:00
kodiakhq[bot] 8fa7e7cbef
Merge pull request #7801 from influxdata/dom/fix-minutes
fix: correct time range buckets
2023-05-16 16:05:36 +00:00
Dom Dwyer 0a31afd00d
fix: correct time range buckets
Turns out there are 60 seconds in a minute, not 3,600.
2023-05-16 17:47:34 +02:00
Jeffrey Smith II 522b1fca4f
Merge pull request #7611 from influxdata/smith/remove-transactions-main
feat: remove transactions from the Catalog
2023-05-16 10:08:05 -04:00
Jeffrey Smith II 87b941180d
Merge branch 'main' into smith/remove-transactions-main 2023-05-16 09:47:51 -04:00
Chunchun Ye 2bb6445668
chore: update DataFusion and arrow / arrow-flight / parquet to `39.0.0` (#7793)
* chore: update DataFusion and arrow/parquet/arrow-flight to 39.0.0

* chore: update DataFusion and arrow/parquet/arrow-flight to 39.0.0 in workspace-hack/Cargo.toml

* chore: Run cargo hakari tasks

* chore: fix CI test and lint

* chore: update csv schema

* refactor: remove type-annotate for `Arc`

---------

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-16 13:42:26 +00:00
Dom f6c3531df7
Merge pull request #7797 from influxdata/dom/file-metrics
feat(metrics): Parquet file attribute statistics
2023-05-16 14:35:25 +01:00
Dom Dwyer 3514667a04
docs: fix comments
Fix two lines.
2023-05-16 15:25:47 +02:00
Jeffrey Smith II 40acf7b86f chore: fix bad merge 2023-05-16 09:22:42 -04:00
Jeffrey Smith II 45628ba4d3 chore: Merge remote-tracking branch 'origin/main' into smith/remove-transactions-main 2023-05-16 08:49:25 -04:00
Dom Dwyer 9b211df053
test(ingester): persist & persistence metrics
Adds a test that asserts (manually triggered) persistence generates a
file, uploads it to object storage, inserts metadata into the catalog,
and emits various persistence metrics.
2023-05-16 14:20:30 +02:00
Dom Dwyer 74210b6257
refactor(ingester): emit Parquet file metrics
Register the ParquetFileInstrumentation as a PersistCompletionObserver
in the persist subsystem.
2023-05-16 14:20:30 +02:00
Dom Dwyer 3114c67cf1
feat: persisted Parquet file attribute metrics
Implements a PersistCompletionObserver that records various attributes
of the generated and persisted Parquet file as histogram metrics to
capture the distribution of values:

    * File size
    * Row count
    * Column count
    * Time range of data (max - min timestamp)

These metrics will give us insight into the generated files instead of
relying on intuition when tuning various configuration parameters.
2023-05-16 14:20:29 +02:00
Dom Dwyer 507ccc2eb5
refactor: parquet metadata in persist notification
Changes the CompletedPersist notification data structure to embed the
generated parquet file's metadata for completion observers.
2023-05-16 14:20:29 +02:00
kodiakhq[bot] 74a7b7102b
Merge pull request #7729 from influxdata/savage/wal-inspect-library-refactor
feat(wal): Add `wal_inspect` crate & a write op entry decoder
2023-05-16 11:33:37 +00:00
Fraser Savage cfb6abf05b
fix(wal_inspect): Make LineProtoWriter flush best effort 2023-05-16 12:26:59 +01:00
kodiakhq[bot] 6bfa269896
Merge branch 'main' into savage/wal-inspect-library-refactor 2023-05-16 11:07:47 +00:00
Fraser Savage a4a22b2732
refactor(wal): Tidy up WriteOpEntryDecoder next() body
Refactor out op comparison for `wal` decode tests to be
more general.
2023-05-16 12:06:54 +01:00
Dom 6febaef382
Merge pull request #7787 from influxdata/dom/parallel-replicate
perf(router): parallel write replication
2023-05-16 11:12:40 +01:00
Dom 3c47226244
Merge branch 'main' into dom/parallel-replicate 2023-05-16 11:05:10 +01:00
Dom 4efc973060
Merge pull request #7796 from influxdata/dependabot/cargo/uuid-1.3.3
chore(deps): Bump uuid from 1.3.2 to 1.3.3
2023-05-16 10:54:11 +01:00
Dom 06a2345708
Merge branch 'main' into dependabot/cargo/uuid-1.3.3 2023-05-16 10:36:46 +01:00
Dom 52a2084637
Merge pull request #7792 from influxdata/cn/refactor-tests
refactor: Extract some catalog test helper methods to reduce places that need to change
2023-05-16 10:36:27 +01:00
Dom f196060fa8
Merge branch 'main' into cn/refactor-tests 2023-05-16 10:26:35 +01:00
dependabot[bot] 3462e29859
chore(deps): Bump uuid from 1.3.2 to 1.3.3
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.3.2 to 1.3.3.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.3.2...1.3.3)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-16 02:00:24 +00:00
Stuart Carnie 5a813fb61f
chore: Simplify first queries 2023-05-16 10:18:34 +10:00
Stuart Carnie d2fe92f71e
chore: Add additional queries to be fixed by #7794 2023-05-16 10:05:04 +10:00
Stuart Carnie d9729ede7d
chore: Fix rustdoc links 2023-05-16 09:21:49 +10:00
Stuart Carnie 2e30442a86
Merge branch 'main' into sgc/issue/6879_subquery_01 2023-05-16 09:15:03 +10:00
Stuart Carnie 4068249349
chore: Fix rustdoc links 2023-05-16 09:13:08 +10:00
Fraser Savage fcd80060be
feat(wal): Make `wal` WriteOpEntryDecoder an interator
Also, implement drop on `wal_inspect`'s LineProtoWriter and
bubble up flush to the caller.
2023-05-15 20:46:56 +01:00
Carol (Nichols || Goulding) 7268ea5c29
refactor: Extract a test helper function to create a basic table 2023-05-15 14:31:24 -04:00
wiedld 2e2aac9ac8 refactor: with updated Authorizer interface, update the metric to delineate the different scenarios 2023-05-15 11:25:01 -07:00
Carol (Nichols || Goulding) 57bedb1c2d
refactor: Extract a test helper function to create a basic namespace 2023-05-15 14:20:38 -04:00
Gavin Cabbage ea3eaee056
chore: accept database in addition to namespace_name (#7788)
* chore: accept database in addition to namespace_name

* chore: accept bucket and bucket-name as well

* test: additional test for sql request namespace aliases
2023-05-15 16:47:24 +00:00
Jeffrey Smith II cfd64e04fb chore: fix bad merge 2023-05-15 12:41:42 -04:00