Commit Graph

828 Commits (master)

Author SHA1 Message Date
Jonathan A. Sternberg ce48262c31
feat(storage/flux): support table.fill() within aggregate window pushdown (#21519) 2021-05-19 20:22:56 -05:00
Sam Arnold a40e2920b4
fix: flux to function name conflict (#21496) 2021-05-18 10:40:06 -04:00
Sam Arnold 8124973e0f
fix: disable MergeFiltersRule until it is more stable (#21356)
Add a test to make sure this stays fixed.
2021-05-03 14:25:44 -04:00
Daniel Moran 942f7095e7
fix(query): make config validation for query controller less strict (#21324)
* fix(query): accept queue-size > 0 when concurrency = 0
* fix(influxd): revert defaults for query settings to avoid validation err
* test: lower the default query concurrency used by test launchers
2021-04-28 17:27:37 -04:00
Faith Chikwekwe 7bde3413b3
feat(query): enable min/max pushdown (#20994)
* feat(query): enable min/max pushdown

* fix(query): fix the group last pushdown to use descending cursors

* test(storage): add read group test with no agg

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2021-04-20 12:56:43 -07:00
Sam Arnold 7b2e122869
refactor: refactor for tasks backport (#21117)
* chore: remove dead code

* refactor: move FluxLanguageService interface to fluxlang

* chore: run fmt

* refactor: move task.go from top level to task/taskmodel

* chore: run formatter

* chore: fix up import ordering with gci
2021-04-07 14:42:55 -04:00
Paul Hummer aab53c3d2f
build(flux): update flux to v0.112.0 (#21150) 2021-04-06 12:19:19 -06:00
Sean Brickley f862f8cb1f
fix(storage): Detect need for descending cursor in WindowAggregate (#21140)
* fix(storage): Detect need for descending cursor in WindowAggregate

* chore: Format + comments

* chore: PR cosmetic feedback (#21141)

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>

* chore: rename testcase and fix comments

Co-authored-by: Phil Bracikowski <pbracikowski+git@influxdata.com>
2021-04-05 18:33:07 -04:00
Daniel Moran ff6b55aca4
feat(flux): allow values of 0 to disable controller limits (#21127)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-04-05 16:58:27 -04:00
Daniel Moran 00afd95cb7
refactor: automated move of errors and id from root to kit (#21101)
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
Jonathan A. Sternberg 7b74bb41fc
fix(fluxtest): fix load storage to read/write the bucket correctly (#21058) 2021-03-26 15:58:04 -05:00
Paul Hummer a0dc56f6b2 build(flux): update flux to v0.107.0 2021-03-09 13:25:39 -07:00
Sean Brickley a0d962cc23 chore(flux): Upgrade Flux to v0.104.0 2021-02-02 16:35:33 -05:00
Jonathan A. Sternberg 719c8495f2
feat(fluxtest): add support for testing/expect planner rules (#20662) 2021-02-01 15:53:05 -06:00
Jonathan A. Sternberg 06020b2a49
feat(query/stdlib): promote schema and fill optimizations from feature flags (#20634) 2021-01-29 09:48:02 -06:00
Daniel Moran 9aefa6f868
fix(tsdb): never use an inmem index (#20313)
And fix the logging setup for the TSDB storage engine
2020-12-23 07:46:57 -08:00
Daniel Moran afdcb18655
refactor: simplify how we set the top-level influxd logger (#20374) 2020-12-21 11:15:08 -08:00
Daniel Moran b044a676de
build: delete dead promQL transpiler adapter & test suite (#20368) 2020-12-17 13:42:45 -08:00
Paul Hummer c0bf14cb24 build(flux): update Flux to v0.99.0 2020-12-14 08:07:45 -07:00
Paul Hummer 1f99399d45 build(flux): update Flux to v0.98.0 2020-12-07 09:08:01 -07:00
Paul Hummer 0cf6a97865 build(flux): update Flux to v0.97.0 2020-12-01 09:19:21 -07:00
Dan Moran 295d89ba22 fix: limit query queue size to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Dan Moran 4540645276 fix: limit query concurrency quota to an int32 to avoid panic 2020-11-24 13:36:27 -05:00
Paul Hummer 37e6203c3c build(flux): update Flux to v0.96.0 2020-11-23 08:43:10 -07:00
Andrew Lamb 4bd819f719 fix: update comment about ReadWindowAggregate semantics 2020-11-17 06:22:12 -05:00
Paul Hummer bf37b8b794 build(flux): update Flux to v0.95.0 2020-11-16 08:52:12 -07:00
Daniel Moran 15b9531273
fix: correct various typos (#19987)
Co-authored-by: kumakichi <xyesan@gmail.com>
2020-11-11 13:54:21 -05:00
Paul Hummer 047d3b0874 build(flux): update Flux to v0.94.0 2020-11-09 09:08:01 -07:00
Adrian Thurston 30ad7281aa
refactor: replace builtin package with fluxinit/static (#3304) (#19909)
A static initialization is not desirable in the main binaries, as it forces all
paths of code to init, but it is still useful in tests. It allows static
intialization to be performed once for all tests and eliminates the need to
always add the FluxInit call. Added a fluxinit/static package that calls
fluxinit.FluxInit() to replace the builtin package. This hides the nature of
the initialization and makes it clear that it is mandatory initialization code
getting called.
2020-11-05 16:54:28 -08:00
Paul Hummer fd95506e75 build(flux): update Flux to v0.93.0 2020-11-02 11:49:11 -07:00
Yiqun (Ethan) Zhang 17fcd2dce2
feat: make sure the query plan nodes have unique ids (#19879) 2020-11-02 11:15:39 -06:00
jl fbe85ef548 feat: enable window pushdowns 2020-10-30 18:09:38 -07:00
jl b9d3dbcc0f refactor: remove capabilities from planner tests 2020-10-30 18:09:38 -07:00
Christopher M. Wolff 5a60488729
fix: make tagKeys and tagValues work for edge cases involving _field (#19856) 2020-10-30 10:37:08 -07:00
Faith Chikwekwe 5e1cdda597 feat(storage): enable window agg mean pushdown 2020-10-27 12:53:54 -07:00
Paul Hummer 5884b26d38 build(flux): update Flux to v0.91.0 2020-10-26 13:00:21 -06:00
Paul Hummer 1a81ca4401 build(flux): update Flux to v0.90.0 2020-10-19 11:30:28 -06:00
Paul Hummer 4da32637ed build(flux): update Flux to v0.89.0 2020-10-14 10:51:01 -06:00
Faith Chikwekwe 47ed97c3fa
Merge pull request #19683 from influxdata/test/unskip-agg-med
feat(query/stdlib): unskip agg window median test
2020-10-05 07:50:04 -07:00
Faith Chikwekwe a68fc740b7 feat(query/stdlib): unskip agg window median test 2020-10-02 13:45:24 -07:00
Adrian Thurston feb41819ec
fix: convert flux e2e test harness to two-pass (#19672)
It appears that the double write caused by using to() inside a separate
execution environment (experimental.chain) causes flux e2e tests to behave
unpredictably, when coupled with the 1.x storage engine. Removing the second
write by using two passes, one to write to the db, then another to run the
test, eliminates the flakiness. Verified by running e2e tests in parallel times
8 for 12 hours without any flakiness observed. Before the fix, the flakiness
would take approx 30 minutes on avgerage to exhibit.

This commit also removes universe/to_time from the skipped tests because it was
added when this flakiness was discovered.
2020-10-01 16:38:21 -07:00
Jonathan A. Sternberg bde1209349
fix(tsdb): exclude the stop time from the array cursor (#19622)
This is a backport of #14262 to the 1.x storage engine. The 1.x storage
engine is now the primary engine for open source so when we switched we
regressed to the old behavior.

This also fixes `go generate` for the tsm1 package by running `tmpl`
with `go run` instead of assuming the correct one is installed in the
path.
2020-09-28 12:04:30 -05:00
Stuart Carnie b17acf8b31 fix: Ensure temporary bucket and test data is removed after each test
This is required to keep the system resources low when running
the Flux end-to-end tests, which create a bucket for each test. A
bucket creates at least 17 files after the first write:

* 8 for the `_series` segment files
* 8 for the `index` log files
* 1 for the `wal`
2020-09-21 13:05:50 -07:00
Christopher M. Wolff e7cbbaa722
feat: upgrade Flux to v0.83.2 (#19569) 2020-09-16 17:59:15 -07:00
Brett Buddin b917d8d9b0
chore(influxdb): Placate the linter. 2020-08-27 15:46:32 -04:00
Stuart Carnie dee8977d2c
chore: move v2/v1/tsdb → v2/tsdb 2020-08-26 10:46:47 -07:00
Stuart Carnie 31f4b1a403
fix(flux): Fix broken tests merged from master 2020-08-26 09:44:45 -07:00
Stuart Carnie 3633a974b6
revert(flux): Revert incompatible commit 6f805cb 2020-08-26 09:33:51 -07:00
Stuart Carnie 39ab4a10c5
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	cmd/influxd/launcher/query_test.go
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
#	task/backend/executor/executor.go
#	task/backend/executor/support_test.go
2020-08-26 09:26:43 -07:00
Stuart Carnie 8ff350557f
fix(flux): Skip failing tests to facilitate merge to master
Closes #19442
2020-08-26 09:24:09 -07:00
Yiqun (Ethan) Zhang 6f805cbc2b
feat: flux query profiler (#19359) 2020-08-25 18:00:02 -05:00
Paul Hummer df41d42457 build(flux): update Flux to v0.82.2 2020-08-25 08:04:28 -06:00
Stuart Carnie 505f40c158
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
2020-08-18 14:04:52 -07:00
Paul Hummer 446c13f06b build(flux): update Flux to v0.81.0 2020-08-17 08:55:48 -06:00
Stuart Carnie 56e27b8893
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
2020-08-12 09:07:30 -07:00
Yiqun (Ethan) Zhang b12c3ef77b
build(flux): update Flux to v0.80.0 (#19298) 2020-08-12 10:29:46 -05:00
George 2636e976b9
chore(query): use dbrp v2 service in flux influx.v1 dependencies (#19108)
* chore(query): use dbrp v2 service in flux influx.v1 dependencies

* chore: update transpiler to use dbrp mapping service v2
2020-08-12 11:08:25 +01:00
Paul Hummer 9d7330a1c1 build(flux): update Flux to v0.79.0 2020-08-11 10:27:12 -06:00
jlapacik 46c7345dae
feat: enable pushdowns for sum, count, min, and max by default (#19227) 2020-08-05 12:06:35 -07:00
Adrian Thurston 08cfb3f773
feat: allow a metadata key to be required for a query to be logged (#19200)
Can specify that a key must be present in the query response metadata before
LoggingProxyQueryService logs the query. Will use this in gateway to only log
the query when the connection to queryd fails.
2020-08-05 18:14:53 +03:00
Faith Chikwekwe d48dc690a6
feat(query/stdlib): add min and max to ReadGroup (#19158)
Enables the mix and max aggregates for the ReadGroupAggregte pushdown behind a feature flag.

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2020-08-05 09:40:26 -05:00
Stuart Carnie 87d153c84b
Merge branch 'master' into sgc/tsm1
# Conflicts:
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
2020-08-03 15:23:33 -07:00
Stuart Carnie 262023ae4a
fix(launcher): influxd inspect commands now start instantly
Lazily initialize flux, which increases startup time considerably
2020-08-03 15:16:36 -07:00
Stuart Carnie 0a644dceed
fix(storage): BufferedPointsWriter is for a single org / bucket only
Fixes experimental.to tests
2020-08-03 15:15:00 -07:00
Stuart Carnie f3501aa338
fix(stdlib): Fix unit tests and improved to implementation 2020-08-03 09:21:04 -07:00
Stuart Carnie 81e3a017de
fix(testing): Fix compilation errors 2020-08-03 09:21:04 -07:00
Stuart Carnie 89532f0c29
refactor: move v2/v1/models to v2/models 2020-08-03 09:20:51 -07:00
Stuart Carnie a71b43ff17
refactor: remove redundant files
* retention will be enforced by the InfluxDB 1.x engine
* storage will be a shim for `tsdb`, so don't need tests here
2020-08-03 09:20:50 -07:00
Edd Robinson 2b175291be
refactor: WIP removing tsbd 2020-08-03 09:18:34 -07:00
Stuart Carnie a97ec1f34e
feat: Initial commit with working reads 2020-08-03 09:18:34 -07:00
Stuart Carnie e3060c291c
refactor: tsdb store builds an runs 2020-08-03 09:18:32 -07:00
Paul Hummer 61fe279d4b
build(flux): update Flux to v0.77.1 2020-08-03 09:51:19 -05:00
Yiqun (Ethan) Zhang baa8323431
fix: group resultset produces 1 table per group (#18968)
This helps resolve #18098
2020-07-29 15:31:29 -05:00
Jonathan A. Sternberg 0cdbd496f4
fix(query): paginate bucket results in the bucket lookup (#19111)
The `buckets()` command would use a bucket lookup that wrapped the
`FindBuckets` API. It did not use the pagination aspect of this API
correctly. When the underlying implementation was changed to a version
that correctly implemented pagination, this broke the query `buckets()`
command. Since it was query that used the API incorrectly rather than a
regression in the `FindBuckets` implementation, this fixes the usage to
correctly use pagination.
2020-07-29 11:23:13 -05:00
Yiqun (Ethan) Zhang 8a27388601
feat: log query plan in metadata (#19100) 2020-07-27 17:14:49 -05:00
George 4a845f64cf
chore: simplify to transform dependencies (#18820)
* chore: remove bucket and org cruft from experimental to transformation

* chore: remove URM dependency from bucket auth middleware
2020-07-24 11:49:23 +01:00
Jonathan A. Sternberg 8678a22ee3
feat(storage/flux): add tests for min/max pushdowns (#19028) 2020-07-23 16:07:30 -05:00
Christopher M. Wolff 8b106bcee1
feat(storage): add mean aggregate array cursor and tests (#19019) 2020-07-23 09:49:53 -07:00
Adrian Thurston c693f0b080
test: use experimental.chain in flux e2e tests to allow a single pass (#18898)
Force the writing of data and running of the test to happen sequentially. As
the results come out, collect them and report an error only if the diff results
are not empty.
2020-07-23 15:58:21 +03:00
yukta dbcd363dad feat(query/stdlib): update flags.yml with mergleFilterRule and add mergeFilterRule to rules.go with corresponding unit tests. 2020-07-22 08:16:35 -07:00
yukta 6448a155c3 feat(query/stdlib): update flags.yml with mergleFilterRule and add mergeFilterRule to rules.go with corresponding unit tests. 2020-07-22 08:16:35 -07:00
Paul Hummer aa33f440a0 build(flux): update Flux to v0.74.0 2020-07-21 09:05:26 -06:00
jlapacik 36987bec0f feat: add offset parameter to window cursors 2020-07-20 12:23:37 -07:00
jlapacik 8907eee68d feat: rewrite rule for pushing down window offset 2020-07-20 12:23:37 -07:00
Christopher M. Wolff c695be2f56
feat(storage/reads): add min/max aggregate array cursors (#18957) 2020-07-20 10:17:04 -07:00
Faith Chikwekwe 43b5ef4db4 fix(query): produce error not panic for influxdb.to() 2020-07-15 08:50:15 -07:00
Paul Hummer 0265acf744
Merge pull request #18934 from influxdata/flux-staging/v0.73.0
build(flux): update Flux to v0.73.0
2020-07-14 11:40:27 -06:00
Paul Hummer 21a1faac6e chore: fix accidental typo 2020-07-14 11:20:42 -06:00
Paul Hummer ed66937378 chore: skip unbounded tests 2020-07-14 10:30:47 -06:00
Christopher M. Wolff a68ecc3d03
fix(query/stdlib): exclude logic and errors in Flux function to() (#18937) 2020-07-13 14:09:40 -07:00
Paul Hummer 4264f63822 build(flux): update Flux to v0.73.0 2020-07-13 11:48:19 -06:00
jlapacik 9f9be91828 feat: remove feature flags for first and last pushdowns 2020-07-07 14:15:48 -07:00
jlapacik 3246b3c7ad refactor: remove feature flag for certain pushdowns 2020-07-01 12:59:39 -07:00
Jonathan A. Sternberg e4e28526f8
build(flux): update Flux to v0.71.0 (#18809)
Additional changes:

* fix(query/stdlib): update rewrite rules for schema mutation

The schema mutator was wrapped in a dual implementation spec so the
rewrite rules were type asserting on the wrong type.
2020-07-01 13:12:50 -05:00
Jonathan A. Sternberg 1cd9569bff
feat(flags): add a feature flag for memory optimized schema mutation (#18794) 2020-06-29 16:53:45 -05:00
Jonathan A. Sternberg 5aeca082c8
chore: update staticcheck and fix newly identified lint checks (#18737) 2020-06-26 18:54:09 -05:00
jlapacik 3958c76604 feat: turn on storage capability for first 2020-06-25 14:37:07 -07:00
Jonathan A. Sternberg 61362e4d63
feat(query/stdlib): add a rule to reorder group and window (#18613)
This rule reorders group and window so it will switch from using
`ReadGroup` to using `ReadWindowAggregate` when the intent is to
aggregate a grouped window. It will then add a group node that groups by
the given columns and the start and stop columns and then reperform the
aggregate. This is more performant than performing the group first.
2020-06-22 11:11:27 -05:00
Yiqun (Ethan) Zhang 95d97c6e29
feat(query): planner rule to push down window and bare first() and la… (#18534) 2020-06-17 14:00:40 -05:00
Adrian Thurston 5d1a759170
feat: annotate context with feature flags when handling flux queries (#18506)
Annotate the context with feature flags when handling flux queries in influxdb.
Taking advantage of this in flux end-to-end tests. Using a custom flagger that
can set overrides based on the test case that is about to be run, allowing us
to enable features in the end-to-end tests.
2020-06-17 16:55:29 +03:00