Commit Graph

738 Commits (feb41819ec33145b2f246ba3deaab992445d025f)

Author SHA1 Message Date
Adrian Thurston feb41819ec
fix: convert flux e2e test harness to two-pass (#19672)
It appears that the double write caused by using to() inside a separate
execution environment (experimental.chain) causes flux e2e tests to behave
unpredictably, when coupled with the 1.x storage engine. Removing the second
write by using two passes, one to write to the db, then another to run the
test, eliminates the flakiness. Verified by running e2e tests in parallel times
8 for 12 hours without any flakiness observed. Before the fix, the flakiness
would take approx 30 minutes on avgerage to exhibit.

This commit also removes universe/to_time from the skipped tests because it was
added when this flakiness was discovered.
2020-10-01 16:38:21 -07:00
Jonathan A. Sternberg bde1209349
fix(tsdb): exclude the stop time from the array cursor (#19622)
This is a backport of #14262 to the 1.x storage engine. The 1.x storage
engine is now the primary engine for open source so when we switched we
regressed to the old behavior.

This also fixes `go generate` for the tsm1 package by running `tmpl`
with `go run` instead of assuming the correct one is installed in the
path.
2020-09-28 12:04:30 -05:00
Stuart Carnie b17acf8b31 fix: Ensure temporary bucket and test data is removed after each test
This is required to keep the system resources low when running
the Flux end-to-end tests, which create a bucket for each test. A
bucket creates at least 17 files after the first write:

* 8 for the `_series` segment files
* 8 for the `index` log files
* 1 for the `wal`
2020-09-21 13:05:50 -07:00
Christopher M. Wolff e7cbbaa722
feat: upgrade Flux to v0.83.2 (#19569) 2020-09-16 17:59:15 -07:00
Brett Buddin b917d8d9b0
chore(influxdb): Placate the linter. 2020-08-27 15:46:32 -04:00
Stuart Carnie dee8977d2c
chore: move v2/v1/tsdb → v2/tsdb 2020-08-26 10:46:47 -07:00
Stuart Carnie 31f4b1a403
fix(flux): Fix broken tests merged from master 2020-08-26 09:44:45 -07:00
Stuart Carnie 3633a974b6
revert(flux): Revert incompatible commit 6f805cb 2020-08-26 09:33:51 -07:00
Stuart Carnie 39ab4a10c5
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	cmd/influxd/launcher/query_test.go
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
#	task/backend/executor/executor.go
#	task/backend/executor/support_test.go
2020-08-26 09:26:43 -07:00
Stuart Carnie 8ff350557f
fix(flux): Skip failing tests to facilitate merge to master
Closes #19442
2020-08-26 09:24:09 -07:00
Yiqun (Ethan) Zhang 6f805cbc2b
feat: flux query profiler (#19359) 2020-08-25 18:00:02 -05:00
Paul Hummer df41d42457 build(flux): update Flux to v0.82.2 2020-08-25 08:04:28 -06:00
Stuart Carnie 505f40c158
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
2020-08-18 14:04:52 -07:00
Paul Hummer 446c13f06b build(flux): update Flux to v0.81.0 2020-08-17 08:55:48 -06:00
Stuart Carnie 56e27b8893
Merge remote-tracking branch 'origin/master' into sgc/tsm1
# Conflicts:
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
2020-08-12 09:07:30 -07:00
Yiqun (Ethan) Zhang b12c3ef77b
build(flux): update Flux to v0.80.0 (#19298) 2020-08-12 10:29:46 -05:00
George 2636e976b9
chore(query): use dbrp v2 service in flux influx.v1 dependencies (#19108)
* chore(query): use dbrp v2 service in flux influx.v1 dependencies

* chore: update transpiler to use dbrp mapping service v2
2020-08-12 11:08:25 +01:00
Paul Hummer 9d7330a1c1 build(flux): update Flux to v0.79.0 2020-08-11 10:27:12 -06:00
jlapacik 46c7345dae
feat: enable pushdowns for sum, count, min, and max by default (#19227) 2020-08-05 12:06:35 -07:00
Adrian Thurston 08cfb3f773
feat: allow a metadata key to be required for a query to be logged (#19200)
Can specify that a key must be present in the query response metadata before
LoggingProxyQueryService logs the query. Will use this in gateway to only log
the query when the connection to queryd fails.
2020-08-05 18:14:53 +03:00
Faith Chikwekwe d48dc690a6
feat(query/stdlib): add min and max to ReadGroup (#19158)
Enables the mix and max aggregates for the ReadGroupAggregte pushdown behind a feature flag.

Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
2020-08-05 09:40:26 -05:00
Stuart Carnie 87d153c84b
Merge branch 'master' into sgc/tsm1
# Conflicts:
#	go.mod
#	go.sum
#	query/promql/internal/promqltests/go.mod
2020-08-03 15:23:33 -07:00
Stuart Carnie 262023ae4a
fix(launcher): influxd inspect commands now start instantly
Lazily initialize flux, which increases startup time considerably
2020-08-03 15:16:36 -07:00
Stuart Carnie 0a644dceed
fix(storage): BufferedPointsWriter is for a single org / bucket only
Fixes experimental.to tests
2020-08-03 15:15:00 -07:00
Stuart Carnie f3501aa338
fix(stdlib): Fix unit tests and improved to implementation 2020-08-03 09:21:04 -07:00
Stuart Carnie 81e3a017de
fix(testing): Fix compilation errors 2020-08-03 09:21:04 -07:00
Stuart Carnie 89532f0c29
refactor: move v2/v1/models to v2/models 2020-08-03 09:20:51 -07:00
Stuart Carnie a71b43ff17
refactor: remove redundant files
* retention will be enforced by the InfluxDB 1.x engine
* storage will be a shim for `tsdb`, so don't need tests here
2020-08-03 09:20:50 -07:00
Edd Robinson 2b175291be
refactor: WIP removing tsbd 2020-08-03 09:18:34 -07:00
Stuart Carnie a97ec1f34e
feat: Initial commit with working reads 2020-08-03 09:18:34 -07:00
Stuart Carnie e3060c291c
refactor: tsdb store builds an runs 2020-08-03 09:18:32 -07:00
Paul Hummer 61fe279d4b
build(flux): update Flux to v0.77.1 2020-08-03 09:51:19 -05:00
Yiqun (Ethan) Zhang baa8323431
fix: group resultset produces 1 table per group (#18968)
This helps resolve #18098
2020-07-29 15:31:29 -05:00
Jonathan A. Sternberg 0cdbd496f4
fix(query): paginate bucket results in the bucket lookup (#19111)
The `buckets()` command would use a bucket lookup that wrapped the
`FindBuckets` API. It did not use the pagination aspect of this API
correctly. When the underlying implementation was changed to a version
that correctly implemented pagination, this broke the query `buckets()`
command. Since it was query that used the API incorrectly rather than a
regression in the `FindBuckets` implementation, this fixes the usage to
correctly use pagination.
2020-07-29 11:23:13 -05:00
Yiqun (Ethan) Zhang 8a27388601
feat: log query plan in metadata (#19100) 2020-07-27 17:14:49 -05:00
George 4a845f64cf
chore: simplify to transform dependencies (#18820)
* chore: remove bucket and org cruft from experimental to transformation

* chore: remove URM dependency from bucket auth middleware
2020-07-24 11:49:23 +01:00
Jonathan A. Sternberg 8678a22ee3
feat(storage/flux): add tests for min/max pushdowns (#19028) 2020-07-23 16:07:30 -05:00
Christopher M. Wolff 8b106bcee1
feat(storage): add mean aggregate array cursor and tests (#19019) 2020-07-23 09:49:53 -07:00
Adrian Thurston c693f0b080
test: use experimental.chain in flux e2e tests to allow a single pass (#18898)
Force the writing of data and running of the test to happen sequentially. As
the results come out, collect them and report an error only if the diff results
are not empty.
2020-07-23 15:58:21 +03:00
yukta dbcd363dad feat(query/stdlib): update flags.yml with mergleFilterRule and add mergeFilterRule to rules.go with corresponding unit tests. 2020-07-22 08:16:35 -07:00
yukta 6448a155c3 feat(query/stdlib): update flags.yml with mergleFilterRule and add mergeFilterRule to rules.go with corresponding unit tests. 2020-07-22 08:16:35 -07:00
Paul Hummer aa33f440a0 build(flux): update Flux to v0.74.0 2020-07-21 09:05:26 -06:00
jlapacik 36987bec0f feat: add offset parameter to window cursors 2020-07-20 12:23:37 -07:00
jlapacik 8907eee68d feat: rewrite rule for pushing down window offset 2020-07-20 12:23:37 -07:00
Christopher M. Wolff c695be2f56
feat(storage/reads): add min/max aggregate array cursors (#18957) 2020-07-20 10:17:04 -07:00
Faith Chikwekwe 43b5ef4db4 fix(query): produce error not panic for influxdb.to() 2020-07-15 08:50:15 -07:00
Paul Hummer 0265acf744
Merge pull request #18934 from influxdata/flux-staging/v0.73.0
build(flux): update Flux to v0.73.0
2020-07-14 11:40:27 -06:00
Paul Hummer 21a1faac6e chore: fix accidental typo 2020-07-14 11:20:42 -06:00
Paul Hummer ed66937378 chore: skip unbounded tests 2020-07-14 10:30:47 -06:00
Christopher M. Wolff a68ecc3d03
fix(query/stdlib): exclude logic and errors in Flux function to() (#18937) 2020-07-13 14:09:40 -07:00