Commit Graph

21 Commits (933a14e16f0232c5ec883644b7363a59da7bcd90)

Author SHA1 Message Date
Jeffrey Smith II fce0d1c863
chore: update to go 1.19 (#24119)
* chore: update to go 1.19.6

* chore: gofmt

* test: fix tests for sort order change

* chore: generate pb

* feat: upgrade flux to v0.188.0 (#23911)

* feat: upgrade flux to 0.171.0

Tests failing, safety commit

First step in https://github.com/influxdata/influxdb/issues/23815

* fix: remove "org" parameter" from writeOptSource

I attempted to implement the "orgOpt" argument in a similar fashion
to f6669f7512. However, it looks like Flux doesn't accept "org" as
a parameter to "load". It responds with:

Error calling function \"load\" @113:16-113:30: error calling function \"to\" @6:19-6:47: unused arguments [org]

This brings us from 194 passing to 570 passing.

* fix: temporarily disable broken flux tests

These tests expect rows to be stored in a certain order. However,
nothing is specifying the sort order. This has been fixed in a
later update to flux: (see 3d6f47ded).

Temporarily disable these tests until we include a fixed
version of the flux tests.

* chore: add tests from a492993012

This fixes "test-flux.sh" so it runs tests within the "flux/"
directory. This uncovered some other issues with the tests
located within "flux/". These also needed to be updated
to match the newer flux API.

* feat: upgrade flux to 0.172.0

This includes changes made in "cbbf4b27da". Since "test.go" in 2.x
diverged from 1.x, some modifications were required to make this
compatible.

* feat: upgrade flux to 0.173.0

* feat: upgrade flux to v0.174.0

* fix: Update the condition when reseting cursor (#23522)

Filters that contain `or` may change between cursor resets so we must remember to update the condition in the read cursor.

```flux
|> filter(fn: (r) => ((r["_field"] == "field1" and r["_value"]==true) or (r["_field"] == "field2" and r["_value"] == false)))
```

Closes https://github.com/influxdata/flux/issues/4804

* feat: upgrade flux to 0.174.1

* feat: upgrade flux to 0.175.0

* chore: remove end-to-end tests

These were removed in a492993 for 2.x. These tests prevent "go test ./..."
from completing. As stated in the original commit, these tests should now be
handled by the "fluxtest" harness.

* feat: upgrade flux to 0.176.0

Some tests needed to be disabled within the flux harness. This is a
result of enabling "Optimize Aggregate Window" in flux@05a1065f.
These tests are not present in 2.x. Therefore, I am unsure if
the breakage is resolved in a later commit.

* feat: upgrade flux to 0.177.0

* feat: upgrade flux to 0.178.0

* feat: upgrade flux to v0.179.0

This removes all invocations of "flux.RegisterOpSpec". According
to flux@e39096d5, "flux.RegisterOpSpec" does nothing in the
current version of flux and was removed.

* chore: update fluxtest skip list (#23633)

* chore: manually backport 785a465e9a

This removes the reference to "flux.Spec".

* build(flux): update flux to v0.181.0 (#23682)

* build(flux): update flux to v0.184.2

* chore: skip more Flux acceptance tests

There are issues for each skip detailed in test-flux.sh.

* feat: upgrade flux to v0.185.0

This adds "FluxTesting" to the "HTTPD" configuration. This option is
hidden and disabled by default. When "FluxTesting" is set, it
enables the default testing flags for "Flux".

These flags allow the "vectorized float tests" and tests requiring
the "removeRedundantSortNodes" and "labelPolymorphism" flag
enabled to work. These changes are based off of d8553c002e.

flux@3d6f47ded is included within this version of Flux. Therefore
we can now include the "group_*" tests.

* feat: upgrade flux to 0.186.0

* feat: upgrade flux to 0.187.0

* feat: upgrade flux to 0.188.0

* fix: re-run ./generate.sh with updated protoc

* fix: restrict cores to match CircleCI documentation

Co-authored-by: davidby-influx <dbyrne@influxdata.com>
Co-authored-by: Markus Westerlind <marwes91@gmail.com>
Co-authored-by: Sean Brickley <sean@wabr.io>
Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>

---------

Co-authored-by: Brandon Pfeifer <bpfeifer@influxdata.com>
Co-authored-by: davidby-influx <dbyrne@influxdata.com>
Co-authored-by: Markus Westerlind <marwes91@gmail.com>
Co-authored-by: Sean Brickley <sean@wabr.io>
Co-authored-by: Jonathan A. Sternberg <jonathan@influxdata.com>
Co-authored-by: Christopher M. Wolff <chris.wolff@influxdata.com>
2023-03-03 10:05:05 -05:00
Sam Arnold c240c7906e
fix: a few suddenly flaky tests involving randomness (#21818)
Closes #21817
2021-07-09 12:17:23 -04:00
Sam Arnold 903b8cd0ea
feat(query): Hyper log log operators in influxql (#20603)
* feat(query): hyper log log counting in query engine

In addition to helping with normal queries, this can improve the 'SHOW CARDINALITY'
meta-queries:

time influx -database mydb -execute 'select count_hll(sum_hll(_seriesKey)) from big'
name: big
time count_hll
---- ---------
0    200767781
influx -database mydb -execute   0.06s user 0.12s system 0% cpu 8:49.99 total
2021-02-08 08:38:14 -05:00
Ben Wells e9bada090f Fix misspelling identified by misspell 2019-02-03 20:27:43 +00:00
Jacob Marble 2dc2b97fb9
tsdb/index: Add Bytes() methods (#9794) 2018-05-04 08:47:05 -07:00
Jacob Marble 63f0eee472 HLL++: estimate memory footprint bytes 2018-04-24 09:28:25 -07:00
Mark Rushakoff 426a9a0b8b Use math/bits exclusively instead of go-bits
We won't be rolling back to pre-Go1.9, so prefer the standard library
over a dependency that provides backwards compatibility.
2018-03-15 12:03:24 -07:00
Edd Robinson e5c8fd9dc5 Ensure nil sketches never returned 2018-02-09 15:29:42 +00:00
Edd Robinson 0d164f3164
WIP - tsi integration sketches 2018-02-07 14:52:13 -07:00
Edd Robinson 544329380f
Add empty series sketches back to tsi1 index
This commit adds initial empty sketches back to the tsi1 index, as well
as ensuring that ephemeral sketches in the index `LogFile` are updated
accordingly.

The commit also adds a test that verifies that the merged sketches at
the store level produce the correct results under writes, deletions and
re-opening of the store.

This commit does not provide working sketches for post-compaction on the
tsi1 index.
2018-02-07 14:52:13 -07:00
Edd Robinson a174f65595 use math/bits in HLL implementation 2017-09-26 12:51:08 +01:00
Seif Lotfy 4cb01c1768 change beta constants for the hll cardinality bias estimator 2017-06-30 07:47:16 -07:00
Seif Lotfy 643b2eb30c Switch to LogLog-Beta Cardinality estimation
The new algorithm uses only one formula and needs no additional bias corrections for the entire range of cardinalities,
therefore, it is more efficient and simpler to implement. Our simulations show that the accuracy provided by the new
algorithm is as good as or better than the accuracy provided by either of HyperLogLog or HyperLogLog++. The sparse
representation was kept in to provide better low cardinality accuracy. However the linear counting and range estimations
are replaced.
2017-06-20 15:25:01 +02:00
Ben Johnson 623ff67221
Fix HLL variableLengthList size decoding. 2017-05-19 11:44:25 -06:00
Edd Robinson 1c4ecb12c1 Don't panic on nil engine 2017-03-22 10:07:29 -06:00
Stuart Carnie 0ebbfb8f77 hll: skip recalc of sparseSet if tmpSet is empty
```
benchmark                                 old ns/op     new ns/op     delta
BenchmarkSet_Count/set_size_1000-8        38095         28.3          -99.93%
BenchmarkSet_Count/set_size_5000-8        152052        30.1          -99.98%
BenchmarkSet_Count/set_size_10000-8       50953         54978         +7.90%
BenchmarkSet_Count/set_size_50000-8       32495         31222         -3.92%
BenchmarkSet_Count/set_size_1000000-8     32603         30800         -5.53%

benchmark                                 old allocs     new allocs     delta
BenchmarkSet_Count/set_size_1000-8        4              0              -100.00%
BenchmarkSet_Count/set_size_5000-8        4              0              -100.00%
BenchmarkSet_Count/set_size_10000-8       0              0              +0.00%
BenchmarkSet_Count/set_size_50000-8       0              0              +0.00%
BenchmarkSet_Count/set_size_1000000-8     0              0              +0.00%

benchmark                                 old bytes     new bytes     delta
BenchmarkSet_Count/set_size_1000-8        16496         0             -100.00%
BenchmarkSet_Count/set_size_5000-8        16497         0             -100.00%
BenchmarkSet_Count/set_size_10000-8       0             0             +0.00%
BenchmarkSet_Count/set_size_50000-8       0             0             +0.00%
BenchmarkSet_Count/set_size_1000000-8     0             0             +0.00%
```
2017-01-31 08:51:05 -07:00
Edd Robinson ab94c1b743 Fixes #7882 2017-01-30 19:12:24 +00:00
Edd Robinson 695adafc00
Add measurement sketches 2017-01-05 10:15:37 -07:00
Edd Robinson 1339c7b146
Initialise HLL with error 2017-01-05 10:15:37 -07:00
Ben Johnson cb93f10120
Remove per-shard in-memory index. 2017-01-05 10:11:09 -07:00
Edd Robinson e2c3b52ca4
Adds a custom HyperLogLog++ implementation 2017-01-05 10:00:14 -07:00