Commit Graph

14909 Commits (db/wait-timeout-utility)

Author SHA1 Message Date
Ayan George 3b9be0145c
fix: address static check warning s1039 (#18135)
This commit quiets staticcheck's warnings about "unnecessary use of
fmt.Sprintf" and "unnecessary use of fmt.Sprint".

Prior to this commit we were wrapping simple constant strings without
any formatting verbs with fmt.Sprintf().
2020-05-18 13:55:05 -04:00
Ayan George c0e391935d
fix(tsdb): address staticcheck warning SA4006 (#18127)
This commit addresses staticcheck warning SA4006 "this value of VAR is
never used" by removing unused variables
2020-05-18 13:11:44 -04:00
Ayan George 1a22af7172
fix(tsdb): address staticcheck warning st1006 (#18126)
* fix(tsdb): address staticcheck ST1006

This patch addresses staticcheck warning "receiver name should not be an
underscore, omit the name if it is unused (ST1006)" for 6 methods.
2020-05-17 22:36:06 -04:00
Ayan George 4cbc6a4269
fix(tsdb): Fix variables masked by a declaration (#18129)
Before this commit, the to and from variables were being re-declared in
a block in such a way that the values were not being used.

This patch uses regular assignment so that the values are visable
outside of the block where they're set.

Closes: 18128
2020-05-17 21:40:12 -04:00
Tristan Su d14acea44d chore: clean up unused functions 2020-05-08 13:45:34 +08:00
Stuart Carnie 0cb09dd09f
Merge pull request #18004 from influxdata/sgc/fix/unique_keys
fix(httpd): Fixes key collisions when serializing /debug/vars
2020-05-07 12:38:56 -07:00
Stuart Carnie 8843a01ebf
fix(httpd): Fixes key collisions when serializing /debug/vars 2020-05-07 11:49:59 -07:00
Ayan George 96bbe2e7af
fix(docs): Update docs to reflect tooling changes (#17798)
Update README.md and CONTRIBUTING.md to reflect our use of go 1.13 (and later) and modules.
2020-04-20 17:49:19 +00:00
Tristan Su af8e66cd25 improvement(query): performance improvement for sorted merge iterator
Sorted merge iterator has cpu-intensive operations to sort the points
from multiple inputs. Typical queries like `SELECT * FROM m GROUP BY *`
do not behave well due to the comparison of points though in many cases
it doesn't necessarily have to use the slow path.

This patch adds a shortcut.  If each input has a single and unique
series we can just return the points input by input.
The detection of the shortcut introduces slight overhead but the gains
are significant in many slow queries.
2020-04-20 21:06:45 +08:00
Ayan George a0f2e0c21a
fix(tsm1): Fix temp directory search bug (#17685)
* fix: verify precision parameter in write requests

This change updates the HTTP endpoints that service v1 and v2 writes to
verify the values passed in the precision parameter.

* fix(tsm1): Fix temp directory search bug

The original code's intention is to scan a directory for the directory
with the higest value when converted to an integer.

So directories may be in the form:

  0.tmp
  1.tmp
  2.tmp
  30.tmp
  ...
  100.tmp

The loop should scan the directory, strip the basename and extension
from the file name to leave just a number, then store the higest number
it finds.

Before this patch, there is a bug that has the code only store the
higest value if there is an error converting the numeric value into an
integer.

This patch primarily fixes that logic.

In addition, this patch will save an indent level by inverting logic in
two places:

Instead if checkig if a file is a directory and has a suffix of ".tmp",
it is probably better to test if a file is NOT a directory OR does NOT
have an extension of ".tmp" then continue.

Also, instead of testig if len(ss) == 2, we can test if len(ss) != 2 and
continue if so.

Both of these save an indent level and keeps our "happy path" to the
left.

Finally, this patch will use string concatination instead of calling
fmt.Sprintf() to add periods to "tmp" and "tsm" extension.

Co-authored-by: David Norton <dgnorton@gmail.com>
2020-04-15 10:29:46 -04:00
Ayan George f51709f09c
refactor(http): Simplify Authorizer (#17704)
Have AuthorizerIsOpen() assert if a given authizer has an
AuthorizeUnrestricted() method and if so, call that to provide the
result of AuthorizerIsOpen().

Otherwise we check if the supplied Authorizer is nil.

This preserves the fast-path for checking tag-level (and other) tsdb
operations.

This simplifies how we handle such authorizers by handling this case in
only one place.
2020-04-15 12:59:52 +00:00
Ben Johnson 5c8b2a9f8e
Merge pull request #17722 from influxdata/1.x-revert-unsafe-roaring-bitmap-marshaling
fix(tsdb): Revert "fix: remove some unsafe marshalling to reduce risk…
2020-04-13 15:25:39 -06:00
Ben Johnson 848ca48225 fix(tsdb): Revert "fix: remove some unsafe marshalling to reduce risk of segfault"
This reverts commit 30dab03310.
2020-04-13 13:22:22 -06:00
David Norton f78b2aa24b
Merge pull request #17638 from influxdata/dn-fix-v2-write-precision
fix: verify precision parameter in write requests
2020-04-07 12:32:57 -04:00
Faith Chikwekwe a02d36d9ae
Merge pull request #17636 from influxdata/fix/sort-key-on-master-1.x
fix(storage/reads): update sortKey sorting method to use null byte as delimiter, not comma
2020-04-07 09:02:11 -07:00
David Norton 129a78ea08 fix: verify precision parameter in write requests
This change updates the HTTP endpoints that service v1 and v2 writes to
verify the values passed in the precision parameter.
2020-04-07 11:29:53 -04:00
Faith Chikwekwe 6e971e5782 fix(storage/reads): update sortKey sorting method to use null byte as delimeter 2020-04-06 17:05:07 -07:00
Jonathan A. Sternberg ac1e55ec37
build(flux): update Flux to v0.65.0 (#17619) 2020-04-06 10:20:24 -05:00
Mustafa d0868f9901
Merge pull request #17532 from influxdata/elbehery-fix#17440
fix(tsdb): Replace panic with error while de/encoding corrupt data
2020-04-02 13:17:27 +02:00
Ben Johnson 3f59d4be8e fix(tsdb): Fix error lists 2020-04-01 14:54:39 -06:00
elbehery 042128b948 fix(tsdb): Replace panic with error while de/encoding corrupt data
fixes #17440

While encoding or decoding corrupt data, the current behaviour is to `panic`.
This commit replaces the `panic` with `error` to be propagated up to the calling `iterator`.
To avoid overwriting other `error`, iterators now wraps a `TSMErrors` which contains ALL the encountered errors.
TSMErrors itself implements `Error()`, the returned string contains all the error msgs, separated by "," delimiter.
2020-04-01 20:51:11 +02:00
Ayan George d3d372fd5d
fix(go.mod): set minimum go version in go.mod file to 1.13 (#17530)
We neglected to update the go version to 1.13 when we converted to
modules.

We plan on using 1.13 features like error wrapping so it is important
that this is correct for 1.8.x.
2020-04-01 14:03:35 -04:00
Ayan George 2f1344ffd7
fix(snapshotter): properly read payload (#17495)
This is a fix and a bit of a refactor of Service.readRequest().

This commit changes the signature so that it accepts an io.Reader
instead of a net.Conn because there's nothing about the function that
relies on it being a Conn; only a Reader.

We also changed the signature so that it returns a *Request instead of
Request -- this way we don't allocate an entire Request value on error.

This commit also attempts to document the few edge cases that the function has to
handle and one quirk where we expect to read a newline which isn't
accounted for by Request.UploadSize.
2020-03-31 12:19:25 -04:00
Hercules Mango Churchill 2e371db2a0 Update changelog 2020-03-16 17:08:16 +00:00
David Norton 137830c0ce
Merge pull request #17274 from influxdata/dn-update-changelog
chore: update CHANGELOG.md
2020-03-16 13:07:54 -04:00
Jonathan A. Sternberg 7cc8eb4986
ci: use go mod vendor instead of dep ensure for constructing vendor (#17278) 2020-03-15 10:56:14 -07:00
David Norton 7ea8fdb566 chore: update CHANGELOG.md 2020-03-13 21:55:04 -04:00
Ayan George cc675adec6
fix: Ensure proper go versions (#17230) 2020-03-13 09:48:36 -04:00
Jonathan A. Sternberg 409de34abf
Merge pull request #14485 from jsign/1.8_accept
fix: services/httpd: parse correctly Accept header with extra test cases
2020-03-12 14:26:12 -05:00
jsign 69519243fe
fix: services/httpd: parse correctly Accept header with extra test case
Signed-off-by: jsign <jsign.uy@gmail.com>
2020-03-12 14:01:49 -05:00
Jonathan A. Sternberg 78bb637ae3
Merge pull request #17188 from influxdata/feat/bound-param-identifiers
feat: add support for complex bound parameters
2020-03-12 13:26:30 -05:00
Jonathan A. Sternberg 8d3496f0a3
feat: add support for complex bound parameters
This updates influxql to a newer version that supports complex bound
parameters. This allows bound parameters to be used as identifiers,
regexes, and durations along with the already existing strings, numbers,
and booleans.

This updates the influxdb client to support passing bound parameters as
part of the parameters json and updates the readme to show how to use
this feature.
2020-03-12 12:26:22 -05:00
Jonathan A. Sternberg 36dc105be1
Merge pull request #17187 from influxdata/flux-backport/v0.63.x
build(flux): update Flux to v0.64.0
2020-03-12 12:19:12 -05:00
Jonathan A. Sternberg 636a27e77f
build(flux): update Flux to v0.64.0 2020-03-11 15:17:44 -05:00
David Norton 25381f97c8
Merge pull request #15952 from influxdata/er-verify-tombstone
feat(inspect): add influx_inspect verify-tombstone tool
2020-03-11 15:56:37 -04:00
David Norton fca2ee03a7 fix: update package comment 2020-03-11 15:37:02 -04:00
David Norton 0fcb240ebb fix: use highest verbosity level specified 2020-03-11 15:32:34 -04:00
David Norton b93bac197c fix: typo in error message 2020-03-11 15:28:45 -04:00
David Norton 5eb92ef1eb
Merge pull request #17204 from influxdata/dn-fwd-fix-for-unsafe-marshalling-1.8
fix: remove some unsafe marshalling to reduce risk of segfault
2020-03-11 14:10:12 -04:00
docmerlin (j. Emrys Landivar) 30dab03310 fix: remove some unsafe marshalling to reduce risk of segfault
We were seing segfaults in Roaring bitmaps sometimes, under very
high load with networked drives.  This may reduce risk of segfault by
forcing marshalling to copy the data.
2020-03-11 13:47:29 -04:00
Ayan George b1ea8eff3e
refactor(query): save an indent level when checking r.prev.Nil (#17125) 2020-03-06 13:48:30 -05:00
Ayan George f24bdb3ee5
feat(handler): Add 2.0 compatible write endpoint (#16908)
This commit adds a /api/v2/write endpoint that maps the supplied bucket
and org to a v1 database and retention policy.

* Add AllowedOrgs to httpd Config type.

* Add /api/v2/write handler
2020-03-06 10:03:25 -05:00
Ayan George 5f47c388df
chore(influxdb): Forward port 16999 (#17032)
* fix: access tsi active log file with READ lock

The activeLogFile pointer may be altered by other routine so the READ
lock is needed.

* Merge pull request #16384 from foobar/tsi-partition-lock

fix: access tsi active log file with READ lock

Co-authored-by: Tristan Su <sooqing@gmail.com>
Co-authored-by: David Norton <dgnorton@gmail.com>
2020-03-04 16:20:58 -05:00
Edd Robinson c3f4382ed8
Merge pull request #16606 from influxdata/BP-1.8-er-tsm-block-fix
fix(storage): ensure all block data returned
2020-03-03 14:32:15 +00:00
Stuart Carnie f71caf5c02
Merge pull request #16709 from influxdata/sgc/18/flux-cost
fix(flux): Reduce influxd and influx startup time if Flux unused
2020-03-02 15:16:13 -07:00
Ayan George 35c07d0127
feat(modules): Use modules & remove Gopkg.*
* Add .mod and .sum files generated by go mod init

* Use a directory outside of $GOPATH for testing.

* Update docker files to use go 1.13.8 instead of 1.12 for most builds.

* Simplify go installation by piping from wget to tar.  This avoids
  having to do any cleanup.

* Build and test in a directory outside of GOPATH since we're using
  modules now.

* Update Jenkinsfile to build using modules by building directly inside
  of $WORKSPACE instead of in $GOPATH.

* Use go mod download instead of dep ensure to collect dependencies

* Remove Gopkg related data files.

* Replace calls to "dep ensure" with "go mod download"

* Replace check for being outside of $GOPATH to a check for being INSIDE
  of $GOPATH which is wrong when we're using modules

* Map $DIR to /root/influxdb in docker run command

* Clone source into /root/influxdb which is outside of our $GOPATH.

* run go mod tidy
2020-02-26 17:27:56 -05:00
Ben Johnson 6b07ed7200
Merge pull request #16762 from influxdata/fix-compact-series-file
fix(tsdb): Fix -compact-series-file flag
2020-02-10 06:42:20 -07:00
Sebastian Borza 2ba35b091b
Merge pull request #16525 from rickif/fix/meta-index
fix(meta): remove double index increment of set meta
2020-02-06 16:45:26 -06:00
Ben Johnson 7a9eb1420c fix(tsdb): Fix -compact-series-file flag 2020-02-06 13:40:19 -07:00
Hercules Mango Churchill a8cf52ba0e Update changelog 2020-02-04 20:45:16 +00:00