Commit Graph

29 Commits (8b8733d171a9882560faa0e788c94aac3f33f71a)

Author SHA1 Message Date
Jacob Marble f33e13a036 storage: Add MeasurementCardinalityStats and MeasurementStats to Engine 2018-10-18 11:10:21 -07:00
Adam ea4c08b327
(feat/query): implement a buckets() source function that retrieves all buckets and retention policy information for the active organization. (#1045) 2018-10-17 10:21:39 -04:00
Mark Rushakoff d479c330fe fix(storage): avoid data race on table Done channel
Fixes #1021.
2018-10-16 08:21:13 -07:00
Jeff Wendling 30418ca2f5 fix(storage): WithEngineID and WithNodeID no longer panic
Both of these options would dereference a nil pointer when attempting
to apply. Instead, set the field to be the address of an integer
containing the right value.
2018-10-15 12:44:46 -06:00
Jonathan A. Sternberg 5c8318d181
Merge pull request #1080 from influxdata/js-go-generate-with-go-run
refactor: modify tooling to take advantage of go run
2018-10-15 11:27:13 -05:00
Jeff Wendling ce3e1fa90d Remove the fields index
This commit removes the remaining bits of the fields index. In doing
so, the buildCursor method on the engine would need to be updated.
It turns out, that code was statically dead, so delete it and anything
that depended on it. Additionally, delete anything as reported by
the unused tool in the tsdb package.
2018-10-15 10:22:07 -06:00
Jonathan A. Sternberg e9600b1f0b
refactor: modify tooling to take advantage of go run
The generate commands have been modified to take advantage of the new
functionality in Go 1.11 that allows `go run` to execute a package
instead of individual files.

This functionality combined with Go modules allows us to execute a
package directly out of our pinned dependencies rather than accidentally
picking up another binary outside of the build environment.

This also simplifies the Makefile because they no longer have to be
responsible for installing the correct tooling since the Go command
takes care of that logic. It also makes it so that the Makefiles with
file generation can now be invoked from their appropriate subdirectories
so they are contained within the directory itself rather than relying on
values in the top level Makefile.

It is now possible to generate all files within this project by using:

    go generate ./...

Or the Makefile can continue to be used.

This commit also copies over the special copy of `tmpl` that the storage
engine uses within the influxdb repository. It was never copied over so
using `go generate` on these packages did not work.
2018-10-15 11:01:27 -05:00
Stuart Carnie 025241629d fix(reads): Ensure error returned from f is captured 2018-10-11 13:41:54 -07:00
Chris Goller 9298a761de fix(storage/retention): change from string to platform.ID
Co-Authored-by: Jeff Wendling <jeff@influxdata.com>
2018-10-11 09:54:19 -05:00
Chris Goller 919aae41c8 refactor(id): update storage, read, and flux from to use IDs 2018-10-11 09:54:19 -05:00
Jonathan A. Sternberg e7e17d6972
refactor: move the logger package from influxdb to platform
We are moving the necessary code for 2.0 from the influxdb 1.X
repository to the platform 2.0 repository. The logger is an unnecessary
dependency on the old influxdb that is making life more complicated.
2018-10-10 15:49:07 -05:00
Edd Robinson f815dcfeb7 Schedule retention enforcer from within engine 2018-10-10 15:45:26 +01:00
Edd Robinson d5ed17adb2 Rename retention service 2018-10-10 15:40:08 +01:00
Edd Robinson 1f2f03d93b Open/close Retention properly 2018-10-10 15:40:08 +01:00
Edd Robinson 031bb7f502 Add prom metrics 2018-10-10 15:40:08 +01:00
Edd Robinson 4597fdea48 Wire Retention service into engine 2018-10-10 15:40:08 +01:00
Edd Robinson 54d1ac09c1 Integrate Retention metrics 2018-10-10 15:40:08 +01:00
Edd Robinson 116456534a Initial Retention Service 2018-10-10 15:39:27 +01:00
Adam 195eb04350
Chore/move from function to platform (#1013)
* moved execution code for from into query/functions/inputs

* rebase with master, disable buckets accessed test
2018-10-09 20:36:41 -04:00
Jeff Wendling 5fa04a8d56 share some more code with reads 2018-10-09 09:51:13 -06:00
Jeff Wendling 6256926563 fix generation and rerun it with the go.mod gogofaster 2018-10-09 09:51:13 -06:00
Jeff Wendling 810833f33f chore: refactor reads service and make it consumable externally
This pulls in the code that allows doing reads with flux into the
platform repo, and removes extra.go.

The reusable portion is under storage/reads, where the concrete
implementation for one of the platform's engines is in
storage/readservice.

In order to make this more reusable, the cursors had to move into
their own package, decoupling it from all of the other code in the
tsdb package. tsdb/cursors is this new package, and type/function
aliases have been added to the tsdb package to point at it.

The models package already is very light on transitive dependencies
and so it was allowed to be depended on in a concrete way in the
cursors package.

Finally, the protobuf definitions for issuing GRPC reads has been
moved into its own package for two reasons:
    1. It's a clean separation, and helps keep it that way.
    2. Many/most consumers will not be using GRPC. We just
       use the datatypes to express the API which helps making
       a GRPC server easier.
It is left up to future refactorings (specifically ones that involve
GPRC) to determine if these types should remain, or if there is a
cleaner way.

There's still some dependencies on both github.com/influxdata/influxql
and github.com/influxdata/influxdb/logger that we can hopefully remove
in future refactorings.
2018-10-09 09:51:13 -06:00
Jeff Wendling 0360fc418c Remove empty index package for tsi1 2018-10-08 13:18:30 -06:00
Jeff Wendling 260ed3eb13 chore: don't depend on details of what platform.ID is
It may become a uint64 in the future, for example. This does mean
that we have to call Decode on some data that we just Encoded, but
we can fix that later.
2018-10-08 12:31:39 -06:00
Edd Robinson 512a015add Address PR feedback 2018-10-05 18:01:59 +01:00
Edd Robinson 6b23165146 Add Engine tests 2018-10-05 15:04:09 +01:00
Edd Robinson 81e0fbabeb Integrate WAL into engine 2018-10-05 12:44:27 +01:00
Edd Robinson cfb3df203a Working storage engine 2018-10-04 10:24:43 +01:00
Edd Robinson 981b2cdbea Skeleton storage engine 2018-10-04 10:24:43 +01:00