Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/storage/reads/store.go

89 lines
2.6 KiB
Go
Raw Normal View History

chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
package reads
import (
"context"
"github.com/gogo/protobuf/proto"
refactor: move v2/v1/models to v2/models
2020-07-28 22:59:11 +00:00
"github.com/influxdata/influxdb/v2/models"
refactor: rewrite imports to include the /v2 suffix for version 2
2020-04-03 17:39:20 +00:00
"github.com/influxdata/influxdb/v2/storage/reads/datatypes"
chore: move v2/v1/tsdb → v2/tsdb
2020-08-26 17:46:47 +00:00
"github.com/influxdata/influxdb/v2/tsdb/cursors"
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
)
type ResultSet interface {
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Next advances the ResultSet to the next cursor. It returns false
// when there are no more cursors.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Next() bool
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Cursor returns the most recent cursor after a call to Next.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Cursor() cursors.Cursor
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Tags returns the tags for the most recent cursor after a call to Next.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Tags() models.Tags
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Close releases any resources allocated by the ResultSet.
Close()
// Err returns the first error encountered by the ResultSet.
Err() error
Integrate scanned values statistics tracking.
2018-11-07 16:31:42 +00:00
Stats() cursors.CursorStats
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
}
type GroupResultSet interface {
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Next advances the GroupResultSet and returns the next GroupCursor. It
// returns nil if there are no more groups.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Next() GroupCursor
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Close releases any resources allocated by the GroupResultSet.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Close()
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Err returns the first error encountered by the GroupResultSet.
Err() error
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
}
type GroupCursor interface {
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Next advances to the next cursor. Next will return false when there are no
// more cursors in the current group.
Next() bool
// Cursor returns the most recent cursor after a call to Next.
Cursor() cursors.Cursor
// Tags returns the tags for the most recent cursor after a call to Next.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Tags() models.Tags
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Keys returns the union of all tag key names for all series produced by
// this GroupCursor.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Keys() [][]byte
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// PartitionKeyVals returns the values of all tags identified by the
// keys specified in ReadRequest#GroupKeys. The tag values values will
// appear in the same order as the GroupKeys.
//
// When the datatypes.GroupNone strategy is specified, PartitionKeyVals will
// be nil.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
PartitionKeyVals() [][]byte
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Close releases any resources allocated by the GroupCursor.
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
Close()
chore(reads): Document ResultSet and GroupResultSet APIs; add func Err * add option to Group sort to specify precedence of `nil` * improve unit tests for GroupResultSet implementation
2018-11-20 18:38:36 +00:00
// Err returns the first error encountered by the GroupCursor.
Err() error
Integrate scanned values statistics tracking.
2018-11-07 16:31:42 +00:00
Stats() cursors.CursorStats
fix: group resultset produces 1 table per group (#18968) This helps resolve #18098
2020-07-29 20:31:29 +00:00
Aggregate() *datatypes.Aggregate
chore: refactor reads service and make it consumable externally This pulls in the code that allows doing reads with flux into the platform repo, and removes extra.go. The reusable portion is under storage/reads, where the concrete implementation for one of the platform's engines is in storage/readservice. In order to make this more reusable, the cursors had to move into their own package, decoupling it from all of the other code in the tsdb package. tsdb/cursors is this new package, and type/function aliases have been added to the tsdb package to point at it. The models package already is very light on transitive dependencies and so it was allowed to be depended on in a concrete way in the cursors package. Finally, the protobuf definitions for issuing GRPC reads has been moved into its own package for two reasons: 1. It's a clean separation, and helps keep it that way. 2. Many/most consumers will not be using GRPC. We just use the datatypes to express the API which helps making a GRPC server easier. It is left up to future refactorings (specifically ones that involve GPRC) to determine if these types should remain, or if there is a cleaner way. There's still some dependencies on both github.com/influxdata/influxql and github.com/influxdata/influxdb/logger that we can hopefully remove in future refactorings.
2018-10-05 22:02:31 +00:00
}
type Store interface {
refactor(storage): ReadFilter storage operation
2019-04-02 21:03:25 +00:00
ReadFilter(ctx context.Context, req *datatypes.ReadFilterRequest) (ResultSet, error)
feat(storage): ReadGroup RPC definitions and storage reader
2019-04-24 23:18:33 +00:00
ReadGroup(ctx context.Context, req *datatypes.ReadGroupRequest) (GroupResultSet, error)
feat(storage): enable window agg mean pushdown
2020-10-22 21:34:22 +00:00
// WindowAggregate will invoke a ReadWindowAggregateRequest against the Store.
WindowAggregate(ctx context.Context, req *datatypes.ReadWindowAggregateRequest) (ResultSet, error)
feat(storage): ReadGroup RPC definitions and storage reader
2019-04-24 23:18:33 +00:00
chore: Move StringIterator to cursors package for wider reuse
2019-04-17 23:11:09 +00:00
TagKeys(ctx context.Context, req *datatypes.TagKeysRequest) (cursors.StringIterator, error)
TagValues(ctx context.Context, req *datatypes.TagValuesRequest) (cursors.StringIterator, error)
feat(storage): ReadGroup RPC definitions and storage reader
2019-04-24 23:18:33 +00:00
fix(storage): meta.Client persists to kv and updates via BucketService Includes initial migration to add metadata bucket
2020-07-30 01:08:04 +00:00
GetSource(orgID, bucketID uint64) proto.Message
feat(query): add interface for the window aggregate capability (#17801) Added an interface for an additional storage capability. This interface will allow for checking if the reader supports the window aggregate call and another method for invoking the call if it does. This is implemented using a single interface. If the reader implements the interface, it indicates that the client is capable of reading the response. The `HasXXX` method is intended to check if the store supports the operation. This method also takes a context because it could require a remote call or to wait for one.
2020-04-21 19:15:38 +00:00
}