Influxdata
/
influxdb
mirror of https://github.com/influxdata/influxdb.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
influxdb/query/call_iterator_test.go

1222 lines
52 KiB
Go
Raw Normal View History

Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
package query_test
refactor query engine
2015-11-04 21:06:06 +00:00
import (
"testing"
"time"
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
"github.com/google/go-cmp/cmp"
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
"github.com/influxdata/influxdb/query"
influxdata/influxdb/influxql -> influxdata/influxql
2017-10-30 21:40:26 +00:00
"github.com/influxdata/influxql"
refactor query engine
2015-11-04 21:06:06 +00:00
)
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
// Ensure that a float iterator can be created for a count() call.
refactor query engine
2015-11-04 21:06:06 +00:00
func TestCallIterator_Count_Float(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&FloatIterator{Points: []query.FloatPoint{
Fix reduce iterators to separate by name Previously reduce iterators just separated points by tags. If you had identical tags but different names, it would group those together so you could have these two points: cpu value=1 mem value=2 When you performed a `mean(value)` call and included both cpu and mem as sources, it would return one mem series with a value of 1.5 instead of two serieses.
2016-02-03 21:56:15 +00:00
{Name: "cpu", Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Name: "cpu", Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
refactor query engine
2015-11-04 21:06:06 +00:00
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Name: "cpu", Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
Fix reduce iterators to separate by name Previously reduce iterators just separated points by tags. If you had identical tags but different names, it would group those together so you could have these two points: cpu value=1 mem value=2 When you performed a `mean(value)` call and included both cpu and mem as sources, it would return one mem series with a value of 1.5 instead of two serieses.
2016-02-03 21:56:15 +00:00
{Name: "cpu", Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Name: "mem", Time: 23, Value: 10, Tags: ParseTags("region=us-west,host=hostB")},
refactor query engine
2015-11-04 21:06:06 +00:00
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
refactor query engine
2015-11-04 21:06:06 +00:00
Expr: MustParseExpr(`count("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
refactor query engine
2015-11-04 21:06:06 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 3, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Name: "cpu", Time: 5, Value: 1, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "mem", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
}
}
// Ensure that an integer iterator can be created for a count() call.
func TestCallIterator_Count_Integer(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&IntegerIterator{Points: []query.IntegerPoint{
Fix reduce iterators to separate by name Previously reduce iterators just separated points by tags. If you had identical tags but different names, it would group those together so you could have these two points: cpu value=1 mem value=2 When you performed a `mean(value)` call and included both cpu and mem as sources, it would return one mem series with a value of 1.5 instead of two serieses.
2016-02-03 21:56:15 +00:00
{Name: "cpu", Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Name: "cpu", Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Name: "cpu", Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
Fix reduce iterators to separate by name Previously reduce iterators just separated points by tags. If you had identical tags but different names, it would group those together so you could have these two points: cpu value=1 mem value=2 When you performed a `mean(value)` call and included both cpu and mem as sources, it would return one mem series with a value of 1.5 instead of two serieses.
2016-02-03 21:56:15 +00:00
{Name: "cpu", Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Name: "mem", Time: 23, Value: 10, Tags: ParseTags("region=us-west,host=hostB")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
Expr: MustParseExpr(`count("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 3, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Name: "cpu", Time: 5, Value: 1, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "mem", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
refactor query engine
2015-11-04 21:06:06 +00:00
}
}
Add unsigned iterators for all types This allows unsigned data to be queried from the storage engine. Binary math is not yet implemented for unsigned types.
2017-09-18 16:28:37 +00:00
// Ensure that an unsigned iterator can be created for a count() call.
func TestCallIterator_Count_Unsigned(t *testing.T) {
itr, _ := query.NewCallIterator(
&UnsignedIterator{Points: []query.UnsignedPoint{
{Name: "cpu", Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Name: "cpu", Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
{Name: "cpu", Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Name: "mem", Time: 23, Value: 10, Tags: ParseTags("region=us-west,host=hostB")},
}},
query.IteratorOptions{
Expr: MustParseExpr(`count("value")`),
Dimensions: []string{"host"},
Interval: query.Interval{Duration: 5 * time.Nanosecond},
Ordered: true,
Ascending: true,
},
)
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
t.Fatalf("unexpected error: %s", err)
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 3, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Name: "cpu", Time: 5, Value: 1, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "mem", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
}
}
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
// Ensure that a string iterator can be created for a count() call.
func TestCallIterator_Count_String(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&StringIterator{Points: []query.StringPoint{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Name: "cpu", Time: 0, Value: "d", Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 2, Value: "b", Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 1, Value: "b", Tags: ParseTags("region=us-west,host=hostA")},
{Name: "cpu", Time: 5, Value: "e", Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Name: "cpu", Time: 1, Value: "c", Tags: ParseTags("region=us-west,host=hostB")},
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Name: "cpu", Time: 23, Value: "a", Tags: ParseTags("region=us-west,host=hostB")},
{Name: "mem", Time: 23, Value: "b", Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
Expr: MustParseExpr(`count("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 3, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Name: "cpu", Time: 5, Value: 1, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "mem", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
}
}
// Ensure that a boolean iterator can be created for a count() call.
func TestCallIterator_Count_Boolean(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&BooleanIterator{Points: []query.BooleanPoint{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Name: "cpu", Time: 0, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 2, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Name: "cpu", Time: 1, Value: true, Tags: ParseTags("region=us-west,host=hostA")},
{Name: "cpu", Time: 5, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Name: "cpu", Time: 1, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Name: "cpu", Time: 23, Value: false, Tags: ParseTags("region=us-west,host=hostB")},
{Name: "mem", Time: 23, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
Expr: MustParseExpr(`count("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 3, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Name: "cpu", Time: 5, Value: 1, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 0, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "cpu", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Name: "mem", Time: 20, Value: 1, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
}
}
refactor query engine
2015-11-04 21:06:06 +00:00
// Ensure that a float iterator can be created for a min() call.
func TestCallIterator_Min_Float(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&FloatIterator{Points: []query.FloatPoint{
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 4, Value: 12, Tags: ParseTags("region=us-east,host=hostA")},
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
refactor query engine
2015-11-04 21:06:06 +00:00
Expr: MustParseExpr(`min("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
refactor query engine
2015-11-04 21:06:06 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.FloatPoint{Time: 1, Value: 10, Tags: ParseTags("host=hostA"), Aggregated: 4}},
{&query.FloatPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.FloatPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.FloatPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
}
}
// Ensure that a integer iterator can be created for a min() call.
func TestCallIterator_Min_Integer(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&IntegerIterator{Points: []query.IntegerPoint{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 4, Value: 12, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
Expr: MustParseExpr(`min("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Time: 1, Value: 10, Tags: ParseTags("host=hostA"), Aggregated: 4}},
{&query.IntegerPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
refactor query engine
2015-11-04 21:06:06 +00:00
}
}
Add unsigned iterators for all types This allows unsigned data to be queried from the storage engine. Binary math is not yet implemented for unsigned types.
2017-09-18 16:28:37 +00:00
// Ensure that a unsigned iterator can be created for a min() call.
func TestCallIterator_Min_Unsigned(t *testing.T) {
itr, _ := query.NewCallIterator(
&UnsignedIterator{Points: []query.UnsignedPoint{
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 4, Value: 12, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
query.IteratorOptions{
Expr: MustParseExpr(`min("value")`),
Dimensions: []string{"host"},
Interval: query.Interval{Duration: 5 * time.Nanosecond},
Ordered: true,
Ascending: true,
},
)
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
t.Fatalf("unexpected error: %s", err)
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.UnsignedPoint{Time: 1, Value: 10, Tags: ParseTags("host=hostA"), Aggregated: 4}},
{&query.UnsignedPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
}
}
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
// Ensure that a boolean iterator can be created for a min() call.
func TestCallIterator_Min_Boolean(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&BooleanIterator{Points: []query.BooleanPoint{
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
{Time: 0, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: true, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: false, Tags: ParseTags("region=us-west,host=hostB")},
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
{Time: 23, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
Expr: MustParseExpr(`min("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.BooleanPoint{Time: 2, Value: false, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.BooleanPoint{Time: 5, Value: false, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.BooleanPoint{Time: 1, Value: false, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.BooleanPoint{Time: 23, Value: true, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
}
}
refactor query engine
2015-11-04 21:06:06 +00:00
// Ensure that a float iterator can be created for a max() call.
func TestCallIterator_Max_Float(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&FloatIterator{Points: []query.FloatPoint{
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
refactor query engine
2015-11-04 21:06:06 +00:00
Expr: MustParseExpr(`max("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
refactor query engine
2015-11-04 21:06:06 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.FloatPoint{Time: 0, Value: 15, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.FloatPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.FloatPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.FloatPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
}
}
// Ensure that a integer iterator can be created for a max() call.
func TestCallIterator_Max_Integer(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&IntegerIterator{Points: []query.IntegerPoint{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
Expr: MustParseExpr(`max("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Time: 0, Value: 15, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
refactor query engine
2015-11-04 21:06:06 +00:00
}
}
Add unsigned iterators for all types This allows unsigned data to be queried from the storage engine. Binary math is not yet implemented for unsigned types.
2017-09-18 16:28:37 +00:00
// Ensure that a unsigned iterator can be created for a max() call.
func TestCallIterator_Max_Unsigned(t *testing.T) {
itr, _ := query.NewCallIterator(
&UnsignedIterator{Points: []query.UnsignedPoint{
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
query.IteratorOptions{
Expr: MustParseExpr(`max("value")`),
Dimensions: []string{"host"},
Interval: query.Interval{Duration: 5 * time.Nanosecond},
Ordered: true,
Ascending: true,
},
)
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
t.Fatalf("unexpected error: %s", err)
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.UnsignedPoint{Time: 0, Value: 15, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.UnsignedPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
}
}
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
// Ensure that a boolean iterator can be created for a max() call.
func TestCallIterator_Max_Boolean(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&BooleanIterator{Points: []query.BooleanPoint{
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
{Time: 0, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: true, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: false, Tags: ParseTags("region=us-west,host=hostB")},
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
{Time: 23, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
Expr: MustParseExpr(`max("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.BooleanPoint{Time: 0, Value: true, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.BooleanPoint{Time: 5, Value: false, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.BooleanPoint{Time: 1, Value: false, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.BooleanPoint{Time: 23, Value: true, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support booleans for min() and max() Fixes #6494.
2016-04-29 18:56:22 +00:00
}
}
refactor query engine
2015-11-04 21:06:06 +00:00
// Ensure that a float iterator can be created for a sum() call.
func TestCallIterator_Sum_Float(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&FloatIterator{Points: []query.FloatPoint{
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
refactor query engine
2015-11-04 21:06:06 +00:00
Expr: MustParseExpr(`sum("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
refactor query engine
2015-11-04 21:06:06 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.FloatPoint{Time: 0, Value: 35, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.FloatPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.FloatPoint{Time: 0, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.FloatPoint{Time: 20, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
}
}
// Ensure that an integer iterator can be created for a sum() call.
func TestCallIterator_Sum_Integer(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&IntegerIterator{Points: []query.IntegerPoint{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
Expr: MustParseExpr(`sum("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Time: 0, Value: 35, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Time: 0, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Time: 20, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
refactor query engine
2015-11-04 21:06:06 +00:00
}
}
Add unsigned iterators for all types This allows unsigned data to be queried from the storage engine. Binary math is not yet implemented for unsigned types.
2017-09-18 16:28:37 +00:00
// Ensure that an unsigned iterator can be created for a sum() call.
func TestCallIterator_Sum_Unsigned(t *testing.T) {
itr, _ := query.NewCallIterator(
&UnsignedIterator{Points: []query.UnsignedPoint{
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 5, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
query.IteratorOptions{
Expr: MustParseExpr(`sum("value")`),
Dimensions: []string{"host"},
Interval: query.Interval{Duration: 5 * time.Nanosecond},
Ordered: true,
Ascending: true,
},
)
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
t.Fatalf("unexpected error: %s", err)
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.UnsignedPoint{Time: 0, Value: 35, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.UnsignedPoint{Time: 5, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 0, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 20, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
}
}
refactor query engine
2015-11-04 21:06:06 +00:00
// Ensure that a float iterator can be created for a first() call.
func TestCallIterator_First_Float(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&FloatIterator{Points: []query.FloatPoint{
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
refactor query engine
2015-11-04 21:06:06 +00:00
Expr: MustParseExpr(`first("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
refactor query engine
2015-11-04 21:06:06 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.FloatPoint{Time: 0, Value: 15, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.FloatPoint{Time: 6, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.FloatPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.FloatPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
}
}
// Ensure that an integer iterator can be created for a first() call.
func TestCallIterator_First_Integer(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&IntegerIterator{Points: []query.IntegerPoint{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
Expr: MustParseExpr(`first("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Time: 0, Value: 15, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Time: 6, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
}
}
Add unsigned iterators for all types This allows unsigned data to be queried from the storage engine. Binary math is not yet implemented for unsigned types.
2017-09-18 16:28:37 +00:00
// Ensure that an unsigned iterator can be created for a first() call.
func TestCallIterator_First_Unsigned(t *testing.T) {
itr, _ := query.NewCallIterator(
&UnsignedIterator{Points: []query.UnsignedPoint{
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
query.IteratorOptions{
Expr: MustParseExpr(`first("value")`),
Dimensions: []string{"host"},
Interval: query.Interval{Duration: 5 * time.Nanosecond},
Ordered: true,
Ascending: true,
},
)
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
t.Fatalf("unexpected error: %s", err)
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.UnsignedPoint{Time: 0, Value: 15, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.UnsignedPoint{Time: 6, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
}
}
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
// Ensure that a string iterator can be created for a first() call.
func TestCallIterator_First_String(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&StringIterator{Points: []query.StringPoint{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 2, Value: "b", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: "d", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: "b", Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: "e", Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: "c", Tags: ParseTags("region=us-west,host=hostB")},
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 23, Value: "a", Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
Expr: MustParseExpr(`first("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.StringPoint{Time: 0, Value: "d", Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.StringPoint{Time: 6, Value: "e", Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.StringPoint{Time: 1, Value: "c", Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.StringPoint{Time: 23, Value: "a", Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
}
}
// Ensure that a boolean iterator can be created for a first() call.
func TestCallIterator_First_Boolean(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&BooleanIterator{Points: []query.BooleanPoint{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 2, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: false, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 23, Value: false, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
Expr: MustParseExpr(`first("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.BooleanPoint{Time: 0, Value: true, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.BooleanPoint{Time: 6, Value: false, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.BooleanPoint{Time: 1, Value: true, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.BooleanPoint{Time: 23, Value: false, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
refactor query engine
2015-11-04 21:06:06 +00:00
}
}
// Ensure that a float iterator can be created for a last() call.
func TestCallIterator_Last_Float(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&FloatIterator{Points: []query.FloatPoint{
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
refactor query engine
2015-11-04 21:06:06 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
refactor query engine
2015-11-04 21:06:06 +00:00
Expr: MustParseExpr(`last("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
refactor query engine
2015-11-04 21:06:06 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.FloatPoint{Time: 2, Value: 10, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.FloatPoint{Time: 6, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.FloatPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.FloatPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
}
}
// Ensure that an integer iterator can be created for a last() call.
func TestCallIterator_Last_Integer(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&IntegerIterator{Points: []query.IntegerPoint{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
Expr: MustParseExpr(`last("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Fixed a bug in first() and last() where the time was lost last() would always return the last output of the iterator (which isn't necessarily the last time value due to how the merge iterator works) and first() would always return the first output of the iterator (wrong for the same reason). Now the time is kept by the reduce function and the times are wiped as part of the reduce iterator after the value has been found.
2016-01-23 03:04:10 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Time: 2, Value: 10, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.IntegerPoint{Time: 6, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.IntegerPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.IntegerPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
}
}
Add unsigned iterators for all types This allows unsigned data to be queried from the storage engine. Binary math is not yet implemented for unsigned types.
2017-09-18 16:28:37 +00:00
// Ensure that an unsigned iterator can be created for a last() call.
func TestCallIterator_Last_Unsigned(t *testing.T) {
itr, _ := query.NewCallIterator(
&UnsignedIterator{Points: []query.UnsignedPoint{
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
}},
query.IteratorOptions{
Expr: MustParseExpr(`last("value")`),
Dimensions: []string{"host"},
Interval: query.Interval{Duration: 5 * time.Nanosecond},
Ordered: true,
Ascending: true,
},
)
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
t.Fatalf("unexpected error: %s", err)
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.UnsignedPoint{Time: 2, Value: 10, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.UnsignedPoint{Time: 6, Value: 20, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.UnsignedPoint{Time: 23, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 1}},
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
}
}
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
// Ensure that a string iterator can be created for a last() call.
func TestCallIterator_Last_String(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&StringIterator{Points: []query.StringPoint{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 2, Value: "b", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: "d", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: "b", Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: "e", Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: "c", Tags: ParseTags("region=us-west,host=hostB")},
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 23, Value: "a", Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
Expr: MustParseExpr(`last("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.StringPoint{Time: 2, Value: "b", Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.StringPoint{Time: 6, Value: "e", Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.StringPoint{Time: 1, Value: "c", Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.StringPoint{Time: 23, Value: "a", Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
}
}
// Ensure that a boolean iterator can be created for a last() call.
func TestCallIterator_Last_Boolean(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewCallIterator(
&BooleanIterator{Points: []query.BooleanPoint{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 2, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 0, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: false, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 6, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
{Time: 23, Value: false, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
Expr: MustParseExpr(`last("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
Support all iterator types for count(), first(), and last() All three of these iterators are supposed to support all four types of iterators, but the implementation was never done for string or boolean. Fixes #5886.
2016-03-02 23:42:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
Modify all of the iterators to allow returning an error on Next() This also switches the remaining iterators to be lazy so they can return errors properly. They needed to be converted to lazy initialization anyway, which has the side effect of making it much easier for us to propagate the underlying error during initialization. Updated the Emitter to return errors when it cannot read properly from the iterators.
2016-04-17 20:00:59 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.BooleanPoint{Time: 2, Value: false, Tags: ParseTags("host=hostA"), Aggregated: 3}},
{&query.BooleanPoint{Time: 6, Value: false, Tags: ParseTags("host=hostA"), Aggregated: 1}},
{&query.BooleanPoint{Time: 1, Value: true, Tags: ParseTags("host=hostB"), Aggregated: 1}},
{&query.BooleanPoint{Time: 23, Value: false, Tags: ParseTags("host=hostB"), Aggregated: 1}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
refactor query engine
2015-11-04 21:06:06 +00:00
}
}
add mode() function & tests
2016-08-03 10:13:00 +00:00
// Ensure that a float iterator can be created for a mode() call.
func TestCallIterator_Mode_Float(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewModeIterator(&FloatIterator{Points: []query.FloatPoint{
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 3, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 4, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 7, Value: 21, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 8, Value: 21, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 22, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 24, Value: 25, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
add mode() function & tests
2016-08-03 10:13:00 +00:00
Expr: MustParseExpr(`mode("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
add mode() function & tests
2016-08-03 10:13:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
add mode() function & tests
2016-08-03 10:13:00 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.FloatPoint{Time: 0, Value: 10, Tags: ParseTags("host=hostA"), Aggregated: 0}},
{&query.FloatPoint{Time: 5, Value: 21, Tags: ParseTags("host=hostA"), Aggregated: 0}},
{&query.FloatPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB"), Aggregated: 0}},
{&query.FloatPoint{Time: 20, Value: 8, Tags: ParseTags("host=hostB"), Aggregated: 0}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
add mode() function & tests
2016-08-03 10:13:00 +00:00
}
}
// Ensure that a integer iterator can be created for a mode() call.
func TestCallIterator_Mode_Integer(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewModeIterator(&IntegerIterator{Points: []query.IntegerPoint{
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 3, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 4, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 7, Value: 21, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 8, Value: 21, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 22, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 24, Value: 25, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
add mode() function & tests
2016-08-03 10:13:00 +00:00
Expr: MustParseExpr(`mode("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
add mode() function & tests
2016-08-03 10:13:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
add mode() function & tests
2016-08-03 10:13:00 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.IntegerPoint{Time: 0, Value: 10, Tags: ParseTags("host=hostA")}},
{&query.IntegerPoint{Time: 5, Value: 21, Tags: ParseTags("host=hostA")}},
{&query.IntegerPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB")}},
{&query.IntegerPoint{Time: 20, Value: 8, Tags: ParseTags("host=hostB")}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
add mode() function & tests
2016-08-03 10:13:00 +00:00
}
}
Add unsigned iterators for all types This allows unsigned data to be queried from the storage engine. Binary math is not yet implemented for unsigned types.
2017-09-18 16:28:37 +00:00
// Ensure that a unsigned iterator can be created for a mode() call.
func TestCallIterator_Mode_Unsigned(t *testing.T) {
itr, _ := query.NewModeIterator(&UnsignedIterator{Points: []query.UnsignedPoint{
{Time: 0, Value: 15, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 10, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 2, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 3, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 4, Value: 10, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 6, Value: 20, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 7, Value: 21, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 8, Value: 21, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: 11, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 22, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: 8, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 24, Value: 25, Tags: ParseTags("region=us-west,host=hostB")},
}},
query.IteratorOptions{
Expr: MustParseExpr(`mode("value")`),
Dimensions: []string{"host"},
Interval: query.Interval{Duration: 5 * time.Nanosecond},
Ordered: true,
Ascending: true,
},
)
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
t.Fatalf("unexpected error: %s", err)
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.UnsignedPoint{Time: 0, Value: 10, Tags: ParseTags("host=hostA")}},
{&query.UnsignedPoint{Time: 5, Value: 21, Tags: ParseTags("host=hostA")}},
{&query.UnsignedPoint{Time: 1, Value: 11, Tags: ParseTags("host=hostB")}},
{&query.UnsignedPoint{Time: 20, Value: 8, Tags: ParseTags("host=hostB")}},
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
}
}
add mode() function & tests
2016-08-03 10:13:00 +00:00
// Ensure that a string iterator can be created for a mode() call.
func TestCallIterator_Mode_String(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewModeIterator(&StringIterator{Points: []query.StringPoint{
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 0, Value: "15", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: "10", Tags: ParseTags("region=us-west,host=hostA")},
{Time: 2, Value: "10", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 3, Value: "10", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 4, Value: "10", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 6, Value: "20", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 7, Value: "21", Tags: ParseTags("region=us-east,host=hostA")},
{Time: 7, Value: "21", Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: "11", Tags: ParseTags("region=us-west,host=hostB")},
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 22, Value: "8", Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: "8", Tags: ParseTags("region=us-west,host=hostB")},
{Time: 24, Value: "25", Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
add mode() function & tests
2016-08-03 10:13:00 +00:00
Expr: MustParseExpr(`mode("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
add mode() function & tests
2016-08-03 10:13:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
add mode() function & tests
2016-08-03 10:13:00 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.StringPoint{Time: 0, Value: "10", Tags: ParseTags("host=hostA")}},
{&query.StringPoint{Time: 5, Value: "21", Tags: ParseTags("host=hostA")}},
{&query.StringPoint{Time: 1, Value: "11", Tags: ParseTags("host=hostB")}},
{&query.StringPoint{Time: 20, Value: "8", Tags: ParseTags("host=hostB")}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
add mode() function & tests
2016-08-03 10:13:00 +00:00
}
}
// Ensure that a boolean iterator can be created for a modBooleanl.
func TestCallIterator_Mode_Boolean(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, _ := query.NewModeIterator(&BooleanIterator{Points: []query.BooleanPoint{
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 0, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 1, Value: true, Tags: ParseTags("region=us-west,host=hostA")},
{Time: 2, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 3, Value: true, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 4, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 6, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 7, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
{Time: 8, Value: false, Tags: ParseTags("region=us-east,host=hostA")},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
{Time: 1, Value: false, Tags: ParseTags("region=us-west,host=hostB")},
add mode() function & tests
2016-08-03 10:13:00 +00:00
{Time: 22, Value: false, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 23, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
{Time: 24, Value: true, Tags: ParseTags("region=us-west,host=hostB")},
}},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
add mode() function & tests
2016-08-03 10:13:00 +00:00
Expr: MustParseExpr(`mode("value")`),
Dimensions: []string{"host"},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Interval: query.Interval{Duration: 5 * time.Nanosecond},
top() and bottom() now returns the time for every point `top()` and `bottom()` will now organize the points by time and also keep the points original time even when a time grouping is used. At the same time, `top()` and `bottom()` will no longer honor any fill options that are present since they don't really make sense for these specific functions. This also fixes the aggregate and selectors to honor the ordered iterator option so iterator remain ordered and to also respect the buckets that are created by the final dimensions of the query so that two buckets don't overlap each other within the same reducer. A test has been added for this situation. This should clarify and encourage the use of the ordered attribute within the query engine.
2017-04-11 15:30:06 +00:00
Ordered: true,
Ascending: true,
add mode() function & tests
2016-08-03 10:13:00 +00:00
},
)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
if a, err := Iterators([]query.Iterator{itr}).ReadAll(); err != nil {
add mode() function & tests
2016-08-03 10:13:00 +00:00
t.Fatalf("unexpected error: %s", err)
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
} else if diff := cmp.Diff(a, [][]query.Point{
{&query.BooleanPoint{Time: 0, Value: true, Tags: ParseTags("host=hostA")}},
{&query.BooleanPoint{Time: 5, Value: false, Tags: ParseTags("host=hostA")}},
{&query.BooleanPoint{Time: 1, Value: false, Tags: ParseTags("host=hostB")}},
{&query.BooleanPoint{Time: 20, Value: true, Tags: ParseTags("host=hostB")}},
Switch influxql to use google/go-cmp to compare the point structs The diff will improve the debug message when a test fails. Right now, the message is unreadable. The diff message from go-cmp is much better.
2017-07-21 18:07:46 +00:00
}); diff != "" {
t.Fatalf("unexpected points:\n%s", diff)
add mode() function & tests
2016-08-03 10:13:00 +00:00
}
}
Remove the non-unreachable panics in the new query engine The only panics left are ones that should be unreachable unless there is a bug. Fixes #5777.
2016-02-22 17:51:45 +00:00
func TestNewCallIterator_UnsupportedExprName(t *testing.T) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
_, err := query.NewCallIterator(
Remove the non-unreachable panics in the new query engine The only panics left are ones that should be unreachable unless there is a bug. Fixes #5777.
2016-02-22 17:51:45 +00:00
&FloatIterator{},
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.IteratorOptions{
Remove the non-unreachable panics in the new query engine The only panics left are ones that should be unreachable unless there is a bug. Fixes #5777.
2016-02-22 17:51:45 +00:00
Expr: MustParseExpr(`foobar("value")`),
},
)
if err == nil || err.Error() != "unsupported function call: foobar" {
t.Errorf("unexpected error: %s", err)
}
}
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
func BenchmarkCountIterator_1K(b *testing.B) { benchmarkCountIterator(b, 1000) }
func BenchmarkCountIterator_100K(b *testing.B) { benchmarkCountIterator(b, 100000) }
func BenchmarkCountIterator_1M(b *testing.B) { benchmarkCountIterator(b, 1000000) }
func benchmarkCountIterator(b *testing.B, pointN int) {
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
benchmarkCallIterator(b, query.IteratorOptions{
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
Expr: MustParseExpr("count(value)"),
StartTime: influxql.MinTime,
EndTime: influxql.MaxTime,
}, pointN)
}
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
func benchmarkCallIterator(b *testing.B, opt query.IteratorOptions, pointN int) {
refactor query engine
2015-11-04 21:06:06 +00:00
b.ReportAllocs()
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
for i := 0; i < b.N; i++ {
// Create a lightweight point generator.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
p := query.FloatPoint{Name: "cpu", Value: 100}
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
input := FloatPointGenerator{
N: pointN,
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Fn: func(i int) *query.FloatPoint { return &p },
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
}
Remove the non-unreachable panics in the new query engine The only panics left are ones that should be unreachable unless there is a bug. Fixes #5777.
2016-02-22 17:51:45 +00:00
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
// Execute call against input.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, err := query.NewCallIterator(&input, opt)
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
if err != nil {
b.Fatal(err)
refactor query engine
2015-11-04 21:06:06 +00:00
}
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.DrainIterator(itr)
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
}
}
Add sample function to query language First Pass at implementing sample Add sample iterators for all types Remove size from sample struct Fix off by one error when generating random number Add benchmarks for sample iterator Add test and associated fixes for off by one error Add test for sample function Remove NumericLiteral from sample function call Make clear that the counter is incr w/ each call Rename IsRandom to AllSamplesSeen Add a rng for each reducer that is created The default rng that comes with math/rand has a global lock. To avoid having to worry about any contention on the lock, each reducer now has its own time seeded rng. Add sample function to changelog
2016-10-04 21:20:35 +00:00
func BenchmarkSampleIterator_1k(b *testing.B) { benchmarkSampleIterator(b, 1000) }
func BenchmarkSampleIterator_100k(b *testing.B) { benchmarkSampleIterator(b, 100000) }
func BenchmarkSampleIterator_1M(b *testing.B) { benchmarkSampleIterator(b, 1000000) }
func benchmarkSampleIterator(b *testing.B, pointN int) {
b.ReportAllocs()
// Create a lightweight point generator.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
p := query.FloatPoint{Name: "cpu"}
Add sample function to query language First Pass at implementing sample Add sample iterators for all types Remove size from sample struct Fix off by one error when generating random number Add benchmarks for sample iterator Add test and associated fixes for off by one error Add test for sample function Remove NumericLiteral from sample function call Make clear that the counter is incr w/ each call Rename IsRandom to AllSamplesSeen Add a rng for each reducer that is created The default rng that comes with math/rand has a global lock. To avoid having to worry about any contention on the lock, each reducer now has its own time seeded rng. Add sample function to changelog
2016-10-04 21:20:35 +00:00
input := FloatPointGenerator{
N: pointN,
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Fn: func(i int) *query.FloatPoint {
Add sample function to query language First Pass at implementing sample Add sample iterators for all types Remove size from sample struct Fix off by one error when generating random number Add benchmarks for sample iterator Add test and associated fixes for off by one error Add test for sample function Remove NumericLiteral from sample function call Make clear that the counter is incr w/ each call Rename IsRandom to AllSamplesSeen Add a rng for each reducer that is created The default rng that comes with math/rand has a global lock. To avoid having to worry about any contention on the lock, each reducer now has its own time seeded rng. Add sample function to changelog
2016-10-04 21:20:35 +00:00
p.Value = float64(i)
return &p
},
}
for i := 0; i < b.N; i++ {
// Execute call against input.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, err := query.NewSampleIterator(&input, query.IteratorOptions{}, 100)
Add sample function to query language First Pass at implementing sample Add sample iterators for all types Remove size from sample struct Fix off by one error when generating random number Add benchmarks for sample iterator Add test and associated fixes for off by one error Add test for sample function Remove NumericLiteral from sample function call Make clear that the counter is incr w/ each call Rename IsRandom to AllSamplesSeen Add a rng for each reducer that is created The default rng that comes with math/rand has a global lock. To avoid having to worry about any contention on the lock, each reducer now has its own time seeded rng. Add sample function to changelog
2016-10-04 21:20:35 +00:00
if err != nil {
b.Fatal(err)
}
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.DrainIterator(itr)
Add sample function to query language First Pass at implementing sample Add sample iterators for all types Remove size from sample struct Fix off by one error when generating random number Add benchmarks for sample iterator Add test and associated fixes for off by one error Add test for sample function Remove NumericLiteral from sample function call Make clear that the counter is incr w/ each call Rename IsRandom to AllSamplesSeen Add a rng for each reducer that is created The default rng that comes with math/rand has a global lock. To avoid having to worry about any contention on the lock, each reducer now has its own time seeded rng. Add sample function to changelog
2016-10-04 21:20:35 +00:00
}
}
add distinct iterator benchmark
2016-04-12 19:22:03 +00:00
func BenchmarkDistinctIterator_1K(b *testing.B) { benchmarkDistinctIterator(b, 1000) }
func BenchmarkDistinctIterator_100K(b *testing.B) { benchmarkDistinctIterator(b, 100000) }
func BenchmarkDistinctIterator_1M(b *testing.B) { benchmarkDistinctIterator(b, 1000000) }
func benchmarkDistinctIterator(b *testing.B, pointN int) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
// Create a lightweight point generator.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
p := query.FloatPoint{Name: "cpu"}
add distinct iterator benchmark
2016-04-12 19:22:03 +00:00
input := FloatPointGenerator{
N: pointN,
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Fn: func(i int) *query.FloatPoint {
add distinct iterator benchmark
2016-04-12 19:22:03 +00:00
p.Value = float64(i % 10)
return &p
},
}
// Execute call against input.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, err := query.NewDistinctIterator(&input, query.IteratorOptions{})
add distinct iterator benchmark
2016-04-12 19:22:03 +00:00
if err != nil {
b.Fatal(err)
}
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.DrainIterator(itr)
add distinct iterator benchmark
2016-04-12 19:22:03 +00:00
}
}
add mode() function & tests
2016-08-03 10:13:00 +00:00
func BenchmarkModeIterator_1K(b *testing.B) { benchmarkModeIterator(b, 1000) }
func BenchmarkModeIterator_100K(b *testing.B) { benchmarkModeIterator(b, 100000) }
func BenchmarkModeIterator_1M(b *testing.B) { benchmarkModeIterator(b, 1000000) }
added benchmark tests for median aggrergator (Package: influxql,influxql_test)
2016-08-04 02:32:19 +00:00
add mode() function & tests
2016-08-03 10:13:00 +00:00
func benchmarkModeIterator(b *testing.B, pointN int) {
added benchmark tests for median aggrergator (Package: influxql,influxql_test)
2016-08-04 02:32:19 +00:00
b.ReportAllocs()
for i := 0; i < b.N; i++ {
// Create a lightweight point generator.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
p := query.FloatPoint{Name: "cpu"}
added benchmark tests for median aggrergator (Package: influxql,influxql_test)
2016-08-04 02:32:19 +00:00
input := FloatPointGenerator{
N: pointN,
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Fn: func(i int) *query.FloatPoint {
add mode() function & tests
2016-08-03 10:13:00 +00:00
p.Value = float64(10)
added benchmark tests for median aggrergator (Package: influxql,influxql_test)
2016-08-04 02:32:19 +00:00
return &p
},
}
// Execute call against input.
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
itr, err := query.NewModeIterator(&input, query.IteratorOptions{})
added benchmark tests for median aggrergator (Package: influxql,influxql_test)
2016-08-04 02:32:19 +00:00
if err != nil {
b.Fatal(err)
}
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
query.DrainIterator(itr)
added benchmark tests for median aggrergator (Package: influxql,influxql_test)
2016-08-04 02:32:19 +00:00
}
}
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
type FloatPointGenerator struct {
i int
N int
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
Fn func(i int) *query.FloatPoint
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
}
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
func (g *FloatPointGenerator) Close() error { return nil }
func (g *FloatPointGenerator) Stats() query.IteratorStats { return query.IteratorStats{} }
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
func (g *FloatPointGenerator) Next() (*query.FloatPoint, error) {
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
if g.i == g.N {
Update SHOW TAG VALUES to use a fast dedupe iterator Include a benchmark test for the fast dedupe iterator.
2016-06-03 03:03:57 +00:00
return nil, nil
refactor query engine
2015-11-04 21:06:06 +00:00
}
tsm1 query engine alloc reduction This commit makes a number of performance improvements to reduce allocations during query execution. Several objects and buffers are now reused across the components to avoid allocations. Previously a simple `count(value)` query across 1M points would require 26,000+ allocations. After the changes in this commit that number has been reduced to 88.
2016-04-04 16:53:39 +00:00
p := g.Fn(g.i)
g.i++
Update SHOW TAG VALUES to use a fast dedupe iterator Include a benchmark test for the fast dedupe iterator.
2016-06-03 03:03:57 +00:00
return p, nil
refactor query engine
2015-11-04 21:06:06 +00:00
}
Optimize top() and bottom() using an incremental aggregator The previous version of `top()` and `bottom()` would gather all of the points to use in a slice, filter them (if necessary), then use a slightly modified heap sort to retrieve the top or bottom values. This performed horrendously from the standpoint of memory. Since it consumed so much memory and spent so much time in allocations (along with sorting a potentially very large slice), this affected speed too. These calls have now been modified so they keep the top or bottom points in a min or max heap. For `top()`, a new point will read the minimum value from the heap. If the new point is greater than the minimum point, it will replace the minimum point and fix the heap with the new value. If the new point is smaller, it discards that point. For `bottom()`, the process is the opposite. It will then sort the final result to ensure the correct ordering of the selected points. When `top()` or `bottom()` contain a tag to select, they have now been modified so this query: SELECT top(value, host, 2) FROM cpu Essentially becomes this query: SELECT top(value, 2), host FROM ( SELECT max(value) FROM cpu GROUP BY host ) This should drastically increase the performance of all `top()` and `bottom()` queries.
2017-05-16 17:37:39 +00:00
Separate the query engine into a separate package This change provides a clear separation between the query engine mechanics and the query language so that the language can be parsed and dealt with separate from the query engine itself.
2017-08-15 19:24:22 +00:00
func MustCallIterator(input query.Iterator, opt query.IteratorOptions) query.Iterator {
itr, err := query.NewCallIterator(input, opt)
Optimize top() and bottom() using an incremental aggregator The previous version of `top()` and `bottom()` would gather all of the points to use in a slice, filter them (if necessary), then use a slightly modified heap sort to retrieve the top or bottom values. This performed horrendously from the standpoint of memory. Since it consumed so much memory and spent so much time in allocations (along with sorting a potentially very large slice), this affected speed too. These calls have now been modified so they keep the top or bottom points in a min or max heap. For `top()`, a new point will read the minimum value from the heap. If the new point is greater than the minimum point, it will replace the minimum point and fix the heap with the new value. If the new point is smaller, it discards that point. For `bottom()`, the process is the opposite. It will then sort the final result to ensure the correct ordering of the selected points. When `top()` or `bottom()` contain a tag to select, they have now been modified so this query: SELECT top(value, host, 2) FROM cpu Essentially becomes this query: SELECT top(value, 2), host FROM ( SELECT max(value) FROM cpu GROUP BY host ) This should drastically increase the performance of all `top()` and `bottom()` queries.
2017-05-16 17:37:39 +00:00
if err != nil {
panic(err)
}
return itr
}