This commit adds a write ahead log to the shard. Entries are cached
in memory and periodically flushed back into the index. The WAL and
the cache are both partitioned into buckets so that flushing doesn't
stop the world as long.
Field values that were out of range for the type would panic the database
when being inserted because the parser would allow them as valid points.
This change prevents those invalid values from being parsed and instead
returns an error.
An alternative fix considered was to handle the error and clamp the value
to the min/max value for the type. This would treat numeric range errors
slightly differently than other type erros which might lead to confusion.
The simplest fix with the current parser would be to just convert each field
to the type at parse time. Unfortunately, this adds extra memory allocations
and lowers throughput significantly. Since out of range values are less common
than in-range values, some heuristics are used to determine when the more
expensive type parsing and range checking is performed. Essentially, we only
do the slow path when we cannot determine that the value is in an acceptable
type range.
Fixes#3127
A field value of just a numeric value would be accepted by the line
protocol parser but the value would be set as the field name and
the value would be nil. Instead, return an error because all field
values need a field name.
Statements were only being normalized if a default database was included
in the query (usually via the query param 'db'). However if no default
database was included, and none was an explicit part of the measurement
name, no database-existence check was run. This result in a later panic
with wildcard expansion.
Fixes#2960
Integers were were written back to line protocol using strconv.FormatFloat
incorrectly. Large integers are written in scientific notation which
causes their type to change to a float when parsed back.
Supported boolean values are now t, T, true, TRUE, f, F, false, and
FALSE. This is what the strconv.ParseBool function supports with
the exception of 1 and 0. 1 and 0 would be parsed as ints in the
line protocol.
Previously, any non-true value would be parsed as false. e.g.
value=blah would parse to false. This will now return an error as
parsing time.
Adds more tests for invalid numbers such as 0.1a, -2.-4, as well
test for supported formats for negative and positive integers/floats
as well as scientific notation.
Philip O'Toole
Jason Wilder
Joseph Crail
Ben Johnson
Sean Beckett
Philip O'Toole
Todd Persen
ben hockey
Pradeep Chhetri
Paul Dix