888689f5d3
Move values loop under type switch
...
All the values read must be of the same type so repeatedly using
the type switch is confusing and less efficiient.
2017-04-20 13:39:49 -06:00
b0988511bf
Use fixed size array instead of slice
2017-04-20 13:38:33 -06:00
da6bdfdda8
Use bufio.Reader when reading wal segments
...
Reduces disk IO due to small reads.
2017-04-20 13:33:42 -06:00
8e9cbd7ffc
Simplify WALSegmentReader.UnmarshalBinary
...
There were two loops over nvals which created some extra allocation
which coudl be replaced with a simplet slice capacity and append.
2017-04-20 13:33:42 -06:00
4da7054221
Merge pull request #8302 from influxdata/jw-writes2
...
Write throughput/concurrency improvements
2017-04-20 13:23:24 -06:00
71825d20c8
Update changelog
2017-04-20 12:31:06 -06:00
02b663b651
Fix lock contention in Index.CreateSeriesListIfNotExists
...
There was contention on the write lock which only needs to be acquired
when checking to see if the log file should be rolled over.
2017-04-20 12:28:42 -06:00
40ec85aacd
Fix lock contention in LogFile.SeriesWithBuffer
...
Under high write load, the check for each series was done sequentially
which caused a lot of CPU time to acquire/release the RLock on LogFile.
This switches the code to check multiple series at once under an RLock
similar to the chang for inmem.
2017-04-20 12:28:42 -06:00
0e715b5b74
Reduce lock contention on MeasurementFields
2017-04-20 12:28:42 -06:00
ef65ee77f4
Switch WAL byte pools to sync/pool
...
The current bytes.Pool will hold onto byte slices indefinitely. Large
writes can cause the pool to hold onto very large buffers over time.
Testing w/ sync/pool seems to perform similarly now so using a sync/pool
will allow these buffers to be GC'd when necessary.
2017-04-20 12:28:42 -06:00
d155d37ca8
Reduce TSM write buffer
...
When many TSM files are being compacted, the buffers can add up fairly
quickly.
2017-04-20 12:28:42 -06:00
3c2825a851
Reduce lock thrashing when checking series
...
The inmem index would call CreateSeriesIfNotExist for each series
which takes and releases and RLock to see if a series exists. Under
high write load, the lock shows up in profiles quite a bit. This
adds a filtering step that obtains a single RLock and checks all the
series and returns the non-existent series to contine though the slow
path.
2017-04-20 12:28:41 -06:00
8aeda47a8e
Avoid growing slice when mapping points to shards
2017-04-20 12:28:41 -06:00
d7c5dd0a3e
Reduce wal sync goroutine churn
...
Under high write load, the sync goroutine would startup, and end
very frequently. Starting a new goroutine so frequently adds a small
amount of latency which causes writes to take long and sometimes timeout.
This changes the goroutine to loop until there are no more waiters which
reduce the churn and latency.
2017-04-20 12:28:34 -06:00
aa9925621b
Fix deadlock in wal
...
If the sync waiters channel was full, it would block sending to the
channel while holding a the wal write lock. The sync goroutine would
then be stuck acquiring the write lock and could not drain the channel.
This increases the buffer to 1024 which would require a very high write
load to fill as well as retuns and error if the channel is full to prevent
the blocking.
2017-04-19 11:33:13 -06:00
a19ce9c10f
Reduce index lock contention
...
Series and Measurment have their own locks and we do not need to
hold locks on the index while using those types.
2017-04-18 16:32:33 -06:00
883b3dcbbb
Reduce lock content in AssignShard
...
The lock shows up under write load. It only needs to be assigned
once so a read lock eliminates the contention.
2017-04-18 16:32:33 -06:00
eeaad877bc
Merge pull request #8297 from influxdata/jw-merge-123
...
Merge 1.2.3 changes to master
2017-04-17 09:18:46 -06:00
5c51ae7319
Merge branch '1.2' into jw-merge-123
2017-04-14 14:36:54 -06:00
e6e5968c3a
Merge pull request #8294 from influxdata/cjl-cli-redact-password-history
...
Cherry Pick sanitize passwords in CLI history
2017-04-14 13:38:10 -05:00
7e6bd5be6b
Fix spelling mistake in HTTP section of config (influxdb#8231)
...
Fixes plutonium#1198
2017-04-14 14:26:31 -04:00
8c0f5a7dbe
redact passwords before saving history in cli
2017-04-14 13:13:56 -05:00
f1882ae705
Fix spelling mistake in HTTP section of config (influxdb#8231)
2017-04-14 14:13:44 -04:00
57a2abbc87
Restrict top() and bottom() selectors to be used with no other functions
2017-04-14 10:23:07 -05:00
0572f14a5c
Merge pull request #8293 from influxdata/cjl-1.2-fixes
...
Cherry Pick Security Fixes to 1.2
2017-04-13 15:25:44 -05:00
775c5d243d
Add changelog for 8187
2017-04-13 13:33:25 -05:00
215b5fc89c
Add HasDefaultDatabase interface to several statements
2017-04-13 13:32:17 -05:00
31db9d6f46
Merge pull request #8292 from influxdata/cjl-changelog-8187
...
Add changelog for 8187
2017-04-13 13:30:30 -05:00
f96b59ed20
Add changelog for 8187
2017-04-13 10:31:31 -05:00
4ca132765b
Merge pull request #8187 from influxdata/cjl-create-subscription-add-default-database
...
Several statements were missing the DefaultDatabase method
2017-04-12 16:57:19 -05:00
9060a2a5ff
Add HasDefaultDatabase interface to several statements
2017-04-12 13:41:28 -05:00
c64524b6b0
Merge pull request #8279 from influxdata/js-junit-test-output
...
Use go junit formatter to output tests in junit format on CircleCI
2017-04-12 10:13:04 -05:00
d653b76b5b
Merge pull request #8282 from influxdata/js-8281-influxql-select-tests
...
Fix influxql select tests
2017-04-11 15:05:08 -05:00
48f1c0a013
Merge pull request #8258 from influxdata/js-8067-forbid-fill-none-linear-for-raw-values
...
Restrict fill(none) and fill(linear) to be usable only with aggregate queries
2017-04-10 20:52:32 -05:00
c6e1b83906
Fix influxql select tests
...
The inputs are now sent to the tested iterator in the correct order so
we can more accurately test each individual select statement.
2017-04-10 20:51:14 -05:00
a550d323c4
Restrict fill(none) and fill(linear) to be usable only with aggregate queries
2017-04-10 15:58:05 -05:00
9d83e4626d
Merge pull request #8267 from influxdata/js-8252-fix-or-condition-with-null-values
...
Implicitly cast null to false in binary expressions with a boolean
2017-04-10 15:54:23 -05:00
d0a9c4c781
Use go junit formatter to output tests in junit format on CircleCI
2017-04-10 11:47:51 -05:00
db430cb434
Merge pull request #8272 from influxdata/jw-12-drop-time
...
Fix dropping fields creating data corruption
2017-04-07 13:16:29 -06:00
ff1270dfeb
Fix dropping fields created data corruption
...
The Point is intended to be immutable after being parsed since it
is shared by several goroutines. When dropping a field (e.g. time),
corrupted data can result if one goroutine is delete the field
while another is marshaling the underlying byte slices.
To avoid this, the shard will just skip invalid fields and series
instead of trying to mutate them by deleting them.
2017-04-07 12:58:42 -06:00
7c5ce27bb0
Merge pull request #8269 from influxdata/jw-points-unmarshal
...
Optimize Point.NewPointFromBytes
2017-04-06 13:18:15 -06:00
1de99cd219
Merge pull request #8268 from influxdata/jw-dedup-measurements
...
Ensure MeasurementNames deduplicates measurements across shards
2017-04-06 13:10:56 -06:00
c3e0748bd9
Optimize Point.NewPointFromBytes
...
There was a check to ensure that fields exists when unmarshalBinary
is called. This created a map and other garbage just to see if any
fields exist.
This changes it to use a FieldIterator that does not allocate as
much as the other method.
2017-04-06 12:51:45 -06:00
927acb5ab9
Ensure MeasurementNames deduplicates measurements across shards
2017-04-06 12:17:29 -06:00
0a5e4bd92b
Implicitly cast null to false in binary expressions with a boolean
...
Also more consistently treat a binary expression with strings so it
produces the same value no matter the direction of the expression.
2017-04-06 12:26:04 -05:00
03338c69b5
Merge pull request #8263 from influxdata/jw-meta-deadlock
...
Fix deadlock in Measurement.SeriesIDsAllOrByExpr
2017-04-05 20:19:14 -04:00
cf100647e0
Fix deadlock in Measurement.SeriesIDsAllOrByExpr
...
SeriesIDsAllOrByExpr took a RLock and ended up calling SeriesIDs
which can take a Lock causing a deadlock.
2017-04-05 16:22:45 -06:00
45895862b7
Merge pull request #8058 from karlding/service-golinting
...
Make services/{admin, httpd, subscriber, udp} golintable
2017-04-05 12:30:11 -05:00
fad4784f9e
Merge pull request #8254 from influxdata/jw-8251-backport
...
Backport - Fix delete time fields creating unparseable points
2017-04-04 18:47:42 -04:00
91bfc5772a
Update changelog
2017-04-04 16:39:53 -06:00
Jason Wilder
Jason Wilder
Cory LaNou
Ryan Betts
Jonathan A. Sternberg