Commit Graph

12694 Commits (ed2ff3b40136c614ef2a7365c92174c851e4a192)

Author SHA1 Message Date
Stuart Carnie dc04eaa8f3 Amendments based on feedback
* Fprint* functions
* No nakedness
* clarify panic messages
* spacing between case statements
* remove break in favor of return
* remove goto in favor of for { continue }
2017-10-25 13:38:07 -07:00
Stuart Carnie b8a6ee7c12 store command to query storage RPC APIs 2017-10-25 13:38:07 -07:00
Stuart Carnie 415ed14c53 storage service
* storage service is disabled by default
* default port 8082
* RPC interface generated using yarpc via service.proto
2017-10-25 13:38:07 -07:00
Stuart Carnie c39f1ad748 Add batch cursor support to tsdb and tsm1
* batch cursors return slices of timestamps and values to reduce call
  overhead. Significantly improved iteration.
* added CreateCursor API to Shard, Engine
* moved build*Cursor to code gen
2017-10-25 13:38:07 -07:00
Stuart Carnie 3e28323a10 Simplified Decode*Block functions
* array has already been sized correctly
* eliminates bounds checking for each element access
* reduces decoding of 30,000,000 points via storage API from
  584ms to 540ms on average
2017-10-25 13:38:07 -07:00
Stuart Carnie fb40c5b825 Update Godeps 2017-10-25 13:38:07 -07:00
Stuart Carnie 0bee9c7068 Merge pull request #9011 from influxdata/sgc-9007
fixes #9007
2017-10-25 13:37:01 -07:00
Stuart Carnie c51ba16287 fixes #9007 2017-10-25 13:08:55 -07:00
Edd Robinson 34836cebee Merge pull request #9009 from influxdata/er-8989-measurement-panic
Fix race in Measurement index
2017-10-25 18:20:03 +01:00
Edd Robinson 0f4c251148 Update CHANGELOG 2017-10-25 13:34:44 +01:00
Edd Robinson 47bd069315 Fix race in Measurement index
Fixes #8989 and #8633.

Previously when issuing commands involving a regex check, walking
through the tags keys/values on a measurement, using the measurement's
index, would be racy.

This commit adds a new `TagKeyValue` type that abstracts away the
multi-layer map we were using as an inverted index from tag keys and
values to series ids. With this abstraction we can also make concurrent
access to this inverted index goroutine safe.

Finally, this commit fixes a very old bug in the index which will affect
any query using a regex. Previously we would always check _every_ tag
against a regex for a measurement, even when we had found a match.
2017-10-25 13:34:21 +01:00
Diego Reiriz 350d736b52 Updated Queries.md
Updated Queries.md to add example statements to Select, GroupBy and Delete
2017-10-25 09:05:52 +02:00
Stuart Carnie 24c769fca3 Merge pull request #9005 from influxdata/sgc-query-interrupt
return query.ErrQueryInterrupted for read on InterruptCh
2017-10-24 14:23:33 -07:00
Stuart Carnie 63d7310572 update CHANGELOG 2017-10-24 14:17:02 -07:00
Stuart Carnie b7579340fe return query.ErrQueryInterrupted for read on InterruptCh 2017-10-24 14:10:28 -07:00
Jason Wilder 955829e7c3 Merge pull request #9003 from influxdata/jw-delete-regression
Delete series in batches
2017-10-24 13:54:33 -06:00
Jason Wilder cbbbe8bedb Delete series in batches
This fixes a regression where deleting series keys would happen
one at a time instead of in bulk.
2017-10-24 11:06:21 -06:00
Stuart Carnie 0bd10eadf0 Merge pull request #9000 from influxdata/sgc-template
Add updated template for EXPLAIN ANALYZE
2017-10-23 16:20:00 -07:00
Stuart Carnie 02a05e86ee Add missing template changes for EXPLAIN ANALYZE 2017-10-23 14:46:36 -07:00
Ben Johnson 63324c2435 Merge pull request #8995 from benbjohnson/tsi-sort-log-tag-values
Sort & validate TSI key value insertion.
2017-10-23 10:46:24 -06:00
Ben Johnson 5a77238f30
Sort & validate TSI key value insertion. 2017-10-23 10:46:01 -06:00
Ben Johnson 6aaa77241a Merge pull request #8963 from benbjohnson/streaming-inmem2tsi
Streaming inmem2tsi conversion.
2017-10-20 15:14:53 -06:00
Ben Johnson 5af2d73867
CHANGELOG 2017-10-20 15:14:39 -06:00
Ben Johnson 1f4e070b3a
Switch to use tsm1.CacheLoader. 2017-10-20 15:14:17 -06:00
Ben Johnson ea43660a55
Streaming inmem2tsi conversion.
Changes the `influx_inspect inmem2tsi` tool to stream each TSM/WAL
file and convert to a TSI index instead of loading the entire shard's
in-memory index first.
2017-10-20 15:14:17 -06:00
Stuart Carnie 21db403342 Merge pull request #8993 from influxdata/sgc-32bit-fixes
fix overflow for 32-bit architecture
2017-10-20 10:58:56 -07:00
Stuart Carnie ac3bf300d3 fix overflow for 32-bit architecture 2017-10-20 10:22:28 -07:00
Stuart Carnie 9119c6ff9a Merge pull request #8991 from influxdata/sgc-go18-fix
make (*T).Helper() optional for Go 1.8 compatibility
2017-10-20 09:35:51 -07:00
Stuart Carnie 4150f64711 Merge pull request #8992 from influxdata/sgc-explain-cl
Update CHANGELOG for EXPLAIN ANALYZE feature
2017-10-20 09:03:02 -07:00
Stuart Carnie 47a2f8745e make (*T).Helper() optional 2017-10-20 08:59:50 -07:00
Stuart Carnie d7b73f9b02 Update CHANGELOG 2017-10-20 08:58:45 -07:00
Stuart Carnie 618f0d0aa7 Merge pull request #8947 from influxdata/sgc-explain
EXPLAIN ANALYZE implementation
2017-10-20 08:35:13 -07:00
Stuart Carnie e9313876ab EXPLAIN ANALYZE
* Introduces EXPLAIN ANALYZE command, which
  produces a detailed tree of operations used to
  execute the query.

introduce context.Context to APIs

metrics package

* create groups of named measurements
* safe for concurrent access

tracing package

EXPLAIN ANALYZE implementation for OSS

Serialize EXPLAIN ANALYZE traces from remote nodes

use context.Background for tests

group with other stdlib packages

additional documentation and remove unused API

use influxdb/pkg/testing/assert

remove testify reference
2017-10-20 08:01:37 -07:00
Jonathan A. Sternberg 4cdb7828d4 Merge pull request #8990 from influxdata/js-pid-file-code-cleanup
Code cleanup for removing the pid file
2017-10-20 09:36:18 -05:00
Jonathan A. Sternberg 75b8d9cd4f Code cleanup for removing the pid file 2017-10-20 09:17:16 -05:00
Jonathan A. Sternberg 8c42345c69 Merge pull request #8987 from influxdata/js-8983-remove-pid-file
Remove the pidfile after the server has exited
2017-10-20 09:16:00 -05:00
Jonathan A. Sternberg 2005a8c8e5 Remove the pidfile after the server has exited 2017-10-19 16:33:59 -05:00
Jonathan A. Sternberg f58a194ee0 Merge pull request #8980 from influxdata/js-8780-prevent-deadlock-during-collectd-shutdown
Prevent deadlock during collectd, graphite, opentsdb, and udp shutdown
2017-10-19 16:33:38 -05:00
Jonathan A. Sternberg 83ecab6299 Prevent deadlock during collectd, graphite, opentsdb, and udp shutdown
All of these services start up goroutines and then wait for the
goroutines to finish. Each of them has a `tsdb.PointBatcher` that may
return a point during the shutdown sequence. During the shutdown
sequence, a lock was held. This lock may get accessed when attempting to
write the point that came back from the `tsdb.PointBatcher`. This caused
the read lock attempt to wait forever for the write lock to be unlocked
during `Close()`.

This modifies these methods so that the write lock is released while
waiting for goroutines to finish in these three services.
2017-10-19 15:57:05 -05:00
Jason Wilder 5650ac7320 Merge pull request #8981 from influxdata/jw-block-iterator
Fix data being dropped during compactions
2017-10-19 11:08:59 -06:00
Jason Wilder 05131f4453 Fix indirectIndex not removing fully deleted series
If multiple tombstones exists for a series that ended up causing the
full data to be deleted, the blocks were not removed from the offsets
in the index.  This causes the TSMReader to report that a key exist
but does not have any data.

During a compaction, every key should have at least one value.  Since
this invariant was broken, the compaction aborted early and ends up
dropping all series keys that are lexigraphically greater than where
the breakage occured.  This would cause data to be dropped during the
compaction.
2017-10-18 18:16:41 -06:00
Jason Wilder 9f102adabe Abort BlockIterator iteration if deletes detected
This fixes a potential bug where the BlockIterator would skip blocks
if the underlying TSMReader had deletes on it concurrently.  This
could possibly occur due to changes in 91eb9de3 that now use the
existing TSMReaders from the FileStore instead of creating new ones
during compaction.
2017-10-18 18:16:37 -06:00
Jason Wilder 1bbf9139d0 Merge pull request #8979 from influxdata/jw-delete-time
Fix data deleted outside of time range
2017-10-18 17:30:21 -06:00
Jason Wilder 4d171f3f40 Fix data deleted outside of time range 2017-10-18 13:39:47 -06:00
Ben Johnson 62093d2641 Merge pull request #8975 from benbjohnson/tsi-copy-returned-bytes
Copy returned bytes from TSI meta functions.
2017-10-18 09:26:02 -06:00
Edd Robinson 9b55ee2b90 Merge pull request #8935 from posquit0/patch-1
Fix mis-typing in README.md of UDP Service
2017-10-18 15:57:50 +01:00
Byungjin Park f7d8ad50e2 Update README.md 2017-10-18 22:46:44 +09:00
Jason Wilder 3bb56c6d5e Merge pull request #8976 from influxdata/jw-tsi-interrupt
Fix max select series limit for tsi
2017-10-18 07:27:23 -06:00
Ben Johnson 264971fc5a
CHANGELOG 2017-10-18 07:22:49 -06:00
Ben Johnson 8ad2048a6b
TSI byte copy usage comments. 2017-10-18 07:21:54 -06:00