2ad8995355
fix: improve delete speed when a measurement is part of the predicate ( #23786 )
...
* fix: improve delete speed when a measurement is part of the predicate
* test: add test for deleting measurement by predicate
* chore: improve error messaging and capturing
* chore: set goland to use the right formatting style
2022-10-14 15:09:32 -04:00
e20b5e99a6
fix: remove nats for scraper processing ( #23107 )
...
* fix: remove nats for scraper processing
Scrapers now use go channels instead of NATS and interprocess communication.
This should fix #23085 .
Additionally, found and fixed #23106 .
* chore: fix formatting
* chore: fix static check and go.mod
* test: fix some flaky tests
* fix: mark NATS arguments as deprecated
2022-02-10 11:23:18 -05:00
b970e359dc
feat: remaining storage metrics from OSS engine ( #22938 )
...
* fix: simplify disk size tracking
* refactor: EngineTags in tsdb package
* fix: fewer compaction buckets and dead code removal
* feat: shard metrics
* chore: formatting
* feat: tsdb store metrics
* feat: retention check metrics
* chore: fix go vet
* fix: review comments
2021-12-02 09:01:46 -05:00
0a740856c7
feat: point write requests have metrics ( #22910 )
...
* fix: remove dead subscriptions code
* feat: storage_writer metrics
2021-11-22 12:58:17 -05:00
dece95d1dd
feat: tsm compaction metrics via prometheus ( #22904 )
...
* feat: tsm compaction metrics via prometheus
* chore: fix formatting
* chore: make activeCompactions a pointer
2021-11-19 14:51:22 -05:00
2795c600c1
feat: add `--storage-write-timeout` flag to set write request timeouts ( #22617 )
2021-10-04 15:38:09 -04:00
ec7841b355
feat: support for flux cardinality query ( #22441 )
...
* feat: works with custom iterator
* feat: works with existing iterators
* chore: cleanup
* test: consistent assertions for tests
* fix: better log message if trying to filter on the value of a field key
* fix: comment for handling boolean literal; handle false boolean as well
* fix: make time range checking inclusive
2021-09-13 13:20:56 -06:00
5015297d40
fix: more expressive errors ( #22448 )
...
* fix: more expressive errors
Closes #22446
* fix: server only logging for untyped errors
* chore: fix formatting
2021-09-13 15:12:35 -04:00
1482bdfa38
feat: partial support for series cardinality flux query ( #22378 )
2021-09-09 12:37:11 -06:00
dc3b501298
refactor: rename Lock/Unlock on KV stores to RLock/RUnlock ( #22357 )
2021-08-31 17:03:54 -04:00
d747e7ec4e
feat: add config parameters to toggle WAL concurrency and timeouts ( #21621 )
...
* feat: add context parameter to Take() method on fixed limiter
* refactor: plumb context through to uses of Take()
* test: update tests to pass context as needed
* feat: add config toggles for setting WAL write concurrency & timeout
2021-06-09 11:03:53 -04:00
b0ea3b6675
feat: backup and restore sqlite database ( #21584 )
...
* feat: new metadata backup endpoint
* feat: added restore/sql API endpoint
* fix: content-type is multipart/mixed, part names are kv and sql
* fix: changed multipart manifest to buckets and made it .json
* feat: added lock for backing up sqlite and bolt dbs
* fix: use read lock instead of write lock on kv during backup
* fix: use filepath.Join for temp dirs
2021-06-02 19:07:53 -04:00
00420fb54c
fix(influxql): make meta queries respect query timeout ( #21545 )
...
Co-authored-by: davidby-influx <dbyrne@influxdata.com>
2021-05-24 21:10:53 -04:00
00afd95cb7
refactor: automated move of errors and id from root to kit ( #21101 )
...
Co-authored-by: Sam Arnold <sarnold@influxdata.com>
2021-03-30 14:10:02 -04:00
ba4099ddb9
feat(cmd/influx): allow setting shard-group durations for buckets via API and CLI ( #20911 )
...
Co-authored-by: Alexander Savinykh <asavinykh@nbrx.de>
2021-03-11 14:51:03 -05:00
9aefa6f868
fix(tsdb): never use an inmem index ( #20313 )
...
And fix the logging setup for the TSDB storage engine
2020-12-23 07:46:57 -08:00
7dafc2cf34
feat(tsdb): Implement delete with predicate.
2020-12-02 14:55:02 -07:00
419b0cf76b
feat: Implement full restore
2020-11-05 10:05:01 -07:00
5f1968b331
fix: Skip deleted shard groups during backup
2020-11-05 10:05:01 -07:00
6e1097c625
feat: Minor updates to backup/restore
2020-11-05 10:05:01 -07:00
23679c2375
feat: Implement backup/restore CLI subcommands.
2020-11-05 10:05:01 -07:00
78d0262217
fix: Remove incorrect implementation of delete APIs and disable HTTP API
...
This commit removes incorrect implementations of the `DeleteBucket`
and `DeleteBucketRangePredicate` APIs from the storage package,
which remained after the transition to the tsdb 1.x storage engine.
Secondly, this PR utilizes the `ENotImplemented` error code to inform
users which call the `/api/v2/delete`
2020-09-24 11:11:46 -07:00
1bc54c7bfd
Merge pull request #19598 from influxdata/rogpeppe-005-storage-close-pointswriter
...
fix: storage: close PointsWriter when Engine is closed
2020-09-22 16:16:07 +01:00
3803bd8e26
fix: storage: close PointsWriter when Engine is closed
...
The PointsWriter has a Close method which seems like it should
be called when the Engine is shut down.
2020-09-22 15:42:50 +01:00
a0ce7c38ef
fix: Use DeleteDatabase to guarantee all files are closed and removed
...
A bucket is represented as a single database and retention policy and
therefore `DeleteDatabase` is the appropriate API to remove a bucket
from TSDB.
Fixes #19600
2020-09-21 13:05:50 -07:00
46db75d9a9
fix: ShardGroupDuration is updated for retention period updates
...
Fixes #19518
2020-09-18 07:31:48 -07:00
8576d1df99
feat: Implement DELETE FROM and DROP MEASUREMENT
...
Closes #19416
2020-09-11 11:37:43 -07:00
6f0cf049ca
feat: Port shard precreation service from InfluxDB 1.x
...
Provides new configuration parameters:
```
--storage-shard-precreator-advance-period
--storage-shard-precreator-check-interval
```
Closes #19520
2020-09-09 13:34:22 -07:00
e265f60b55
fix: Improvements in response to PR feedback
...
* Pass context.Context to Service.Open
* Remove redundant comments
* Bind to retention.Config configuration to be consistent with 1.x
2020-09-09 10:32:06 -07:00
7c2be6e780
feat: Port 1.x retention policy enforcement service
...
Configuration of the check interval is available via
```
--storage-retention-check-interval
```
Closes #19309
2020-09-09 10:32:06 -07:00
b3138d2ead
fix: Ensure TemporaryEngine returns new inner TSDB store
2020-09-01 10:50:44 -07:00
a607ac30b9
chore: Re-enable `TemporaryEngine` via `--e2e-testing`
2020-08-31 13:50:20 -07:00
dee8977d2c
chore: move v2/v1/tsdb → v2/tsdb
2020-08-26 10:46:47 -07:00
06fac044e5
fix(storage): Implement SeriesCardinality, which is per-bucket
...
Fixed unit tests
2020-08-25 14:52:00 -07:00
d7434216e2
fix(storage): Use correct engine paths
...
Remove redundant repl.go after rebasing on 8dd8d1f79e
2020-08-03 09:21:04 -07:00
21c3733464
fix(storage): meta.Client persists to kv and updates via BucketService
...
Includes initial migration to add metadata bucket
2020-08-03 09:21:02 -07:00
89532f0c29
refactor: move v2/v1/models to v2/models
2020-08-03 09:20:51 -07:00
26c18ac0a5
chore: Converging on 1.x tsdb
2020-08-03 09:20:48 -07:00
2b175291be
refactor: WIP removing tsbd
2020-08-03 09:18:34 -07:00
a97ec1f34e
feat: Initial commit with working reads
2020-08-03 09:18:34 -07:00
89efcb3d74
refactor: writes work
2020-08-03 09:18:34 -07:00
e3060c291c
refactor: tsdb store builds an runs
2020-08-03 09:18:32 -07:00
5f426195bf
refactor: start inserting tsdb store into engine
2020-08-03 09:17:24 -07:00
8c4bb1d65f
refactor: strip contents of storage engine
2020-08-03 09:17:24 -07:00
3cc2638bbf
feat(tsi1): Add optional mincore limiter to TSI
2020-07-22 10:17:42 -06:00
c28eb70856
feat(mincore): Add page fault limiter
...
This commit adds `mincore.Limiter` which throttles page faults caused
by mmap() data. It works by periodically calling `mincore()` to determine
which pages are not resident in memory and using `rate.Limiter` to
throttle accessing using a token bucket algorithm.
2020-07-17 09:37:31 -06:00
9784ebb781
feat(storage): Add option to disable WritePoints() validation.
...
This commit adds the `WithWritePointsValidationEnabled()` option
to disable validation in `Engine.WritePoints()` as the same
validation is performed earlier in the call stack by cloud.
2020-06-23 09:07:49 -06:00
3e87ef52b6
feat(storage): Add compaction enablement API to Engine.
2020-04-28 08:23:34 -06:00
0ae8bebd75
refactor: rewrite imports to include the /v2 suffix for version 2
2020-04-03 12:39:20 -05:00
386098da36
refactor(storage): move and remove to help cleanup tsdb package ( #17275 )
...
* refactor(tsdb): move series file config to seriesfile package
* refactor(tsdb): removed unchecked const EOF
* refactor(tsdb): unexport errors
* refactor(tsdb): remove unused TagValueIterators
* refactor(tsdb): remove SeriesIDIterator usage in tsdb/seriesfile
* refactor(tsdb): remove one-use MeasurementIterators
* refactor(tsdb): remove unused type measurementSliceIterator
* refactor(tsdb): remove unused types TagKeyIterators and tagKeySliceIterator
* refactor(storage): remove unused method Engine.ApplyFnToSeriesIDSet
* refactor(tsdb): rename AllSeriesIDs() -> SeriesIDs()
2020-03-16 12:23:15 -07:00
Jeffrey Smith II
Sam Arnold
Daniel Moran
William Baker
Ben Johnson
Stuart Carnie
Roger Peppe
Edd Robinson
Jonathan A. Sternberg
Jacob Marble