a85d95d2e9
refactor: cleanup
2021-09-02 17:25:41 -04:00
bbc0fcf4c0
refactor: cleanup
2021-09-02 17:21:55 -04:00
e2274a9f41
feat: parser for delete predicate
2021-09-02 17:02:05 -04:00
ce6030a3cb
feat: Wire list deleted databases through gRPC and CLI APIs
2021-09-02 15:48:07 -04:00
581e49e38f
feat: Add a list_deleted_databases API
...
Fixes #2198 .
2021-09-02 14:57:27 -04:00
fbf18813ac
fix: Correct a mistaken assumption in in-memory object storage
...
The implementation of list_with_delimiter for the in-memory object
storage assumed that paths returned from the BTreeMap keys that sorted
greater than the prefix given to list_with_delimiter and for whom
prefix_matches returned true would also have parts after the prefix.
This didn't account for paths that started with the prefix but didn't
immediately have the delimiter after the prefix: that is,
prefix = 1/database_name
would match the in-memory paths:
1/database_name/0/rules.pb
1/database_name_and_another_thing/0/rules.pb
The first path here *would* return some parts_after_prefix, but the
second path would not and the previously existing code would panic for
the added path in the list_with_delimiter test case.
2021-09-02 14:57:27 -04:00
6da9cf246f
feat: Introduce a public DeletedDatabase type
2021-09-02 14:57:27 -04:00
8703aaff63
feat: Introduce a public GenerationId wrapper type
2021-09-02 14:57:27 -04:00
f47d7e6d92
feat: Return the time the database was deleted
2021-09-02 14:57:27 -04:00
51b6631d8f
feat: Implement a list_all_databases API that returns generation info too
2021-09-02 14:57:27 -04:00
1d50eb9880
test: Add a unit-ish test for list_possible_databases
2021-09-02 14:57:27 -04:00
a63eb53ac5
feat: forward connection config to Kafka write buffer
2021-09-02 16:53:31 +02:00
bdb772a7f7
Merge pull request #2458 from influxdata/crepererum/issue2455a
...
refactor: more flexible writer buffer config
2021-09-02 14:51:19 +00:00
81e672db14
feat: add `auto_create_sequencers` write buffer flag
2021-09-02 16:41:35 +02:00
ecf1f99ddb
refactor: more flexible writer buffer config
...
This allows:
- different types (instead of guessing through the connection URL)
- sequencer counts (not used yet but will be by #2455 )
- extensible configs (e.g. to configure Kafka in a more granular way,
not wired up yet)
- future extensions (since we use a message now instead of a single
string)
**BREAKING: This requires changes for deployed systems / existing DBs!**
2021-09-02 16:41:35 +02:00
025ea2b841
Merge pull request #2459 from influxdata/crepererum/massif
...
docs: add massif guide
2021-09-02 14:13:15 +00:00
bb94751d98
docs: add massif guide
2021-09-02 15:58:43 +02:00
09a5d27434
chore: ignore massif profiler outputs
2021-09-02 15:58:43 +02:00
9a246d59e5
Merge pull request #2453 from influxdata/crepererum/dump_catalog
...
feat: "dump catalog" debug CLI
2021-09-02 06:17:10 +00:00
4a863993ec
feat: "dump catalog" debug CLI
2021-09-02 08:08:20 +02:00
581ee64049
feat: add functions to dump catalog data to text
2021-09-02 08:07:07 +02:00
efcdc8e50f
feat: implement delete parser
2021-09-01 17:31:49 -04:00
6055152589
Merge pull request #2447 from influxdata/cn/server-startup-delete
...
test: Database startup error when there are multiple active generations
2021-09-01 13:41:00 +00:00
b3d04b3e26
Merge branch 'main' into cn/server-startup-delete
2021-09-01 13:32:34 +00:00
5808d73541
Merge pull request #2444 from influxdata/cn/list-but-not-deleted-databases
...
feat: Wire database deletion through the CLI, add tests for not including deleted databases in database list
2021-09-01 13:13:28 +00:00
c89ad70d07
test: Ensure we deleted some tombstone file
...
To guard against forgetting to change this test if we change the tombstone file.
Co-authored-by: Marco Neumann <marco@crepererum.net>
2021-09-01 09:04:25 -04:00
e183ecb3e7
Merge branch 'main' into cn/list-but-not-deleted-databases
2021-09-01 13:03:09 +00:00
983ae49494
Merge pull request #2450 from influxdata/crepererum/dump_catalog_prep
...
test: simplify testing around preserved catalog
2021-09-01 09:34:30 +00:00
06c941d798
refactor: split up `make_record_batch`
2021-09-01 11:26:05 +02:00
6ce586a2ac
docs: add docstrings to `PreservedCatalog` members
2021-09-01 11:26:05 +02:00
70a5ffeae7
test: allow creation of deterministic chunks and transactions
2021-09-01 11:26:05 +02:00
06833110ab
test: allow creation of less complex parquet chunks
2021-09-01 11:26:05 +02:00
27248850e5
refactor: use `byte::Bytes` for metadata in protobuf messages
...
That simplifies printing a bit since we `Vec<u8>` prints quite badly.
2021-09-01 11:26:05 +02:00
a4183de411
feat: more progress on the delete flow from grpc API to catalog chunks
2021-08-31 17:42:07 -04:00
6ba25728e1
chore: exclude non-source code from package ( #2452 )
...
Before this change, modifications to the following files would trigger a
rebuild on `cargo build`.
```
2,13d1
< .circleci/config.yml
< .circleci/get-deploy-tags.sh
< .editorconfig
< .gitattributes
< .github/ISSUE_TEMPLATE/bug_report.md
< .github/ISSUE_TEMPLATE/feature_request.md
< .github/PULL_REQUEST_TEMPLATE.md
< .github/dependabot.yml
< .github/semantic.yml
< .gitignore
< .kodiak.toml
< CONTRIBUTING.md
17,22d4
< Dockerfile
< Dockerfile.dockerignore
< LICENSE-APACHE
< LICENSE-MIT
< README.md
< buf.yaml
24,76d5
< docker/Dockerfile.ci
< docker/Dockerfile.ci.dockerignore
< docker/Dockerfile.ci.integration
< docker/Dockerfile.ci.integration.dockerignore
< docker/Dockerfile.iox
< docker/Dockerfile.iox.dockerignore
< docker/ci-kafka-docker-compose.yml
< docker/integration_test.sh
< docs/README.md
< docs/catalog_persistence.md
< docs/data_management.md
< docs/drawings/data_lifecycle.monopic
< docs/drawings/data_organization.monopic
< docs/encoding_thoughts.md
< docs/env.example
< docs/images/flame_graph.png
< docs/local_filesystems.md
< docs/metrics.md
< docs/multi_core_tasks.md
< docs/profiling.md
< docs/protobuf.md
< docs/regenerating_flatbuffers.md
< docs/server_startup.md
< docs/sql.md
< docs/style_guide.md
< docs/testing.md
< docs/tracing.md
< docs/valgrind.md
< perf/logs/iox_router.log
< perf/logs/iox_writer.log
< perf/logs/jaeger.log
< perf/logs/kafka.log
< perf/logs/minio.log
< perf/logs/zookeeper.log
< perf/volumes/minio/.minio.sys/buckets/.bloomcycle.bin
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/.bloomcycle.bin/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/.usage-cache.bin/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/.usage.json/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/iox1/.usage-cache.bin/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/iox2/.usage-cache.bin/fs.json
< perf/volumes/minio/.minio.sys/buckets/.tracker.bin
< perf/volumes/minio/.minio.sys/buckets/.usage-cache.bin
< perf/volumes/minio/.minio.sys/buckets/.usage.json
< perf/volumes/minio/.minio.sys/buckets/iox1/.metadata.bin
< perf/volumes/minio/.minio.sys/buckets/iox1/.usage-cache.bin
< perf/volumes/minio/.minio.sys/buckets/iox2/.metadata.bin
< perf/volumes/minio/.minio.sys/buckets/iox2/.usage-cache.bin
< perf/volumes/minio/.minio.sys/config/config.json
< perf/volumes/minio/.minio.sys/config/iam/format.json
< perf/volumes/minio/.minio.sys/format.json
< perf/volumes/zookeeper/data/myid
< perf/volumes/zookeeper/data/version-2/log.1
< perf/volumes/zookeeper/data/version-2/snapshot.0
79,87d7
< scripts/edit_db_rules
< scripts/genlp.py
< scripts/git-log-prs
< scripts/grpcurl
< scripts/jq_edit_db_rules
< scripts/logfmts
< scripts/prototxt
< scripts/valgrind.supp
< scripts/valigrind_leak
161d80
< tools/iplan
```
2021-08-31 17:14:02 +00:00
9972e04a18
Merge pull request #2440 from influxdata/dependabot/cargo/opentelemetry-jaeger-0.15.0
...
chore(deps): bump opentelemetry-jaeger from 0.14.0 to 0.15.0
2021-08-31 09:40:41 +00:00
79ad48ac3a
chore: rename "labels" to "attributes"
2021-08-31 11:31:15 +02:00
a5b8c1631b
chore: upgrade `opentelemetry[-prometheus]`
...
- "labels" are now named "attributes"
- `PrometheusExpoerter` is now created via a builder pattern which
provides way better defaults (e.g. it construct a sane controller
itself)
2021-08-31 10:39:57 +02:00
7ff28405ea
chore: fix `opentelemetry-jeager` upgrade
2021-08-31 10:39:57 +02:00
29b13ad2a6
chore(deps): bump opentelemetry-jaeger from 0.14.0 to 0.15.0
...
Bumps [opentelemetry-jaeger](https://github.com/open-telemetry/opentelemetry-rust ) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-rust/releases )
- [Commits](https://github.com/open-telemetry/opentelemetry-rust/compare/v0.14.0...v0.15.0 )
---
updated-dependencies:
- dependency-name: opentelemetry-jaeger
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-08-31 10:39:57 +02:00
4b46e3064d
Merge pull request #2443 from influxdata/crepererum/issue2403
...
test: use random free port for end2end tests
2021-08-31 08:25:21 +00:00
7e7855f727
test: avoid picking the same port twice under multi-threaded tests
2021-08-31 10:16:22 +02:00
69eae490ef
test: avoid picking the same port twice
2021-08-31 10:16:22 +02:00
e6afb844ff
test: use random free port for end2end tests
...
Fixes #786 .
Fixes #2403 .
2021-08-31 10:16:22 +02:00
f962d0ef2e
feat: first step to add delete_predicate into chunk catalog
2021-08-30 17:16:08 -04:00
2ff5a163ce
Merge pull request #2445 from influxdata/ntran/potential_overlap
...
fix: time column is alwyas last even if the column is sorted lexicographically
2021-08-30 20:42:19 +00:00
b42784d8a0
refactor: address review comments
2021-08-30 16:31:59 -04:00
396bc6a3ad
test: Database startup error when there are multiple active generations
...
Fixes #2196 .
2021-08-30 15:49:12 -04:00
7edb3fd270
fix: time column is alwyas last even if the column is sorted lexicographically
2021-08-30 14:48:18 -04:00
3558e46cae
test: Check deleted databases are omitted from management API list
2021-08-30 14:14:32 -04:00
Nga Tran
Carol (Nichols || Goulding)
Marco Neumann
kodiakhq[bot]
Carol (Nichols || Goulding)
Jacob Marble
dependabot[bot]