Commit Graph

6545 Commits (5d3c9de7aa73483ad89b47c49bc2450e95443784)

Author SHA1 Message Date
Nga Tran 2091007c08 fix: title of next topic 2021-12-17 12:22:06 -05:00
Nga Tran 1c962b05a4 fix: address review comments 2021-12-17 12:18:28 -05:00
kodiakhq[bot] c14674a212
Merge pull request #3393 from influxdata/crepererum/issue3226
refactor: make object store prefixes segment based
2021-12-17 16:46:52 +00:00
kodiakhq[bot] 099d9dc6e1
Merge branch 'main' into crepererum/issue3226 2021-12-17 16:37:16 +00:00
Carol (Nichols || Goulding) 24fd2e549b
docs: Fix some typos and outdated comments 2021-12-17 11:33:36 -05:00
Nga Tran 8be572812d
chore: Apply suggestions from code review
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-12-17 11:23:20 -05:00
kodiakhq[bot] d0f588d3b8
Merge pull request #3391 from influxdata/crepererum/object_store_azure_test
test: `objest_store` azure support via Azurite
2021-12-17 08:39:03 +00:00
Marco Neumann 9f2694bf1b test: `objest_store` azure support via Azurite 2021-12-17 09:30:21 +01:00
Andrew Lamb 64f915d860
fix: flaky end to end system_tables test (#3397) 2021-12-17 08:13:09 +00:00
kodiakhq[bot] c6ddfb4799
Merge pull request #3384 from influxdata/pd/add-big-db-dg-schema
chore: add data generator config for large database
2021-12-16 15:22:42 +00:00
kodiakhq[bot] 32b7f6e658
Merge branch 'main' into pd/add-big-db-dg-schema 2021-12-16 15:11:54 +00:00
Marco Neumann 61ee70d222 refactor: make object store prefixes segment based
Don't use string prefixes, e.g. `foo/bar/` is a prefix of `foo/bar/x`
but NOT of `foo/bar_baz/y`.

This also removes some heuristics during the cloud storage parsing that
assumed that file names always contain a dot but directories don't.

Technically we should now always be able to know whether a path points
to a file or a directory:

- Rust (manually constructed): we use `DirsAndFileName` which knows the
  difference (i.e. if `file_name` is set)
- in-mem store: we also use `DirsAndFileName`
- file system: this was fixed by #1523
  (ccd094dfcf and 464667d8b8)
- cloud: cloud doesn't know about directories. So all paths that these
  APIs return and that end with a `/` are directories (can only occur in
  `list_with_delimiter`); everyting else is a file

Path string representations are now acting occurdingly (i.e. always end
with an `/` if they point to a directory).

Fixes #3226.
2021-12-16 14:40:45 +01:00
Andrew Lamb ccba68fe3e
chore: Update datafusion and sqlparser, and arrow (#3385)
* chore: Update datafusion and sqlparser, and arrow

* fix: cargo hakari

* test: fix metadata size

* fix: update some metadata sizes

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 12:13:40 +00:00
Raphael Taylor-Davies 2f3e25922a
refactor: use redpanda docker image (#3390)
* refactor: use redpanda docker image

* chore: remove unused gpg key
2021-12-16 11:59:10 +00:00
kodiakhq[bot] edcc2b5b99
Merge pull request #3388 from influxdata/crepererum/object_store_aws_test
test: `object_store` aws test using localstack
2021-12-16 11:41:43 +00:00
kodiakhq[bot] 32d8c9bf5f
Merge branch 'main' into crepererum/object_store_aws_test 2021-12-16 11:32:26 +00:00
Marco Neumann 36ba011653 docs: explain CI docker images 2021-12-16 12:31:55 +01:00
Andrew Lamb 499265b01e
chore: add db_name to background worker log messages (#3386)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-16 11:13:13 +00:00
Marco Neumann 5d58b06e64 test: fix some environment variables influencing our tests 2021-12-16 11:17:36 +01:00
Marco Neumann f02d846389 test: `object_store` aws test using localstack
This removes 3 "nonexisting region" tests that where testing very
specific error behavior that no local emulator (minio and localstack)
replicate and that don't add much value. It's better to test our AWS
code at all than being to picky.
2021-12-16 10:10:06 +01:00
kodiakhq[bot] 32ddd08445
Merge pull request #3379 from influxdata/crepererum/issue3375
fix: treat early server worker exit as proper error
2021-12-16 08:39:52 +00:00
Marco Neumann d8810074e8 refactor: better test action CLI 2021-12-16 09:29:25 +01:00
Marco Neumann f4fde15810 refactor: nicer code
Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2021-12-16 09:29:25 +01:00
Marco Neumann ed775431b6 fix: treat early server worker exit as proper error
Instead of just logging the issue, also make sure the error gets
propagated all the way up to the exit code.

Fixes #3375.
2021-12-16 09:29:25 +01:00
Nga Tran d36db7eea2 chore: cleanup 2021-12-15 17:29:33 -05:00
Nga Tran 36b2cb021d fix: typos 2021-12-15 17:26:52 -05:00
Nga Tran e877bc7033 docs: all 3 catalogs 2021-12-15 17:06:12 -05:00
Paul Dix 182032da8a chore: add data generator config for large database 2021-12-15 14:56:57 -05:00
Andrew Lamb da0330fc5f
fix: print newline at end of `database list --detailed` (#3382) 2021-12-15 19:33:24 +00:00
kodiakhq[bot] 9608bb1da2
Merge pull request #3361 from influxdata/crepererum/issue3335
refactor: allow database init to recover from errors
2021-12-15 17:00:20 +00:00
kodiakhq[bot] 290f8c7630
Merge branch 'main' into crepererum/issue3335 2021-12-15 16:50:57 +00:00
Marcus Ilgner 7263c80833
feat(influxdb2_client): add FieldValue::U64 (#3374) 2021-12-15 13:50:30 +00:00
Andrew Lamb 758b65dd29
feat: Add database initialization state and errors to CLI and remove list_databases_detailed gRPC (#3377)
* feat: Add database initialization state and errors to CLI:

* fix: do not use optional in protobuf

* fix: clippy

* fix: correct check I broke appeasing clippy
2021-12-15 12:18:41 +00:00
Marco Neumann e9041e6da0 refactor: improve database init routine
1. Simpler locking/freezing
2. Don't commit the rollback from `ReplayError` to `RulesLoaded` because
   it will confuse the heck out of our users/admins.
2021-12-15 12:43:33 +01:00
Marco Neumann 3841598296 fix: typo
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-12-15 12:41:30 +01:00
Marco Neumann 8520b9b9db fix: improve docs and naming 2021-12-15 12:41:30 +01:00
Marco Neumann 0682fc208f feat: also recover from `NoActiveDatabase` 2021-12-15 12:41:30 +01:00
Marco Neumann 0f3cfb024b test: add database startup recovery test 2021-12-15 12:41:30 +01:00
Marco Neumann e80ac94af5 fix: database state diagram 2021-12-15 12:41:30 +01:00
Marco Neumann 7a1f70f970 fix: move warning to better place 2021-12-15 12:41:30 +01:00
Marco Neumann 11df3091a1 fix: misleading comment 2021-12-15 12:41:30 +01:00
Marco Neumann 496043a767 fix: PR feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-12-15 12:41:30 +01:00
Marco Neumann 10a51bac61 refactor: allow database init to recover from errors
Closes #3335.
2021-12-15 12:41:30 +01:00
Nga Tran fa98b89524 docs: in-memory catalog 2021-12-14 18:33:29 -05:00
Nga Tran 9d2020b906 fix: typo 2021-12-14 14:14:37 -05:00
Nga Tran 2e9362839f docs: initial verstion to test text color 2021-12-14 14:13:43 -05:00
Edd Robinson 7fe6897c59
refactor: add support for handling `TagValuesGroupedByMeasurementAndTagKeyRequest` (#3373)
* chore: update Storage service

* refactor: handle rpc

* refactor: log request fields

* refactor: add new RPC to client

* test: basic test of RPC

* refactor: fmt

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-14 18:42:12 +00:00
kodiakhq[bot] 07ef11dc8c
Merge pull request #3376 from influxdata/hackari
fix: Don't swallow exit code when hakari test fails
2021-12-14 14:44:46 +00:00
Marko Mikulicic b0c6713bd7 fix: Don't swallow exit code when hakari test fails 2021-12-14 14:49:14 +01:00
kodiakhq[bot] 8717bfa747
Merge pull request #3363 from influxdata/er/feat/grpc_show_measurements_pred
feat: wire up general predicates to measurement_names
2021-12-14 12:24:14 +00:00