Edd Robinson
a08246e3c7
Merge branch 'main' into er/refactor/debug_rub_query
2021-12-02 19:42:46 +00:00
Carol (Nichols || Goulding)
0683f851e9
fix: Derive Default instead of doing an equivalent manual impl
...
Identified by clippy. The impl did have `#[cfg(test)]` on it, but the
entire struct does too, so this is fine.
https://rust-lang.github.io/rust-clippy/master/index.html#derivable_impls
2021-12-02 11:52:02 -05:00
Carol (Nichols || Goulding)
5d0fd1c603
fix: Allow dead code on fields that are now detected as never read
2021-12-02 11:52:01 -05:00
Edd Robinson
4a4e11f08b
refactor: adjust debug logging
2021-12-02 12:19:51 +00:00
Nga Tran
f085af034e
refactor: not persist empty chunk resulting from deleting & deduplicating ( #3274 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-01 20:57:30 +00:00
kodiakhq[bot]
e8edab0b43
Merge branch 'main' into ntran/compact_os_preserved_catalog
2021-12-01 18:26:33 +00:00
Nga Tran
de54d2843c
chore: remove unused error
2021-12-01 13:24:12 -05:00
Nga Tran
60807c2d95
refactor: address review comments
2021-12-01 13:11:17 -05:00
Nga Tran
254d35a6c2
refactor: address review comment
2021-12-01 12:25:42 -05:00
Nga Tran
878c59c0d2
chore: cleanup
2021-12-01 10:25:35 -05:00
Nga Tran
1ac32f1d90
refactor: avoid lock gap while updating in-memory catalog
2021-12-01 10:24:58 -05:00
Raphael Taylor-Davies
05064eba62
fix: don't error on delete to non-existent table ( #3259 ) ( #3263 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-12-01 00:40:05 +00:00
Nga Tran
f53cdca010
feat: handling empty compacted stream
2021-11-30 18:13:36 -05:00
Nga Tran
6bb7a796e7
chore: Merge branch 'main' into ntran/compact_os_preserved_catalog
2021-11-30 16:55:38 -05:00
Nga Tran
e8034cdc53
refactor: handle delete predicates for compacting os
2021-11-30 16:45:04 -05:00
Raphael Taylor-Davies
c161d1176a
refactor: cleanup Db write path error handling ( #3258 )
2021-11-30 15:05:02 +00:00
Raphael Taylor-Davies
1e515a1dec
feat: load RUB from object store ( #3224 ) ( #3250 )
2021-11-30 14:39:52 +00:00
Andrew Lamb
20da62eb87
chore: Update `arrow`, `arrow-flight`, and `parquet` to 6.3.0 ( #3254 )
...
* chore: Update `arrow`, `arrow-flight`, and `parquet` to 6.3.0
* fix: update metadata size
2021-11-30 12:30:17 +00:00
Raphael Taylor-Davies
88acf3788e
feat: rebuild catalog ( #3225 ) ( #3253 )
...
* feat: rebuild catalog (#3225 )
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-30 12:17:27 +00:00
Nga Tran
7afc20142b
test: add tests that do actual compacting chunks
2021-11-29 22:59:34 -05:00
Nga Tran
37c4bb402b
feat: update os compacted chunk in both in memory and preserved catalog
2021-11-29 15:17:00 -05:00
Raphael Taylor-Davies
837e029ed8
refactor: cleanup database recovery ( #3225 ) ( #3245 )
...
* refactor: cleanup database recovery (#3225 )
* chore: fix errors
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 18:49:46 +00:00
Raphael Taylor-Davies
0517adf752
feat: test interaction of write buffer and lifecycle ( #2811 ) ( #3211 )
...
* feat: test interaction of write buffer and lifecycle (#2811 )
* chore: more docs
2021-11-29 14:16:43 +00:00
kodiakhq[bot]
23dffefcf8
Merge branch 'main' into crepererum/remove_routing_from_database_mode_7
2021-11-29 11:38:12 +00:00
Raphael Taylor-Davies
197634ed50
feat: reload chunk back into read buffer ( #3209 ) ( #3216 )
...
* feat: reload chunk back into read buffer (#3209 )
* chore: fix logical conflict
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-29 11:34:55 +00:00
Marco Neumann
4e043ecb55
refactor: remove old routing / sharding config
...
This is superseded by the new router subsystem.
2021-11-29 12:33:48 +01:00
dependabot[bot]
3bedf6e62a
chore(deps): bump crc32fast from 1.2.1 to 1.2.2
...
Bumps [crc32fast](https://github.com/srijs/rust-crc32fast ) from 1.2.1 to 1.2.2.
- [Release notes](https://github.com/srijs/rust-crc32fast/releases )
- [Commits](https://github.com/srijs/rust-crc32fast/compare/v1.2.1...v1.2.2 )
---
updated-dependencies:
- dependency-name: crc32fast
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 09:00:01 +00:00
Raphael Taylor-Davies
a286960285
refactor: remove delete predicates from read filter interface ( #3212 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-26 12:15:29 +00:00
Marco Neumann
7f2e4f4342
refactor: remove write buffer direction
...
The direction was required when a database could read or write from/to a
write buffer. Now it is clear from the usage context of a write buffer
context which of the two applications is meant (databases read, routers
write) so the direction flag is no longer required.
2021-11-26 12:38:40 +01:00
Marco Neumann
ac20fecf6f
refactor: remove write buffer producer logic from `Db`
...
As a side effect, writing to database no longer requires `async`.
Closes #2243 .
2021-11-26 12:02:27 +01:00
Marco Neumann
e5bfa08910
feat: disallow unsequenced deletes when reading from write buffer
...
Similar to how writes to a databases that reads from a write buffer are
forbidden deletes should be rejected as well.
2021-11-26 11:40:08 +01:00
Marco Neumann
22936abb23
refactor: remove connection manager and resolver from `Server`
2021-11-26 11:00:19 +01:00
Marco Neumann
0e06026fbd
refactor: remove write routing logic from `Server`
...
API users should use `Database` instead, there's no need any longer to mirror
this API in `Server`. Note that `Database` is better than `Db` in this
case, because the former can also check if we're writing from a write
buffer and can easily reject unsequenced writes. The `pub` modifiers
were adjusted to make it impossible to write through `Db` directly.
2021-11-26 09:26:43 +01:00
Nga Tran
4e7e9f6fc9
chore: Merge branch 'ntran/compact_os_blocks' of https://github.com/influxdata/influxdb_iox into ntran/compact_os_blocks
2021-11-23 16:56:59 -05:00
Nga Tran
b2b033f00c
chore: rerun fmt after merging
2021-11-23 16:56:38 -05:00
kodiakhq[bot]
1d1622efa3
Merge branch 'main' into ntran/compact_os_blocks
2021-11-23 21:47:05 +00:00
Nga Tran
ade7ea6cfb
chore: Merge branch 'ntran/compact_os_blocks' of https://github.com/influxdata/influxdb_iox into ntran/compact_os_blocks
2021-11-23 16:44:48 -05:00
Nga Tran
a0fb6a4758
chore: cleanup
2021-11-23 16:44:30 -05:00
Nga Tran
4fc09484ed
chore: Apply suggestions from code review
...
Co-authored-by: Raphael Taylor-Davies <1781103+tustvold@users.noreply.github.com>
2021-11-23 16:43:12 -05:00
Nga Tran
4e28b4c43b
refactor: address review comments
2021-11-23 16:39:45 -05:00
Nga Tran
e64ac10c40
refactor: address review comments and compute max checkpoints
2021-11-23 15:32:46 -05:00
Raphael Taylor-Davies
42e3a41825
fix: flaky test_chunk_access_time ( #3197 ) ( #3200 )
...
* fix: flaky test_chunk_access_time (#3197 )
* chore: fix lint
Co-authored-by: Edd Robinson <me@edd.io>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-23 17:39:57 +00:00
Raphael Taylor-Davies
bfa4bfe735
refactor: lifecycle observability cleanup ( #2286 ) ( #3189 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-23 15:42:24 +00:00
Marco Neumann
f9110eecad
test: migrate `DistibutedTest` to use proper router
2021-11-23 15:39:53 +01:00
Nga Tran
124f5bb4c5
chore: make comments clearer
2021-11-22 23:30:18 -05:00
Nga Tran
8788dd6d39
chore: fix comment
2021-11-22 23:24:28 -05:00
Nga Tran
3182e935ed
chore: cleanup
2021-11-22 23:22:34 -05:00
Nga Tran
06aba60068
feat: another piece: tests for contiguous chunks and read IOx metadata to get checkpoints
2021-11-22 23:15:20 -05:00
kodiakhq[bot]
29aaafe808
Merge branch 'main' into ntran/compact_os_catalog
2021-11-22 19:19:57 +00:00
Nga Tran
5e45c5a349
refactor: add meessage to assert
2021-11-22 13:28:08 -05:00
Nga Tran
731dee4e91
refactor: address review comments
2021-11-22 13:18:54 -05:00
kodiakhq[bot]
d16a7759ca
Merge branch 'main' into cn/workspace-hack
2021-11-22 17:05:31 +00:00
kodiakhq[bot]
b5d6e201e6
Merge branch 'main' into ntran/compact_os_framework
2021-11-22 15:17:13 +00:00
Nga Tran
09fd25d28e
refactor: address to cleanup-lock issue
2021-11-22 10:15:43 -05:00
Raphael Taylor-Davies
0b55cee5f2
feat: skip applying deletes to persisted chunks during replay ( #3125 ) ( #3153 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-22 10:38:25 +00:00
Raphael Taylor-Davies
73d60539ad
refactor: use ChunkGenerator in parquet_catalog ( #2209 ) ( #3167 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-22 10:29:33 +00:00
Nga Tran
3b2e5c445b
chore: Merge branch 'main' into ntran/compact_os_framework
2021-11-19 16:18:24 -05:00
Nga Tran
40ddcf2a2f
refactor: address review comments
2021-11-19 16:08:01 -05:00
Raphael Taylor-Davies
ff78f5a4f5
feat: reduce cardinality of job metrics ( #3169 )
...
* feat: reduce cardinality of job metrics
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 19:05:49 +00:00
Andrew Lamb
76d02deb4c
refactor: form `system.column_chunks` column by column rather than row by row ( #3155 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 18:34:29 +00:00
Carol (Nichols || Goulding)
9fd4a560f5
feat: Results of running cargo hakari manage-deps
2021-11-19 09:21:57 -05:00
Carol (Nichols || Goulding)
25d55cd08a
feat: Move server config paths beneath 'nodes' ( #3144 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 09:54:32 +00:00
Raphael Taylor-Davies
e32d367e85
feat: flush delete mailbox on persist ( #3126 ) ( #3147 )
...
* feat: flush delete mailbox on persist (#3126 )
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-19 09:45:29 +00:00
Nga Tran
c148251dcb
feat: implement step2: compact and persist os chunks
2021-11-18 18:18:55 -05:00
Carol (Nichols || Goulding)
a2454b542d
fix: Small cleanups in Cargo.tomls ( #3160 )
...
* fix: Add tokio rt-multi-thread feature so cargo test -p client_util compiles
* fix: Alphabetize dependencies
* fix: Add the data_types_conversions feature to get tests passing
* fix: Remove dev dependencies already listed under normal dependencies
* fix: Make sure the workspace is using the new resolver
2021-11-18 22:26:33 +00:00
kodiakhq[bot]
c9f02f83e7
Merge branch 'main' into cn/alias-db-commands
2021-11-18 15:13:43 +00:00
Nga Tran
ccef3b535a
feat: clean up and add comments for next steps
2021-11-18 10:11:51 -05:00
Raphael Taylor-Davies
714fc85c8d
refactor: extract Mailbox type ( #3126 ) ( #3142 )
...
* refactor: extract Mailbox type (#3126 )
* fix: doc
* chore: review feedback
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-11-18 09:34:06 +00:00
Nga Tran
a5c04e5fe4
feat: framework for compact os chunks
2021-11-17 18:12:51 -05:00
Carol (Nichols || Goulding)
f69d37e9a8
fix: Remove database delete/restore entirely
2021-11-17 12:03:11 -05:00
Carol (Nichols || Goulding)
7783e4a7ff
fix: Make delete/restore aliases for release/claim; remove tombstone
...
Fixes #2680
2021-11-17 11:41:08 -05:00
Raphael Taylor-Davies
8155747735
feat: add write buffer delete encoding ( #2731 ) ( #3127 )
...
* feat: add write buffer delete encoding (#2731 )
* chore: fix doc
* chore: review feedback
* chore: review feedback
* chore: fmt
* chore: review feedback
2021-11-17 16:12:19 +00:00
Andrew Lamb
47acd181c5
chore: Update datafusion + arrow/parquet/arrow-flight 6.2.0 ( #3136 )
...
* chore: Update datafusion and arrow
* chore: update arrow/parquet/arrow-flight to 6.2.0
* refactor: Add table_exists to SchemaProvider impl
* fix: clippy
* fix: clippy 2
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-17 14:04:49 +00:00
Marco Neumann
e6fdd79a0f
feat: emit Kafka stats as metrics instead of logs
...
This maps a subset of Kafka stats as metrics. The set can -- of course
-- be changed in the future depending on our needs.
Fixes #3100 .
2021-11-16 17:18:41 +01:00
Raphael Taylor-Davies
553e412226
refactor: DMLOperation write path ( #2731 ) ( #3121 )
...
* refactor: DMLOperation write path (#2731 )
* chore: fmt
* chore: review feedback
2021-11-16 12:42:19 +00:00
Carol (Nichols || Goulding)
bc11244828
feat: Rename database disown/adopt to release/claim ( #3111 )
...
* fix: Rename 'disown' to 'release' database
Connects to #3110
* fix: Rename 'adopt' to 'claim' database
Fixes #3110 .
2021-11-15 20:28:09 +00:00
Jake Goulding
af28cfa2a6
feat: Add an adopt database API
...
Fixes #2679 .
2021-11-15 09:26:06 -05:00
Raphael Taylor-Davies
3d091208af
refactor: move delete predicate into data_types ( #2731 ) ( #3094 )
...
* refactor: move delete predicate into dml (#2731 )
* refactor: move DeletePredicate to data_types
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:28:58 +00:00
Raphael Taylor-Davies
a6d83a3026
feat: WriteBufferReader use DmlOperation ( #2731 ) ( #3096 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-15 10:19:54 +00:00
Carol (Nichols || Goulding)
48f36fab1f
fix: Box an error so the enum variants don't differ greatly in size
2021-11-12 14:53:07 -05:00
kodiakhq[bot]
376471591f
Merge branch 'main' into cn/disown
2021-11-12 19:31:03 +00:00
Carol (Nichols || Goulding)
46c96940f0
feat: Improve log messages to be more specific
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-11-12 14:21:44 -05:00
Carol (Nichols || Goulding)
63ee5b8317
fix: Shut down databases before deleting or disowning them
2021-11-12 13:37:24 -05:00
Carol (Nichols || Goulding)
56c16414a1
fix: Use shared application time rather than passing in a timestamp
2021-11-12 11:32:02 -05:00
Carol (Nichols || Goulding)
315cc3836b
fix: Combine checks of the database state
2021-11-12 11:17:17 -05:00
Carol (Nichols || Goulding)
8bdfb253ce
fix: Remove context from disown operation
2021-11-12 11:10:20 -05:00
Carol (Nichols || Goulding)
2b8f0b7eeb
fix: Don't hardcode timestamp in restore code
2021-11-11 11:44:19 -05:00
Raphael Taylor-Davies
6f268f8260
refactor: extract DML types ( #2731 ) ( #3084 )
...
* refactor: extract DML types (#2731 )
* chore: fmt
2021-11-11 12:34:07 +00:00
Carol (Nichols || Goulding)
7e13cb77ac
feat: add transactions for context to database owner info on disown
2021-11-10 16:49:32 -05:00
Carol (Nichols || Goulding)
fb7bde527f
feat: Implement disown in Server and Database
2021-11-10 16:49:32 -05:00
Carol (Nichols || Goulding)
489fad0040
feat: Add disown database to the gRPC API
2021-11-10 16:49:31 -05:00
Carol (Nichols || Goulding)
e160954b57
refactor: Extract functions for creating and updating owner info to match the fetching function
2021-11-10 16:49:31 -05:00
Carol (Nichols || Goulding)
d42c890333
refactor: Extract a function for reading a database's owner file
2021-11-10 16:49:31 -05:00
Carol (Nichols || Goulding)
3dda694520
refactor: Extract a macro for matching errors in tests
2021-11-08 11:14:23 -05:00
Carol (Nichols || Goulding)
c8d80e5c28
fix: Change database paths to be under /dbs/ instead of under /[server id]/
2021-11-05 10:14:06 -04:00
Raphael Taylor-Davies
898567e221
feat: migrate server to DbWrite ( #2724 ) ( #3035 )
...
* feat: migrate server to DbWrite (#2724 )
* chore: print perf log output
* fix: don't suppress CI status code
* chore: review feedback
* fix: don't error on empty line protocol write payloads
* fix: test
* fix: test
2021-11-05 11:09:33 +00:00
Raphael Taylor-Davies
06dfdc4af8
feat: simplify sharding ( #3031 )
...
* feat: simplify sharding
* chore: fix lint
2021-11-04 16:57:19 +00:00
Raphael Taylor-Davies
d28749bd93
refactor: remove Entry from Db interfaces ( #2724 ) ( #3027 )
2021-11-04 13:28:47 +00:00
Raphael Taylor-Davies
07ba629e2b
feat: migrate write buffer producer to MutableBatch and pbdata ( #2743 ) ( #3021 )
...
* feat: migrate write buffer producer to MutableBatch and pbdata (#2743 )
* fix: Kafka message content type header
* chore: fix doc
2021-11-04 10:20:40 +00:00
Andrew Lamb
46213a5a28
chore: update dependencies
2021-11-03 17:00:20 -04:00
Carol (Nichols || Goulding)
0f62bb1686
fix: Distinguish between restoring twice and name collisions on restore
...
And return them in the correct situations.
2021-11-03 15:02:22 -04:00
Carol (Nichols || Goulding)
6615001dc7
fix: Extract loading the database name out of the rules file into a function
2021-11-03 15:02:21 -04:00
Carol (Nichols || Goulding)
3e44f17698
fix: Return a useful error on attempt to restore an active database
2021-11-03 15:02:18 -04:00
Carol (Nichols || Goulding)
207d701bdb
fix: Have DatabaseState keep track of the UUID after reading it from object storage
2021-11-03 15:01:27 -04:00
Carol (Nichols || Goulding)
b1504f17ed
fix: Detangle UUID from ProvidedDatabaseRules
2021-11-03 15:01:27 -04:00
Carol (Nichols || Goulding)
f2cb79bfa0
test: Add more tests around database restore
2021-11-03 15:01:26 -04:00
Carol (Nichols || Goulding)
62e7376394
fix: Restore database should only need UUID, not database name too
2021-11-03 15:01:26 -04:00
kodiakhq[bot]
bbdfc47b02
Merge branch 'main' into crepererum/warn_on_failed_wb_reads
2021-11-03 17:34:14 +00:00
Marco Neumann
10de63f9b6
refactor: warn when we receive broken data from write buffer
...
We need to understand why this is the case and where broken data gets
stuck, because at the moment it seems that some Kafka hiccups can
poison the write buffer but it seems restarting IOx recovers the
situation.
2021-11-03 16:03:42 +01:00
Raphael Taylor-Davies
873ce27b0c
feat: reduce lock metric cardinality ( #3013 )
2021-11-02 17:57:02 +00:00
Marco Neumann
0d0c0cb42b
refactor: move write buffer configs to new home
...
Write buffer configs will partially be shared by database and router
nodes, so lets move them into a shared home.
2021-11-02 10:17:01 +01:00
Raphael Taylor-Davies
f1a6468e7b
feat: migrate write buffer consumer to use DbWrite ( #2724 ) ( #3003 )
...
* feat: migrate write buffer consumer to use DbWrite (#2724 )
* fix: doc
* chore: fmt
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-11-01 16:38:48 +00:00
kodiakhq[bot]
c724bb2ee8
Merge branch 'main' into dependabot/cargo/tokio-1.13.0
2021-11-01 11:30:33 +00:00
dependabot[bot]
c540b40f05
chore(deps): bump tokio from 1.12.0 to 1.13.0
...
Bumps [tokio](https://github.com/tokio-rs/tokio ) from 1.12.0 to 1.13.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.12.0...tokio-1.13.0 )
---
updated-dependencies:
- dependency-name: tokio
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 11:21:59 +00:00
kodiakhq[bot]
fd5d053d33
Merge branch 'main' into er/feat/read_buffer/meta_delete
2021-11-01 10:36:01 +00:00
Edd Robinson
b1096d2a36
Merge branch 'main' into er/feat/read_buffer/meta_delete
2021-11-01 10:26:10 +00:00
dependabot[bot]
a1f978e280
chore(deps): bump tokio-util from 0.6.8 to 0.6.9
...
Bumps [tokio-util](https://github.com/tokio-rs/tokio ) from 0.6.8 to 0.6.9.
- [Release notes](https://github.com/tokio-rs/tokio/releases )
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-util-0.6.8...tokio-util-0.6.9 )
---
updated-dependencies:
- dependency-name: tokio-util
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com>
2021-11-01 01:28:41 +00:00
Raphael Taylor-Davies
6ceab054ab
refactor: move DbWrite to mutable_batch ( #2986 )
...
* refactor: move DbWrite to mutable_batch
* chore: fix doc
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-29 15:13:05 +00:00
kodiakhq[bot]
fc25e313c6
Merge branch 'main' into cn/iox-obj-sto-refactors
2021-10-29 13:53:26 +00:00
Carol (Nichols || Goulding)
aefdb00bc2
fix: Remove empty test
2021-10-29 09:47:49 -04:00
Carol (Nichols || Goulding)
c02e476175
fix: Split database name from rules at the last possible function
...
When updating database rules, the database name should come from the
rules. Enforce this by only taking the rules as a parameter, rather than
having two arguments that could possibly disagree with each other.
2021-10-29 09:46:23 -04:00
Andrew Lamb
4675dd5331
docs: Add some doc comments ( #2984 )
2021-10-29 11:21:39 +00:00
Raphael Taylor-Davies
8a2410e161
feat: mutable batch write entry ( #2724 ) ( #2973 )
...
* feat: mutable batch write entry (#2724 )
* chore: lint
* chore: review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-28 20:15:28 +00:00
Carol (Nichols || Goulding)
c61806f919
fix: Apply commenting and logging suggestions from code review
...
Co-authored-by: Andrew Lamb <alamb@influxdata.com>
2021-10-28 15:32:33 -04:00
Carol (Nichols || Goulding)
f8b5886110
test: Add more test coverage for new behavior
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding)
d2bd8f05db
fix: Clean up error messages and comments referencing generations
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding)
0bccd35f8c
fix: List details about databases only prints active dbs and their UUIDs
...
We can no longer list deleted databases because the server no longer
knows about them, and we now have UUIDs that are useful to know about,
so change the detailed listing of databases to return UUID instead of
possible deleted at time.
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding)
ce55cf401c
feat: Return UUID when creating a database
...
Seems polite and is useful for some test setup
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding)
4077d3a20a
fix: Take a UUID when restoring a database
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding)
95ba84edca
fix: Return UUID when a database is deleted
...
So that it can be immediately used to restore elsewhere.
2021-10-28 13:20:29 -04:00
Carol (Nichols || Goulding)
e80b902d15
test: Delete tests either irrelevant or covered elsewhere
...
Now that the database paths use UUIDs instead of names+generation ids,
and that IoxObjectStore checks there are no files in a directory before
it'll create a database directory (and has tests for that).
2021-10-28 13:20:28 -04:00
Carol (Nichols || Goulding)
990f768cda
fix: Assign a UUID when creating a database
2021-10-28 13:20:28 -04:00
Carol (Nichols || Goulding)
f160712f5e
fix: Remove the concept of generations
...
From iox_object_store and everywhere that fails to compile as a result.
Also make path handling in iox_object_store a bit more consistent.
2021-10-28 13:20:28 -04:00
Carol (Nichols || Goulding)
8198c1ff2a
refactor: Rename IoxObjectStore constructors to better match what server does with Databases
2021-10-28 13:20:27 -04:00
kodiakhq[bot]
04b4eeb6ec
Merge branch 'main' into cn/only-read-server
2021-10-25 15:15:17 +00:00
Marco Neumann
bc7244c48e
chore: use Rust edition 2021
2021-10-25 10:58:20 +02:00
Marco Neumann
cd30e6beab
refactor: use `SpanRecorder` instead of hand-rolled logic
2021-10-25 10:16:05 +02:00
Marco Neumann
f3cc238d1c
feat: create span contexts on write buffer consumer side
2021-10-25 10:04:08 +02:00
Andrew Lamb
52cf1a85b9
fix(metadata): Do not report table_names for tables that have no non-null values that match predicate ( #2947 )
...
* fix(metadata): Do not report table_names for tables that have no non-null values that match predicate
* fix: make apply_predicate_to_metadata precise
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-24 11:03:30 +00:00
Carol (Nichols || Goulding)
f69a470b90
refactor: Use TestDbBuilder to rebuild db in replay tests ( #2950 )
...
* refactor: Use TestDbBuilder to rebuild db in replay tests
* refactor: Organize uses
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-23 10:42:05 +00:00
Carol (Nichols || Goulding)
98e84cd624
feat: Only read the server config to list the databases a server owns
2021-10-22 15:56:46 -04:00
kodiakhq[bot]
9b70001aa4
Merge branch 'main' into cn/remove-list-deleted-databases
2021-10-22 19:36:39 +00:00
Carol (Nichols || Goulding)
948cf92aaa
fix: Remove the ability to list deleted databases
...
Once we're relying on the server config file to know about databases,
IOx shouldn't be listing files in object storage to try and find deleted
databases.
When the disown API is implemented for the floating database design, it
will return the UUID of the database just disowned, so that it can
immediately be used in an adopt API call.
2021-10-22 14:50:06 -04:00
Carol (Nichols || Goulding)
b4506c2f72
docs: Clean up some comments
...
Remove some that weren't accurately describing the code, fix some line
wrapping
2021-10-22 14:34:57 -04:00
Carol (Nichols || Goulding)
739fb075a7
refactor: Organize uses
2021-10-22 14:34:43 -04:00
Carol (Nichols || Goulding)
0c3ace5573
refactor: Call TestDbBuilder.build again instead of making a new builder
2021-10-22 14:32:32 -04:00
Carol (Nichols || Goulding)
e838a22f92
refactor: Make TestDbBuilder reusable for rebuilding databases
...
Many tests want to exercise behavior that happens on reloading or
replay of a database. This changes the way the TestDbBuilder works to
make it possible to call `build` on a builder multiple times to create
databases with the same options to simulate reload of a database without
having to keep track of which pieces need to match.
2021-10-22 14:31:31 -04:00
kodiakhq[bot]
f4eb3884f3
Merge branch 'main' into cn/write-db-owner-file
2021-10-20 13:51:45 +00:00
Raphael Taylor-Davies
ce0127a1f7
feat: MutableBatch write API ( #2090 ) ( #2724 ) ( #2882 )
...
* feat: MutableBatch write API (#2090 ) (#2724 )
* chore: fix lint
* fix: handle dictionaries with unused mappings
* chore: review feedback
* chore: further review feedback
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-20 08:44:14 +00:00
Andrew Lamb
a82dc6f5f0
chore: Update datafusion + arrow ( #2903 )
...
* chore: Update datafusion to latest, arrow to 6.0.0
* fix: Update tests
* fix: bubble internal error
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-10-19 17:14:08 +00:00