Commit Graph

10 Commits (a686c2ab028c9f6f98e48f9d26ea6a59f494bc9d)

Author SHA1 Message Date
Dom Dwyer 44e9eaf92b test(iox_catalog): isolated catalog tests
This commit changes the iox_catalog test harness so that each test is
run in an independent, randomly generated schema to avoid concurrent
tests interfering with each other.

Each test creates a randomly-named schema, grants permissions to the new
schema, runs the full migration stack against the new schema, and then
executes the code under test.
2022-02-17 14:19:01 +00:00
Luke Bond a66e29e5b3
chore: port sqlx-hotswap-pool over from conductor (#3750)
* chore: port sqlx-hotswap-pool over from conductor

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>

* chore: workspace hack fixes

* fix: unique schema per test db connection

* fix: adjust search path in catalog pg tests to see if it fixes test schema issue

* fix: actually fixed sqlx hotswap pool test

Co-authored-by: Marko Mikulicic <mkm@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-15 16:18:36 +00:00
Marco Neumann 5de4d6203f
refactor: catalog transaction (#3660)
* refactor: catalog Unit of Work (= transaction)

Setup an inteface to handle Units of Work within our catalog. Previously
both the Postgres and the in-mem backend used "mini-transactions on
demand". Now the caller has a clear way to establish boundaries and
gets read and write isolation. A single `Arc<dyn Catalog>` can create as
many `Box<dyn UnitOfWork>` as you like, but note that depending on the
backend you may not scale infinitely (postgres will likely impose
certain limits and the in-mem backend limits concurrency to 1 to keep
things simple).

* docs: improve wording

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>

* refactor: rename Unit of Work to Transaction

* test: improve `test_txn_isolation`

* feat: clearify transaction drop semantics

Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-02-08 13:38:33 +00:00
Dom 9201023ea4
feat: schema validation for MutableBuffer instances (#3554)
* refactor: Debug bounds on Catalog trait

* feat: validate MutableBatch schema

Changes the schema validation code to validate MutableBatch instances
(coming from a pre-parsed LP write, and non-LP-based writes) instead of
parsed LP lines.

* refactor: Send bound on boxed errors

* refactor: clippy

Allow assert_eq!(bool, bool) for readability.

* refactor: no PartialEq<MB Column> for ColumnSchema

Remove the PartialEq<mutable_buffer::Column> for ColumnSchema - it's
definitely more readable as a method call.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 20:55:18 +00:00
Dom ce568ab447
build(iox_catalog): remove unused dependencies (#3552)
* build: don't pull in all of tokio

We already specify the tokio features we need so "full" (all features)
is not necessary.

* build: remove chrono dependency

Appears unused.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-01-27 15:14:42 +00:00
Paul Dix 41038721e1 feat: Add parquet file records to iox_catalog
* Adds ParquetFile and scaffolding to IOx catalog
* Changed the file_location in parquet_file to object_store_id which is a uuid
2022-01-19 14:14:54 -05:00
Paul Dix d4d4b4c087 fix: workspace hack after adding iox_catalog crate 2022-01-17 10:04:24 -05:00
Paul Dix 4f908021e4 chore: Update iox_catalog Postgres test to only run on integration 2022-01-17 09:56:20 -05:00
Paul Dix 4e5bca93fb chore: update iox_catalog to Snafu 0.7 2022-01-17 09:56:20 -05:00
Paul Dix 4764e71c54 feat: Add initial iox_catalog skeleton 2022-01-17 09:56:20 -05:00