efe077da8f
Merge branch 'main' into crepererum/issue1313
2021-05-26 14:46:18 +00:00
9f451423d5
feat: log files that are deleted
2021-05-26 12:49:44 +02:00
24ec1a472e
fix: do NOT delete parquet files that are reachable by time travel
2021-05-26 12:38:54 +02:00
c03b8a3963
refactor: remove tables from ChunkSnapshot ( #1295 ) ( #1558 )
2021-05-26 10:37:40 +00:00
1fb6af2364
refactor: split DB background loop into lifecycle and cleanup
...
This should prevent one from blocking / stalling the other.
2021-05-26 11:09:30 +02:00
5983336366
refactor: rename `parquet_file::{utils => test_utils}`
2021-05-26 11:09:29 +02:00
d7e3bc569e
refactor: shorten time we hold the transaction lock during clean-up
2021-05-26 11:04:57 +02:00
18f5dd9ae1
test: ensure transaction lock exists during cleanup planning
2021-05-26 11:04:57 +02:00
b55eae98da
fix: do not delete non-parquet files during catalog-driven cleanup
2021-05-26 11:04:57 +02:00
5ed16ff294
refactor: improve error message in `parquet_file::cleanup`
2021-05-26 11:04:57 +02:00
dd6bbeec42
feat: add background task to clean up OS
...
Closes #1313 .
2021-05-26 11:04:56 +02:00
14fdf3b7c7
feat: implement object store cleanup core routine
2021-05-26 11:02:40 +02:00
cc78b5317d
feat: add method to get all parquet files from catalog state
2021-05-26 11:02:40 +02:00
953114af2e
feat: add method to abort catalog transaction
2021-05-26 11:02:40 +02:00
92fcd7e940
feat: add a way to get OS, server ID and DB name from catalog
2021-05-26 11:02:40 +02:00
9daa4d00d6
test: re-organize `parquet_file` test utils a bit
2021-05-26 11:02:39 +02:00
38183928c8
refactor: extract path generator for data location
2021-05-26 10:59:40 +02:00
4d808f9bdd
Merge pull request #1531 from influxdata/crepererum/in_file_metadata
...
feat: preserve transaction metadata in parquets
2021-05-26 07:44:54 +00:00
166851d952
Merge branch 'main' into crepererum/in_file_metadata
2021-05-26 07:39:53 +00:00
638d754e0f
chore: Update datafusion + other deps ( #1557 )
2021-05-25 22:32:14 +00:00
4fa073af4c
Merge pull request #1555 from influxdata/simpler_routing_config
...
feat: Add simpler RoutingConfig
2021-05-25 19:56:21 +00:00
bae5e5aee3
feat: Add simpler RoutingConfig
2021-05-25 21:51:54 +02:00
ba83ebe35c
Merge pull request #1542 from influxdata/serving_status
...
feat: Implement service readiness API
2021-05-25 14:14:24 +00:00
c01d3d44bf
Merge branch 'main' into serving_status
2021-05-25 14:08:46 +00:00
c2fd85209c
feat: wait for task shutdown on DedicatedExecutor ( #1537 )
2021-05-25 11:33:55 +00:00
0a8ddc843d
Merge pull request #1551 from influxdata/fix-jemalloc-reporting
...
fix: Fix 'active' jemalloc stat misreporting
2021-05-25 08:42:56 +00:00
7a475dc82a
Merge branch 'main' into fix-jemalloc-reporting
2021-05-25 08:38:37 +00:00
7f4b181c67
Merge pull request #1552 from influxdata/jemalloc-config
...
feat: log jemalloc build conf
2021-05-25 08:38:09 +00:00
3f8db50e04
Merge branch 'main' into jemalloc-config
2021-05-25 08:32:56 +00:00
4a94963a4e
Merge pull request #1553 from influxdata/jemalloc-features
...
feat: Use jemalloc everywhere instead of malloc
2021-05-25 08:32:17 +00:00
19a2733d30
feat: preserve transaction metadata in parquets
2021-05-25 09:56:12 +02:00
fe8e6301fe
refactor: move `read_schema_from_parquet_metadata` back to `parquet_file::metadata`
...
Let us pool all metadata handling in a single module, which makes it
easier to review.
2021-05-25 09:37:53 +02:00
ac83d99f66
feat: add a way to get current revision and UUID from transaction handle
2021-05-25 09:37:53 +02:00
fdc553b257
refactor: replace unwrap with expect
2021-05-25 09:37:53 +02:00
0a37cb80f7
feat: Use jemalloc everywhere instead of malloc
...
There may be many reasons for the discrepancy in jemalloc reported allocations total sizes and RSS.
One of them is that our binary doesn't use jmalloc for all the allocations.
Turns out that jemallocator only sets the global rust allocator. Any call to `malloc` will still
go throught the system allocator. Presumably those calls come from linked C code,
but it's also not impossible that not all rust code honours the global allocator (I have no idea, but let's see)
2021-05-25 03:42:52 +02:00
7939d1f027
feat: log jemalloc build conf
2021-05-25 03:30:18 +02:00
a4215f0a56
fix: Fix 'acive' jemalloc stat misreporting
2021-05-25 02:55:27 +02:00
2af6cb78fb
Merge pull request #1550 from influxdata/tran/pushdown_parquet
...
feat: support predicate pushdown for parquet files
2021-05-24 21:29:56 +00:00
018e1e0246
chore: add a comment to trick github to check semantic
2021-05-24 17:25:14 -04:00
40a5d7d4ba
chore: Merge branch 'main' into tran/pushdown_parquet
2021-05-24 16:31:06 -04:00
e72ae81a8e
feat: support predicate pushdown for parquet files
2021-05-24 16:22:52 -04:00
9dfc653bac
Merge pull request #1548 from influxdata/er/refactor/scalar_comp
...
feat: teach Read Buffer to compare literals of different integer types
2021-05-24 17:07:37 +00:00
db96286ed7
Merge branch 'main' into er/refactor/scalar_comp
2021-05-24 17:02:14 +00:00
c464ffadad
refactor: remove special case timestamp_range in parquet chunk ( #1543 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-24 16:19:44 +00:00
9765c53bb4
feat: Implement service readiness API
2021-05-24 17:57:45 +02:00
14ba25f86d
chore: Update datafusion and use released version of arrow crates ( #1546 )
...
* chore: Update datafusion and use released version of arrow crate
* fix: Update for change in API
2021-05-24 15:37:22 +00:00
eace6c9201
fix: ensure scalars compare correctly
2021-05-24 16:19:28 +01:00
abe64c6edc
test: uncomment tests to fix
2021-05-24 16:18:53 +01:00
5c5064bdac
fix: Set default line timestamp and default partition time to same value ( #1512 )
...
* refactor: Rearrange to allow injection of the current time in tests
* test: Failing test showing a point can be in the wrong partition
* fix: Only get the default time once per ShardedEntry creation, in router
2021-05-24 14:55:11 +00:00
27e5b8fabf
refactor: Remove multiple table support from Parquet Chunk ( #1541 )
2021-05-24 08:40:31 -04:00
kodiakhq[bot]
Marco Neumann
Raphael Taylor-Davies
Andrew Lamb
Marko Mikulicic
Nga Tran
Edd Robinson
Carol (Nichols || Goulding)