3a150594ab
refactor: push chunk and partition summary into catalog ( #1103 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-01 13:34:21 +00:00
a59a6edbb8
feat: add SHOW TABLES and `select * from information_schema.columns` ( #1102 )
...
* chore: update dependencies
* test: add tests for new information schema
2021-04-01 12:50:37 +00:00
0d599eeaf4
refactor: remove Db::writeable ( #1101 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-01 11:45:40 +00:00
b0e21e5f9e
feat: automated chunk lifecycle ( #1091 )
...
* feat: automated chunk lifecycle
* chore: use >= for lifecycle timestamp comparisons
* chore: review fixes
2021-04-01 11:05:58 +00:00
8f69a12ec6
refactor: Add rwlock around db rules
2021-04-01 10:43:46 +00:00
830676d843
chore: Add the serde feature to bytes dep
2021-04-01 09:00:29 +00:00
bfd73b868a
Merge pull request #1080 from influxdata/pd-write-entry-fbs
...
feat: add new flatbuffer definitions for entries
2021-03-31 17:18:10 +00:00
fb57a61b81
Merge branch 'main' into pd-write-entry-fbs
2021-03-31 17:06:14 +00:00
3a7c023d35
chore: update entry flatbuffer null mask
2021-03-31 12:43:01 -04:00
b61875e0b2
feat: Add TableSummary calculation to ReadBuffer ( #1092 )
2021-03-31 16:26:37 +00:00
abc68ba4bf
feat: update flatbuffer table batch to column oriented
2021-03-31 11:52:06 -04:00
569f90d937
feat: Add ability to get PartitionSummary statistics from a Db ( #1090 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-31 14:18:53 +00:00
f77164c9c3
Merge pull request #1065 from influxdata/cn/here-we-go-round-again
...
chore: Move regular CI to CircleCI
2021-03-31 14:04:46 +00:00
7c6d2bdd18
Merge branch 'main' into cn/here-we-go-round-again
2021-03-31 13:50:49 +00:00
13612047c0
feat: chunk sorting ( #1077 )
2021-03-31 12:30:59 +00:00
209d1ec0c8
feat: add entry flatbuffer generated code
2021-03-30 18:57:50 -04:00
b7d5087639
feat: add consistency high water mark to segment
2021-03-30 18:57:50 -04:00
66b50be1d5
feat: add new flatbuffer definitions for entries
2021-03-30 18:57:50 -04:00
7154dfd5f6
feat: Add timestamps to ChunkSummary ( #1079 )
...
* refactor: Move timestamps from mutable_buffer::Chunk to catalog::Chunk
* feat: Add timestamps to ChunkSummary
* feat: Add timestamp conversion logic to protobuf types
* test: Add tests
* fix: Update data_types test
* fix: handle negative nanos during conversion
* fix: clippy
* fix: more clippy
* fix: even more clippy
* fix: even more clippy
2021-03-30 19:03:23 +00:00
a51b1008a2
Merge pull request #1078 from influxdata/ntran/for1001
...
feat: make it easy to get OperationMetadata from Operation
2021-03-30 17:30:40 +00:00
a630c119ab
feat: make it easy to get OperationMetadata from Operation
2021-03-30 12:57:11 -04:00
d8d5b1568a
feat: automated chunk closing ( #1074 )
...
* feat: automated chunk closing
* chore: remove unused error variant
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 15:31:09 +00:00
569099fc6e
feat: Derive serde for protos
...
Rationale
---------
Our CLI needs to be able to accept configuration as JSON and render configuration as JSON.
Protobufs technically have an official JSON encoding rule called 'jsonpb` but prost doesn't
offer native supprot for it.
`prost` allows us to specify arbitrary derive metadata to be added to generated
code. We emit the `serde` derive directives in the two packages that generate prost code
(`generated_types` and `google_types`).
We use the `serde(rename_all = "camelCase")` to approximate `jsonpb`.
We instruct `prost` to use `bytes::Bytes` for some types, hence we must turn on the `serde` feature
on the `bytes` dependency.
We also use json to serialize the output of the `database get` command, to showcase the feature
and get rid of a TODO. In a subsequent PR I'll teach `database create` (and the yet to be done `database update`) to accept an option JSON configuration body so we can configure partitioning, lifecycle, sharding etc rules etc.
Caveats
-------
This is not technically `jsonpb`. Main issues:
1. default values not omitted
2. no special rendering of special types like `google.protobuf.Any`
Future work
-----------
Figure out if we can get fully compliant `jsonpb`, or at least a decent approximation.
Effect
------
```console
$ cargo run -- database get foobar_weather
{
"name": "foobar_weather",
"partitionTemplate": {
"parts": [
{
"part": {
"time": "%Y-%m-%d %H:00:00"
}
}
]
},
"lifecycleRules": {
"mutableLingerSeconds": 0,
"mutableMinimumAgeSeconds": 0,
"mutableSizeThreshold": 0,
"bufferSizeSoft": 0,
"bufferSizeHard": 0,
"sortOrder": {
"order": 2,
"sort": {
"createdAtTime": {}
}
},
"dropNonPersisted": false,
"immutable": false
},
"walBufferConfig": null,
"shardConfig": {
"specificTargets": null,
"hashRing": null,
"ignoreErrors": false
}
}
```
2021-03-30 15:16:31 +00:00
bbfc49bacd
Merge pull request #1072 from influxdata/alamb/show_tables
...
feat: Add support for listing tables via information_schema.tables
2021-03-30 13:52:12 +00:00
f0b411cd43
feat: enable information_schema
2021-03-30 09:01:43 -04:00
865910ae5b
chore: update dependencies
2021-03-30 09:01:35 -04:00
0b7dea09d5
refactor: Remove `MutableBufferDb` and Partition ( #1068 )
...
* refactor: Move test for missing tags as null to db
* refactor: move last_write_at and create_time to catalog partition
* refactor: remove mutable buffer partition and port tests
* fix: port over partition sorting logic
* fix: remove stale comment
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 12:51:37 +00:00
15e1f9c6ec
refactor: use optional non-zero types ( #1071 )
2021-03-30 11:53:54 +00:00
18f3124721
refactor: remove DbCatalog newtype wrapper ( #1067 )
...
* refactor: remove DbCatalog newtype wrapper
* chore: review comments
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 07:52:16 +00:00
c30eaffe94
Merge pull request #998 from jeivardan/feature-setup
...
feat: Add support for setup API to influxdb_client
2021-03-29 20:38:06 +00:00
d01ed8dc1a
Merge branch 'main' into feature-setup
2021-03-29 20:25:13 +00:00
6a48001d13
refactor: Manage storage directly in the Catalog ( #1057 )
...
* refactor: Manage mutable buffer chunks directly
* fix: do not use mutable_buffer for listing table names
2021-03-29 17:55:07 +00:00
9d2fb2ad7f
chore: Move regular CI to CircleCI
2021-03-29 12:55:56 -04:00
1a4c793270
feat: lifecycle rules ( #1045 )
2021-03-29 15:32:36 +00:00
eb0122655d
refactor: Remove async from PartitionChunk ( #1062 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-29 13:00:36 +00:00
02ae743e8e
refactor: Remove async from Database ( #1063 )
2021-03-29 12:48:12 +00:00
fb130ea99d
feat: use CatalogProvider and SchemaProvider ( #1058 )
...
* feat: use CatalogProvider and SchemaProvider
* refactor: review comments
2021-03-29 11:08:46 +00:00
b33d97c946
refactor: write nanosecond timestamps to parquet ( #1030 )
...
* feat: write nanosecond timestamps to parquet
* chore: update arrow deps
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-27 19:00:57 +00:00
0ca9ad7285
refactor: Remove async from `PartitionChunk::table_schema` ( #1060 )
2021-03-27 18:08:12 +00:00
663d4fb6f7
docs: Use Scan rather than InMemoryScan for clarity ( #1049 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-26 14:22:49 +00:00
98fe75c989
feat: Add ShardConfig to DatabaseRules proto
2021-03-26 14:07:59 +00:00
790e5d3348
refactor: Add fine grained (object level) catalog locking ( #1053 )
...
* refactor: inline catalog crate to server
* refactor: Add fine grained (object level) catalog locking
* fix: Move mod definition and use to top of file
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-26 13:48:24 +00:00
cd94544e93
feat: Add prototxt script
2021-03-26 12:40:56 +00:00
82cabae571
fix: Remove unnecessary unwrap
...
Replace Option<bool> with bool
Signed-off-by: Aakash Hemadri <aakashhemadri123@gmail.com>
2021-03-26 09:41:17 +05:30
5750a69c75
Merge pull request #1050 from influxdata/ntran/1048
...
fix: support upper case for scientific notation
2021-03-25 17:33:34 +00:00
c15a2251f5
Merge branch 'main' into ntran/1048
2021-03-25 17:21:31 +00:00
dd3a6a2d2e
refactor: make a tuple for 4 choices
2021-03-25 13:20:32 -04:00
5dfa84f0bb
fix: add e- and E-
2021-03-25 12:48:16 -04:00
1b3fa7c5e8
fix: increase timing tolerance for tracker timing test ( #1051 ) ( #1052 )
...
* fix: increase timing tolerance for tracker timing test (#1051 )
* fix: fmt
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
2021-03-25 15:09:09 +00:00
31389f88df
refactor: use alt to reduce code line
2021-03-25 11:02:08 -04:00
Raphael Taylor-Davies
Andrew Lamb
Marko Mikulicic
kodiakhq[bot]
Paul Dix
Nga Tran
Andrew Lamb
Carol (Nichols || Goulding)
Wakahisa
Aakash Hemadri