cf93c91973
Merge pull request #1111 from influxdata/jgm-metrics
...
feat: add Prometheus metrics endpoint
2021-04-02 16:59:22 +00:00
89205bb3c6
chore: improve comments and simplify a bit
2021-04-02 09:43:56 -07:00
e885cf072e
feat: add Prometheus metrics endpoint
...
This change adds HTTP endpoint /metrics. Now, I need to add a metric.
2021-04-02 09:35:16 -07:00
90e13a7f89
Merge pull request #1108 from influxdata/ntran/parquet_write_2
...
feat: add 2 more chunk states for writing chunks to object store
2021-04-02 14:57:56 +00:00
5a04c85da5
Merge branch 'main' into ntran/parquet_write_2
2021-04-02 14:43:46 +00:00
6cc84c8a50
Merge pull request #1110 from influxdata/cn/trace-together
...
chore: Group all tracing-related crates under one crate for easier upgrade management
2021-04-02 14:41:28 +00:00
0b880d3534
chore: Group all tracing-related crates under one crate for easier upgrade management
2021-04-02 09:54:39 -04:00
eba799e4e7
Merge branch 'main' into ntran/parquet_write_2
2021-04-01 21:54:19 +00:00
839fe514a2
fix: do not make rows for empty partitions or empty tables
2021-04-01 21:22:04 +00:00
18ca5c5292
refactor: Implement `as_str()` directly rather than using a crate
2021-04-01 21:22:04 +00:00
e52c6a8292
test: Add / update query tests
2021-04-01 21:22:04 +00:00
d0e901f503
chore: fix row_estimate to match comment
2021-04-01 21:22:04 +00:00
c0abeba792
feat: system tables
2021-04-01 21:22:04 +00:00
f66c67692c
feat: add 2 more chunk states for writing chunks to object store
2021-04-01 17:04:08 -04:00
30701bff3e
Merge pull request #1097 from influxdata/ntran/parquet_write
...
feat: Write Chunks to Parquet Files in Object Store
2021-04-01 17:37:51 +00:00
49267114d3
chore: merge main into branch and resolve conflicts
2021-04-01 13:22:49 -04:00
1463c6645f
feat: Add ChunkState::ObjectStore and rename ParquetChunk to Chunk
2021-04-01 11:53:03 -04:00
3a150594ab
refactor: push chunk and partition summary into catalog ( #1103 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-01 13:34:21 +00:00
a59a6edbb8
feat: add SHOW TABLES and `select * from information_schema.columns` ( #1102 )
...
* chore: update dependencies
* test: add tests for new information schema
2021-04-01 12:50:37 +00:00
0d599eeaf4
refactor: remove Db::writeable ( #1101 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-04-01 11:45:40 +00:00
b0e21e5f9e
feat: automated chunk lifecycle ( #1091 )
...
* feat: automated chunk lifecycle
* chore: use >= for lifecycle timestamp comparisons
* chore: review fixes
2021-04-01 11:05:58 +00:00
8f69a12ec6
refactor: Add rwlock around db rules
2021-04-01 10:43:46 +00:00
830676d843
chore: Add the serde feature to bytes dep
2021-04-01 09:00:29 +00:00
de25dc1b53
refactor: remove stuff moved to different crate
2021-03-31 20:02:34 -04:00
19a453a483
feat: finally have some framework with clear todos for writing a chunk into parquet files
2021-03-31 16:21:53 -04:00
bfd73b868a
Merge pull request #1080 from influxdata/pd-write-entry-fbs
...
feat: add new flatbuffer definitions for entries
2021-03-31 17:18:10 +00:00
fb57a61b81
Merge branch 'main' into pd-write-entry-fbs
2021-03-31 17:06:14 +00:00
3a7c023d35
chore: update entry flatbuffer null mask
2021-03-31 12:43:01 -04:00
b61875e0b2
feat: Add TableSummary calculation to ReadBuffer ( #1092 )
2021-03-31 16:26:37 +00:00
abc68ba4bf
feat: update flatbuffer table batch to column oriented
2021-03-31 11:52:06 -04:00
569f90d937
feat: Add ability to get PartitionSummary statistics from a Db ( #1090 )
...
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-31 14:18:53 +00:00
f77164c9c3
Merge pull request #1065 from influxdata/cn/here-we-go-round-again
...
chore: Move regular CI to CircleCI
2021-03-31 14:04:46 +00:00
7c6d2bdd18
Merge branch 'main' into cn/here-we-go-round-again
2021-03-31 13:50:49 +00:00
13612047c0
feat: chunk sorting ( #1077 )
2021-03-31 12:30:59 +00:00
cd409b471f
feat: continue the implementation
2021-03-30 21:31:51 -04:00
209d1ec0c8
feat: add entry flatbuffer generated code
2021-03-30 18:57:50 -04:00
b7d5087639
feat: add consistency high water mark to segment
2021-03-30 18:57:50 -04:00
66b50be1d5
feat: add new flatbuffer definitions for entries
2021-03-30 18:57:50 -04:00
bccac45c75
chore: Merge remote-tracking branch 'origin' into ntran/parquet_write
2021-03-30 18:31:32 -04:00
0bcd52d5c9
feat: Add more changes
2021-03-30 18:31:09 -04:00
7154dfd5f6
feat: Add timestamps to ChunkSummary ( #1079 )
...
* refactor: Move timestamps from mutable_buffer::Chunk to catalog::Chunk
* feat: Add timestamps to ChunkSummary
* feat: Add timestamp conversion logic to protobuf types
* test: Add tests
* fix: Update data_types test
* fix: handle negative nanos during conversion
* fix: clippy
* fix: more clippy
* fix: even more clippy
* fix: even more clippy
2021-03-30 19:03:23 +00:00
a51b1008a2
Merge pull request #1078 from influxdata/ntran/for1001
...
feat: make it easy to get OperationMetadata from Operation
2021-03-30 17:30:40 +00:00
a630c119ab
feat: make it easy to get OperationMetadata from Operation
2021-03-30 12:57:11 -04:00
d8d5b1568a
feat: automated chunk closing ( #1074 )
...
* feat: automated chunk closing
* chore: remove unused error variant
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 15:31:09 +00:00
569099fc6e
feat: Derive serde for protos
...
Rationale
---------
Our CLI needs to be able to accept configuration as JSON and render configuration as JSON.
Protobufs technically have an official JSON encoding rule called 'jsonpb` but prost doesn't
offer native supprot for it.
`prost` allows us to specify arbitrary derive metadata to be added to generated
code. We emit the `serde` derive directives in the two packages that generate prost code
(`generated_types` and `google_types`).
We use the `serde(rename_all = "camelCase")` to approximate `jsonpb`.
We instruct `prost` to use `bytes::Bytes` for some types, hence we must turn on the `serde` feature
on the `bytes` dependency.
We also use json to serialize the output of the `database get` command, to showcase the feature
and get rid of a TODO. In a subsequent PR I'll teach `database create` (and the yet to be done `database update`) to accept an option JSON configuration body so we can configure partitioning, lifecycle, sharding etc rules etc.
Caveats
-------
This is not technically `jsonpb`. Main issues:
1. default values not omitted
2. no special rendering of special types like `google.protobuf.Any`
Future work
-----------
Figure out if we can get fully compliant `jsonpb`, or at least a decent approximation.
Effect
------
```console
$ cargo run -- database get foobar_weather
{
"name": "foobar_weather",
"partitionTemplate": {
"parts": [
{
"part": {
"time": "%Y-%m-%d %H:00:00"
}
}
]
},
"lifecycleRules": {
"mutableLingerSeconds": 0,
"mutableMinimumAgeSeconds": 0,
"mutableSizeThreshold": 0,
"bufferSizeSoft": 0,
"bufferSizeHard": 0,
"sortOrder": {
"order": 2,
"sort": {
"createdAtTime": {}
}
},
"dropNonPersisted": false,
"immutable": false
},
"walBufferConfig": null,
"shardConfig": {
"specificTargets": null,
"hashRing": null,
"ignoreErrors": false
}
}
```
2021-03-30 15:16:31 +00:00
1a0e698571
feat: initial of write chunks to parquet files
2021-03-30 09:56:53 -04:00
bbfc49bacd
Merge pull request #1072 from influxdata/alamb/show_tables
...
feat: Add support for listing tables via information_schema.tables
2021-03-30 13:52:12 +00:00
f0b411cd43
feat: enable information_schema
2021-03-30 09:01:43 -04:00
865910ae5b
chore: update dependencies
2021-03-30 09:01:35 -04:00
0b7dea09d5
refactor: Remove `MutableBufferDb` and Partition ( #1068 )
...
* refactor: Move test for missing tags as null to db
* refactor: move last_write_at and create_time to catalog partition
* refactor: remove mutable buffer partition and port tests
* fix: port over partition sorting logic
* fix: remove stale comment
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-30 12:51:37 +00:00
kodiakhq[bot]
Jacob Marble
Carol (Nichols || Goulding)
Andrew Lamb
Raphael Taylor-Davies
Nga Tran
Raphael Taylor-Davies
Andrew Lamb
Marko Mikulicic
Paul Dix