Commit Graph

15 Commits (f4693e36c065c065599141d12d452b73a4674321)

Author SHA1 Message Date
Andrew Lamb 00e735ef0d
chore: remove unused dependencies (#1583) 2021-05-29 10:31:57 +00:00
Raphael Taylor-Davies 216903a949
refactor: move protobuf conversion logic to generated_types (#1437)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-05-06 15:49:27 +00:00
Raphael Taylor-Davies 1997324344
feat: mutable buffer snapshotting (#1179)
* feat: mutable buffer snapshotting

* chore: review feedback
2021-04-13 12:14:54 +00:00
Jacob Marble 80d55d0829 chore: rename tracing_deps to observability_deps
OpenTelemetry makes this necessary.
2021-04-02 13:14:30 -07:00
Carol (Nichols || Goulding) 0b880d3534 chore: Group all tracing-related crates under one crate for easier upgrade management 2021-04-02 09:54:39 -04:00
Marko Mikulicic 830676d843 chore: Add the serde feature to bytes dep 2021-04-01 09:00:29 +00:00
Marko Mikulicic 569099fc6e feat: Derive serde for protos
Rationale
---------

Our CLI needs to be able to accept configuration as JSON and render configuration as JSON.

Protobufs technically have an official JSON encoding rule called 'jsonpb` but prost doesn't
offer native supprot for it.

`prost` allows us to specify arbitrary derive metadata to be added to generated
code. We emit the `serde` derive directives in the two packages that generate prost code
(`generated_types` and `google_types`).

We use the `serde(rename_all = "camelCase")` to approximate `jsonpb`.

We instruct `prost` to use `bytes::Bytes` for some types, hence we must turn on the `serde` feature
on the `bytes` dependency.

We also use json to serialize the output of the `database get` command, to showcase the feature
and get rid of a TODO. In a subsequent PR I'll teach `database create` (and the yet to be done `database update`) to accept an option JSON configuration body so we can configure partitioning, lifecycle, sharding etc rules etc.

Caveats
-------

This is not technically `jsonpb`. Main issues:
1. default values not omitted
2. no special rendering of special types like `google.protobuf.Any`

Future work
-----------

Figure out if we can get fully compliant `jsonpb`, or at least a decent approximation.

Effect
------

```console
$ cargo run -- database get foobar_weather
{
  "name": "foobar_weather",
  "partitionTemplate": {
    "parts": [
      {
        "part": {
          "time": "%Y-%m-%d %H:00:00"
        }
      }
    ]
  },
  "lifecycleRules": {
    "mutableLingerSeconds": 0,
    "mutableMinimumAgeSeconds": 0,
    "mutableSizeThreshold": 0,
    "bufferSizeSoft": 0,
    "bufferSizeHard": 0,
    "sortOrder": {
      "order": 2,
      "sort": {
        "createdAtTime": {}
      }
    },
    "dropNonPersisted": false,
    "immutable": false
  },
  "walBufferConfig": null,
  "shardConfig": {
    "specificTargets": null,
    "hashRing": null,
    "ignoreErrors": false
  }
}
```
2021-03-30 15:16:31 +00:00
Carol (Nichols || Goulding) c47a252c15 feat: Check in generated WAL flatbuffers code 2021-03-22 09:38:59 -04:00
Carol (Nichols || Goulding) 24af745b23 chore: Upgrade to flatbuffers 0.8 2021-03-22 09:38:58 -04:00
Raphael Taylor-Davies 65f7a1ac5b
fix: use consistent crate versions (#989)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-15 15:42:19 +00:00
Raphael Taylor-Davies d2859a99d0
feat: add google longrunning operations stubs (#959)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-03-10 17:34:07 +00:00
Raphael Taylor-Davies 6d3c21b952 feat: Use Bytes instead of Vec<u8> in prost generated code
Also add google.rpc.* Protobuf definitions
2021-02-12 17:10:58 +00:00
Carol (Nichols || Goulding) fbf776c6b3
chore: Clean up Cargo.tomls (#754)
* fix: test_helpers crate should only be a dev-dep

* fix: object_store no longer has a build script, so no longer needs a build dep

* chore: Alphabetize all Cargo.tomls
2021-02-04 18:56:02 -05:00
Andrew Lamb f3bd8bd0e3
chore: update deps (tokio 1.0 and ecosystem) (#707)
* chore: Update arrow + tokio deps

* chore: Use bleeding edge azure

* chore: Update aws + other deps

* fix: fmt

* fix: Switch to in-house version of routerify

* fix: Upgrade to hyper 0.14

The hyper::error module is now private; hyper::Error is the public
re-export

* fix: Upgrade cloud storage to get tokio upgrade

* fix: Upgrade open_telemetry

* fix: Do not call `panic::set_hook` during another panic

Doing so leads to a double panic which aborts the process.

* fix: new h2 error who dis

Co-authored-by: Carol (Nichols || Goulding) <carol.nichols@integer32.com>
Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
2021-01-29 16:11:55 -05:00
Andrew Lamb a52e0001c5
refactor: rename all crates that start with`delorean_` in preparation for rename (#415)
* refactor: rename delorean_cluster --> cluster

* refactor: rebane delorean_generated_types --> generated_types

* refactor: rename delorean_write_buffer --> write_buffer

* refactor: rename delorean_ingest --> ingest

* refactor: rename delorean_storage --> storage

* refactor: rename delorean_tsm --> tsm

* refactor: rename delorean_test_helpers --> test_helpers

* refactor: rename delorean_arrow --> arrow_deps

* refactor: rename delorean_line_parser --> influxdb_line_protocol
2020-11-05 13:44:36 -05:00