Commit Graph

245 Commits (8e9dd227534f57e607c7d6b89fad055bde6e02fe)

Author SHA1 Message Date
dependabot[bot] 7a6f381e69
chore(deps): bump arrow-flight from 5.4.0 to 5.5.0
Bumps [arrow-flight](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: arrow-flight
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 13:27:26 +00:00
kodiakhq[bot] 181145eca1
Merge branch 'main' into dependabot/cargo/arrow-5.5.0 2021-10-04 13:10:42 +00:00
dependabot[bot] 6d9205d87c chore(deps): bump parquet from 5.4.0 to 5.5.0
Bumps [parquet](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: parquet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 12:15:11 +01:00
dependabot[bot] d1f5209869
chore(deps): bump arrow from 5.4.0 to 5.5.0
Bumps [arrow](https://github.com/apache/arrow-rs) from 5.4.0 to 5.5.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.5.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.4.0...5.5.0)

---
updated-dependencies:
- dependency-name: arrow
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-10-04 08:55:38 +00:00
Raphael Taylor-Davies 86cee568d5
feat: use upstream pbjson (#2650)
* feat: use upstream pbjson

* chore: fmt
2021-09-28 16:29:26 +00:00
Raphael Taylor-Davies daa2ec2f4c
revert: "fix: Revert "feat: remove routerify (#2586)" (#2623)" (#2649)
This reverts commit 18941fcbff.
2021-09-28 14:42:17 +00:00
Andrew Lamb 18941fcbff
fix: Revert "feat: remove routerify (#2586)" (#2623)
This reverts commit 9fba7b0ba0.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-24 09:50:24 +00:00
Raphael Taylor-Davies 92e6173f49
feat: implement jaeger-agent protocol directly (#2607)
* feat: implement jaeger-agent protocol directly

* chore: review feedback

* fix: remove jaeger feature flag
2021-09-22 17:30:37 +00:00
Raphael Taylor-Davies 46088a7ff1
feat: disable rustyline dirs-next dependency (#2579) 2021-09-20 14:42:07 +00:00
Raphael Taylor-Davies 9fba7b0ba0
feat: remove routerify (#2586)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-20 08:57:19 +00:00
dependabot[bot] 1112597c44
chore(deps): bump dirs from 3.0.2 to 4.0.0
Bumps [dirs](https://github.com/soc/dirs-rs) from 3.0.2 to 4.0.0.
- [Release notes](https://github.com/soc/dirs-rs/releases)
- [Commits](https://github.com/soc/dirs-rs/commits)

---
updated-dependencies:
- dependency-name: dirs
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-20 08:28:04 +00:00
Raphael Taylor-Davies 3ffb16daa6
feat: remove parse_duration (#2574)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-19 11:27:24 +00:00
Raphael Taylor-Davies 1d55d9a1b5
feat: add pbjson support (#2468)
* feat: add pbjson support

* chore: fix test
2021-09-16 07:33:27 +00:00
Raphael Taylor-Davies c66095cad1
feat: remove metrics crate (#2552) 2021-09-15 19:43:33 +00:00
Raphael Taylor-Davies 6e7fa3e574
feat: migrate http ingest metrics (#2542)
* feat: migrate http ingest metrics

* chore: review feedback

* refactor: RAII entry ingest recorder
2021-09-15 19:01:10 +00:00
Marco Neumann bfaba78dc3 refactor: move `predicate` into its own crate
Two reasons:

1. I wanna decouple `parquet_file` from `query` (nearly done, needs a
   small follow-up PR).
2. `predicate` will have more and more features (like serialization)
   which justifies a new home
2021-09-14 17:13:02 +02:00
Marco Neumann 3f2e46c397 feat: prune old transactions from preserved catalog 2021-09-14 12:08:17 +02:00
Raphael Taylor-Davies e07bc85670
feat: enable tracing for http API (#2273) (#2498)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-09-10 11:21:02 +00:00
dependabot[bot] b67610d9b9
chore(deps): bump tokio from 1.10.1 to 1.11.0
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.10.1 to 1.11.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.10.1...tokio-1.11.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 09:11:38 +00:00
kodiakhq[bot] 06311970fa
Merge branch 'main' into dependabot/cargo/parquet-5.3.0 2021-09-06 09:00:27 +00:00
dependabot[bot] ccfd6ad3da
chore(deps): bump parquet from 5.2.0 to 5.3.0
Bumps [parquet](https://github.com/apache/arrow-rs) from 5.2.0 to 5.3.0.
- [Release notes](https://github.com/apache/arrow-rs/releases)
- [Changelog](https://github.com/apache/arrow-rs/blob/5.3.0/CHANGELOG.md)
- [Commits](https://github.com/apache/arrow-rs/compare/5.2.0...5.3.0)

---
updated-dependencies:
- dependency-name: parquet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 08:52:30 +00:00
kodiakhq[bot] 0fd92c8b45
Merge branch 'main' into dependabot/cargo/structopt-0.3.23 2021-09-06 08:51:32 +00:00
dependabot[bot] b1bb390893
chore(deps): bump parking_lot from 0.11.1 to 0.11.2
Bumps [parking_lot](https://github.com/Amanieu/parking_lot) from 0.11.1 to 0.11.2.
- [Release notes](https://github.com/Amanieu/parking_lot/releases)
- [Changelog](https://github.com/Amanieu/parking_lot/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Amanieu/parking_lot/compare/0.11.1...0.11.2)

---
updated-dependencies:
- dependency-name: parking_lot
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 01:18:24 +00:00
dependabot[bot] 2cf9505029
chore(deps): bump structopt from 0.3.22 to 0.3.23
Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.22 to 0.3.23.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.22...v0.3.23)

---
updated-dependencies:
- dependency-name: structopt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-09-06 01:17:41 +00:00
Marco Neumann 09a5d27434 chore: ignore massif profiler outputs 2021-09-02 15:58:43 +02:00
Marco Neumann 4a863993ec feat: "dump catalog" debug CLI 2021-09-02 08:08:20 +02:00
Jacob Marble 6ba25728e1
chore: exclude non-source code from package (#2452)
Before this change, modifications to the following files would trigger a
rebuild on `cargo build`.

```
2,13d1
< .circleci/config.yml
< .circleci/get-deploy-tags.sh
< .editorconfig
< .gitattributes
< .github/ISSUE_TEMPLATE/bug_report.md
< .github/ISSUE_TEMPLATE/feature_request.md
< .github/PULL_REQUEST_TEMPLATE.md
< .github/dependabot.yml
< .github/semantic.yml
< .gitignore
< .kodiak.toml
< CONTRIBUTING.md
17,22d4
< Dockerfile
< Dockerfile.dockerignore
< LICENSE-APACHE
< LICENSE-MIT
< README.md
< buf.yaml
24,76d5
< docker/Dockerfile.ci
< docker/Dockerfile.ci.dockerignore
< docker/Dockerfile.ci.integration
< docker/Dockerfile.ci.integration.dockerignore
< docker/Dockerfile.iox
< docker/Dockerfile.iox.dockerignore
< docker/ci-kafka-docker-compose.yml
< docker/integration_test.sh
< docs/README.md
< docs/catalog_persistence.md
< docs/data_management.md
< docs/drawings/data_lifecycle.monopic
< docs/drawings/data_organization.monopic
< docs/encoding_thoughts.md
< docs/env.example
< docs/images/flame_graph.png
< docs/local_filesystems.md
< docs/metrics.md
< docs/multi_core_tasks.md
< docs/profiling.md
< docs/protobuf.md
< docs/regenerating_flatbuffers.md
< docs/server_startup.md
< docs/sql.md
< docs/style_guide.md
< docs/testing.md
< docs/tracing.md
< docs/valgrind.md
< perf/logs/iox_router.log
< perf/logs/iox_writer.log
< perf/logs/jaeger.log
< perf/logs/kafka.log
< perf/logs/minio.log
< perf/logs/zookeeper.log
< perf/volumes/minio/.minio.sys/buckets/.bloomcycle.bin
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/.bloomcycle.bin/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/.usage-cache.bin/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/.usage.json/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/iox1/.usage-cache.bin/fs.json
<
perf/volumes/minio/.minio.sys/buckets/.minio.sys/buckets/iox2/.usage-cache.bin/fs.json
< perf/volumes/minio/.minio.sys/buckets/.tracker.bin
< perf/volumes/minio/.minio.sys/buckets/.usage-cache.bin
< perf/volumes/minio/.minio.sys/buckets/.usage.json
< perf/volumes/minio/.minio.sys/buckets/iox1/.metadata.bin
< perf/volumes/minio/.minio.sys/buckets/iox1/.usage-cache.bin
< perf/volumes/minio/.minio.sys/buckets/iox2/.metadata.bin
< perf/volumes/minio/.minio.sys/buckets/iox2/.usage-cache.bin
< perf/volumes/minio/.minio.sys/config/config.json
< perf/volumes/minio/.minio.sys/config/iam/format.json
< perf/volumes/minio/.minio.sys/format.json
< perf/volumes/zookeeper/data/myid
< perf/volumes/zookeeper/data/version-2/log.1
< perf/volumes/zookeeper/data/version-2/snapshot.0
79,87d7
< scripts/edit_db_rules
< scripts/genlp.py
< scripts/git-log-prs
< scripts/grpcurl
< scripts/jq_edit_db_rules
< scripts/logfmts
< scripts/prototxt
< scripts/valgrind.supp
< scripts/valigrind_leak
161d80
< tools/iplan
```
2021-08-31 17:14:02 +00:00
dependabot[bot] 3e154b3b02
chore(deps): bump serde_json from 1.0.66 to 1.0.67
Bumps [serde_json](https://github.com/serde-rs/json) from 1.0.66 to 1.0.67.
- [Release notes](https://github.com/serde-rs/json/releases)
- [Commits](https://github.com/serde-rs/json/compare/v1.0.66...v1.0.67)

---
updated-dependencies:
- dependency-name: serde_json
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-30 11:28:35 +00:00
dependabot[bot] 52fbaef2df
chore(deps): bump thiserror from 1.0.26 to 1.0.28
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.26 to 1.0.28.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.26...1.0.28)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-30 11:05:11 +00:00
dependabot[bot] 13161de003
chore(deps): bump rustyline from 8.2.0 to 9.0.0
Bumps [rustyline](https://github.com/kkawakam/rustyline) from 8.2.0 to 9.0.0.
- [Release notes](https://github.com/kkawakam/rustyline/releases)
- [Changelog](https://github.com/kkawakam/rustyline/blob/master/History.md)
- [Commits](https://github.com/kkawakam/rustyline/compare/v8.2.0...v9.0.0)

---
updated-dependencies:
- dependency-name: rustyline
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-30 01:19:49 +00:00
Raphael Taylor-Davies fcec394a28
feat: connect up new metrics (#2428) 2021-08-27 12:55:35 +00:00
Raphael Taylor-Davies 3a26f7857f
feat: add new metrics abstraction (#2076) (#2365)
* feat: add new metrics abstraction (#2076)

* chore: fix lint

* feat: add tests

feat: histogram export

* chore: further docs

* chore: docs

* chore: log on prometheus export error

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-27 11:54:23 +00:00
Raphael Taylor-Davies 1773bf5d37
feat: add storage client to influxdb_iox_client (#2404)
* feat: add storage client to influxdb_iox_client

* chore: fix type_url

* refactor: split storage into separate crate

* chore: fix doctest

* chore: review feedback

* chore: add generated_types cleanup ticket

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-26 10:30:59 +00:00
Raphael Taylor-Davies 2c9c191b17
refactor: split out trace http (#2388)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-24 13:37:20 +00:00
Raphael Taylor-Davies 3fdc0e9a6a
chore: remove orphaned code from trogging (#2371)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-24 10:12:59 +00:00
Raphael Taylor-Davies 58716dee71
refactor: remove mem-qe (#2344)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-23 14:35:12 +00:00
dependabot[bot] 0d4bc4190f
chore(deps): bump assert_cmd from 1.0.8 to 2.0.0
Bumps [assert_cmd](https://github.com/assert-rs/assert_cmd) from 1.0.8 to 2.0.0.
- [Release notes](https://github.com/assert-rs/assert_cmd/releases)
- [Changelog](https://github.com/assert-rs/assert_cmd/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/assert_cmd/compare/v1.0.8...v2.0.0)

---
updated-dependencies:
- dependency-name: assert_cmd
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-23 01:17:06 +00:00
Marko Mikulicic bbbd38bb12
fix(heappy): Prevent re-entering a running heap profile
Upgrades heappy to include https://github.com/mkmik/heappy/pull/2
and adapts to the API change.
2021-08-20 14:36:36 +02:00
Marko Mikulicic 4a2f0b0a52
fix: Make heappy and nonheappy mutually exclusive
Closes #2288
2021-08-20 11:35:07 +02:00
Raphael Taylor-Davies db97069ecd
feat: jaeger span export (#2273) (#2348)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-20 08:57:16 +00:00
Paul Dix d5f01a2a68 refactor: move data generator to IOx repo and fix build 2021-08-19 14:26:15 -04:00
Raphael Taylor-Davies 04ba139cb2
refactor: split trace crate (#2343)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-19 11:00:32 +00:00
Andrew Lamb 5e1cb244f7
feat: make pprof optional (#2331) 2021-08-18 15:13:37 +00:00
Marko Mikulicic a50b1646bd
fix(heappy): Report free as positive numbers 2021-08-18 14:30:41 +02:00
Marko Mikulicic fc473995fa
feat(iox): Upgrade heappy to present free memory events
Currently heappy is reporting "bytes allocated" and "inuse_bytes" ("bytes allocated - freed bytes").

Since the allocation site and free site can be wildly different, it's very hard to correlate these events
and have a meaningful "inuse_bytes" graph.

I have a plan for how to fix this for good, but in the meantime
I'd like to report raw "freed bytes".
2021-08-18 13:04:48 +02:00
Raphael Taylor-Davies 1d6a8703af
feat: support sinking traces to an OTEL SpanExporter (#2319)
* feat: support sinking traces to an OTEL SpanExporter

* chore: consistent logging

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

* chore: review feedback

Co-authored-by: Andrew Lamb <alamb@influxdata.com>

Co-authored-by: Andrew Lamb <alamb@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-18 08:05:21 +00:00
Raphael Taylor-Davies efa776bd03
feat: fix log crate output (#2325)
* feat: fix log crate output

* chore: fix doc

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2021-08-17 16:19:22 +00:00
Marko Mikulicic 94ff77126e
feat: Enable freed memory tracking in heappy
In aed37ab50a I fixed the main
problem that rendered free tracking useless: the allocator was actually allocating more bytes than requested; we were tracking the amount of memory requested by the user and not the size of the block that was returned to the user. However, when we were tracking a call to free we used the size of the memory block which was bigger than the amount that was tracked; this led to negative numbers of "in use memory".
2021-08-17 15:09:28 +02:00
dependabot[bot] 1d85a3fef6
build(deps): bump predicates from 1.0.8 to 2.0.2
Bumps [predicates](https://github.com/assert-rs/predicates-rs) from 1.0.8 to 2.0.2.
- [Release notes](https://github.com/assert-rs/predicates-rs/releases)
- [Changelog](https://github.com/assert-rs/predicates-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/assert-rs/predicates-rs/compare/v1.0.8...v2.0.2)

---
updated-dependencies:
- dependency-name: predicates
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-08-16 15:36:49 +00:00
Marko Mikulicic 500019cff9
Merge branch 'main' into ntran/heappy2 2021-08-13 17:27:38 +02:00