Commit Graph

33 Commits (1a4c7932702b52600233c5215fcc649334c47f52)

Author SHA1 Message Date
Raphael Taylor-Davies 96f9395299 feat: smaller IOx build images 2021-02-25 15:05:54 +00:00
Dom 56b5751172 fix: filter docker images with specified tag 2021-01-18 11:13:50 +00:00
Dom 3acdc9818f ci: output deployment tags for perf build
Once the perf image build is complete, output the JSON metadata needed
to deploy the container.
2021-01-15 16:02:18 +00:00
Dom 7e1e24ca21 ci: only run CircleCI for /perf branches
Filters the triggered CircleCI workflow to only run against branches with /perf
in their name.

No changes to the scheduled build image generation.
2020-11-27 11:34:56 +00:00
Dom 9c785a3b0f ci: add GitHub actions
Adds a GitHub CI action:

    * Uses the rust-toolchain file to install the requested version.
    * Performs a build with default cargo target
    * Runs workspace tests
    * Checks cargo fmt is happy
    * Runs clippy for lints, denying warnings

All commands taken from the existing CircleCI pipeline.

Currently no caching / reuse of build container.
2020-11-27 11:34:56 +00:00
Dom e93941e487 docs(ci): extra CI documentation
Adds some extra quick-glance docs for the CI flow.
2020-11-26 13:46:07 +00:00
Andrew Lamb 3a9ee88f00
chore: update to latest version of arrow + update code (#486)
* chore: update to latest version of arrow + update code

* chore: Update rust toolchain to match arrow

* fix: clippy
2020-11-25 14:46:35 -05:00
Andrew Lamb 831a0875d6
chore: update to latest arrow + Rust nightly-2020-11-14 (#454)
* chore: update to latest arrow + Rust nightly-2020-11-14

* chore: update ci

* fix: update for clippy lints

* fix: Allow redundant_field_names in generated types crate

* fix: clippy about try_for_each

* fix: clippy uneeded-collect

* fix: clippy about default values

* fix: clippy mathces --> matches!

* fix: clippy sort --> sort_by_key

* fix: clippy about default values again
2020-11-16 11:48:42 -05:00
Andrew Lamb 986436300b
fix: update the branch referece in nightly ci image to main (#438) 2020-11-11 06:40:38 -05:00
Andrew Lamb 0532821ed9
chore: Make the project root neat and tidy (#416)
* chore: move example env file into docs

* chore: move dockerfiles out of root

* chore: move storectl-bench.sh into benches
2020-11-10 10:00:22 -05:00
Andrew Lamb e1b230caab
chore: give executor more memory (#421) 2020-11-06 10:24:47 -05:00
Andrew Lamb c646f29794
feat: Add source checkout and initial build to docker CI image, don't use build cache (#381)
* feat: Add source checkout and initial build to docker CI image

* fix: remove tidbits
2020-10-22 05:50:36 -04:00
Andrew Lamb 3d7d4111be
fix: Upgrade the resource class used to run CI tests (#332) 2020-10-01 14:56:32 -04:00
Carol (Nichols || Goulding) 3340c45053 fix: Switch to fusion quay repo for performance builds
Connects to influxdata/fusion#29.
2020-07-22 09:33:42 -04:00
Carol (Nichols || Goulding) fc80b463d5 fix: Set ROARING_ARCH to build a portable binary using CRoaring
See https://github.com/saulius/croaring-rs/pull/62
2020-07-08 10:03:02 -04:00
Carol (Nichols || Goulding) 6d87d16754 feat: When a branch ends with /perf, build in release mode and upload to quay 2020-07-03 11:42:33 -04:00
Andrew Lamb 8475b6d183
feat: Add parquet writer, hook up conversion in dstool (#124)
* feat: Add parquet writer, hook up conversion in dstool

* fix: use bigger executor for test

* fix: less cloning

* fix: make unsupported messages less pejorative

* fix: fmt

* fix: Rename writer and do not require std::File, add example

* fix: clippy and fmt

* fix: remove unnecessary module in end to end tests

* fix: remove strange use of tempfile

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: Apply suggestions from code review

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: cleanup use

* fix: Use more specific error messages

* fix: comment tweak

* fix: touchup temp path creation

* fix: clippy!

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-08 16:25:24 -04:00
Andrew Lamb 2200def8ea
feat: Use rust nightly (#123) 2020-06-05 17:45:44 -04:00
Andrew Lamb 18b05ce9ef
fix: move test of dstool to its delorean_storage_tool package (#107) 2020-06-02 16:10:30 -04:00
AJ Bond 86949a6fc1
fix(circleci): remove nightly install tools 2020-06-02 10:14:10 -04:00
AJ Bond 5aafbf1977
chore(build): setup nightly ci image build
We can reduce the number of install steps by using a custom container. This container will build nightly and publish to Quay
2020-06-02 10:00:17 -04:00
Andrew Lamb 1a2efdfd71
feat: Add dstool command line tool (#102)
* feat: Add dstool command line tool

* clippy

* Update delorean_storage_tool/src/main.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Update delorean_storage_tool/src/main.rs

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* Add in tests + PR comments

* fmt

* build first then run tests

* actually build before test

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
2020-06-02 07:33:43 -04:00
Jake Goulding 0ea0cf6a93 ci: install the Flatbuffers compiler 2020-05-11 15:38:47 -04:00
Jake Goulding 083ec01d0b ci: No longer need to limit parallelism due to RocksDB 2020-05-08 10:29:48 -04:00
Carol (Nichols || Goulding) 05c7d5949a test: CI commands need some more flags now that we're a workspace 2020-04-27 09:42:53 -04:00
Carol (Nichols || Goulding) f81c8764d0 improvement: Fail the build on warnings too 2020-02-19 08:37:11 -05:00
Carol (Nichols || Goulding) e03b3e9326 fix: Run Clippy on test targets too by using --all-targets instead
It's confusing that `cargo fmt` takes `--all` to format everything but
`cargo clippy` takes `--all-targets` :-/
2020-02-19 08:37:11 -05:00
Edd Robinson 0b3c412cfc chore: make clippy required CI step 2020-02-14 18:27:53 +00:00
Jake Goulding fd76a7394c ci: Enforce standard formatting 2020-02-14 08:02:24 -05:00
Jake Goulding d8f3c31b8c ci: Prevent rustfmt and clippy from stopping the build for now 2020-02-09 22:23:27 -05:00
Jake Goulding 81468f9e5e ci: Adjust builds to run on highly parallel CI machines
By default, the RocksDB C library is compiled using the number of
cores on the machine. In CircleCI, this is 36 cores. Unfortunately,
that appears to completely blow out the memory usage, causing the C
compiler invocations to be killed.

This commit reduces the *entire* parallelism of the build to avoid
that.
2020-02-09 22:22:10 -05:00
AJ Bond 7a65f5550f
chore(circleci): add tiered cache configuration 2020-02-07 12:25:32 -05:00
AJ Bond c2de967e36
chore(circleci): setup circleci
This configures a circleci pipeline that runs fmt, lint, test, and build operations.
I changed the fmt command to --check since --overwrite is no longer supported.
The pipeline will always run on nightly builds of rust.
2020-02-07 11:46:47 -05:00