Commit Graph

218 Commits (70b44f78eed52c647b6ffe43aefdce332f3726e3)

Author SHA1 Message Date
Marco Neumann d9ce92dad1
fix: do not override all rustflags in circleci (#8061)
As we've learned in #8048 and #8052, rustflags do NOT stack. Since we
only want to change one specific parameter (the debug feature), use the
env variable that cargo provides us.

**In contrast to the linked PRs, this only changes the test excution. Prod
builds remain untouched.**

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-27 14:35:04 +00:00
Marco Neumann 4d1c6f805c
refactor: clean up rustflags and build args (#8047)
- move `tokio_unstable` to cargo config, so all we can use it within
   our code (e.g. for #7982)
- disable incremental builds for prod docker builds. this was tried
  before but got lost at some point because build params weren't passed
  to docker correclty
- fix `CARGO_NET_GIT_FETCH_WITH_CLI` for docker builds (env wasn't
  passed through)
2023-06-22 09:58:46 +00:00
Marco Neumann b1cb0b7fc0
chore: bump `max_connections` for integration test PG (#8024)
It seems that sometimes our tests are fast enough to trigger the
connection limit, see

https://app.circleci.com/pipelines/github/influxdata/influxdb_iox/34517/workflows/abd68ab0-a0a4-47b4-8399-cbf61bd27ab9/jobs/322837

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-06-20 11:39:46 +00:00
Marco Neumann 8cd82728cd
fix: deploy_release after debian upgrade (#7988) 2023-06-14 09:16:15 +00:00
Marco Neumann fad11b524f
chore: fix JDK for integration tests (#7980)
Broken after the first nightly CI image was generated after #7972.

Move JDK installation to image generation and fix version.
2023-06-13 10:51:26 +00:00
Carol (Nichols || Goulding) e12329ac2b
fix: Remove temporary limiting of test threads 2023-05-08 20:24:35 -04:00
Andrew Lamb 23a74111c8
chore: Stop using cargo cache in CI (#7721)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-05-04 10:33:15 +00:00
Martin Hilton f87f4c1869
fix: racy JDBC tests (#7678)
* fix: racing JDBC tests

The JDBC tests have been flaky since adding additional tests. Use
the makefile to build the client to avoid the clients racing.

* chore: pre-download JDBC drive in integration test

* fix: remove stray lockfile
2023-04-27 09:07:06 +00:00
Andrew Lamb 3263934db1
chore: Run end to end integration tests as part of a separate circle CI Job (#7658)
* chore: Run end to end integration tests as part of a separate circle CI Job

* fix: but

* feat: update job name

* chore: Update resources

---------

Co-authored-by: Dom <dom@itsallbroken.com>
2023-04-26 15:22:13 +00:00
Carol (Nichols || Goulding) f561d599d2
fix: Remove uses of KAFKA_CONNECT, redpanda, and some talk of Kafka 2023-04-12 11:53:23 -04:00
Marco Neumann 6a99e181cb
chore: streamline CD (#7165)
* chore: streamline CD

Use our own CI image for CD and avoid installing skopeo every time. Also
streamline the script a bit to make it more readable and less prone to
errors/typos.

This should hopefully safe a good amount of CD time.

Also see https://github.com/influxdata/idpe/issues/17098 .

* chore: remote docker no longer required for CD

* chore: gate deploy on Flight SQL integration tests

* chore: use more CPUs to build release image

* fix: `skopeo` login permissions
2023-03-09 11:53:57 +00:00
Marco Neumann c8184c9947 revert: streamline CD (#7163)
This reverts commit d6265c988f.

It seems that Debian's skopeo is so horribly old or the packaging is so
broken that you cannot run it out of the box as a normal non-root user.
I need to fix the CI first before we can try again.
2023-03-09 11:46:23 +01:00
Marco Neumann d6265c988f
chore: streamline CD (#7163)
Use our own CI image for CD and avoid installing skopeo every time. Also
streamline the script a bit to make it more readable and less prone to
errors/typos.

This should hopefully safe a good amount of CD time.

Also see https://github.com/influxdata/idpe/issues/17098 .
2023-03-09 10:20:14 +00:00
Andrew Lamb a9433302dd
Revert "chore: preserve `query_tests2` output in CI (#6782)" (#6844)
This reverts commit d7d8c84203.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-02-03 17:49:33 +00:00
Marco Neumann d7d8c84203
chore: preserve `query_tests2` output in CI (#6782)
This shall help to debug flaky tests.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2023-01-31 15:11:15 +00:00
Andrew Lamb 7853a19953
feat: JDBC integration tests with FlightSQL (#6693)
* feat: basic JDBC integration test

* fix: do not run test without env set

* docs: add maven link

* refactor: clean up java with switch statement
2023-01-25 22:21:18 +00:00
Carol (Nichols || Goulding) f803c31e84
fix: Limit tests in CI to 8 threads to not use up Postgres connections
This is only needed until we switch over to ingester2 completely.

Old ingester tests need to be run on non-shared servers because I'm
unable to implement persistence per-namespace. Rather than spending time
figuring that out, limit the parallelization to limit the Postgres
connections that CI uses at one time.
2023-01-25 10:37:05 -05:00
Dom Dwyer 38eeffac0d
revert: "ci: use rust 1.66.0 for CI"
This reverts commit 71f54528d0.
2023-01-12 10:36:26 +01:00
Dom Dwyer 5639db7c84
docs: ref-link lack of 1.66.1 issue
Link to the upstream issue.
2023-01-11 14:21:19 +01:00
Dom Dwyer 71f54528d0
ci: use rust 1.66.0 for CI
Temporary fix as there's no official 1.66.1 docker image (yet?), so use
1.66 for now.

I kept the toolchain version pinned and switched only the CI version so
that we are all using the fixed cargo. I will revert this commit when
there's a 1.66.1 image.
2023-01-11 14:05:17 +01:00
Carol (Nichols || Goulding) f5231da970
fix: Remove the test_rpc_write CI job 2022-12-16 17:11:53 -05:00
Carol (Nichols || Goulding) aec98015d7
fix: Remove the rpc_write feature flag and use INFLUXDB_IOX_MODE env var instead
And standardize on ingester2 and router2 for consistency.

Connects to #6402.
2022-12-15 14:06:59 -05:00
Carol (Nichols || Goulding) 0a4df1f3fb
chore: Run tests in CI in both RPC write mode and not 2022-12-08 17:40:04 -05:00
Carol (Nichols || Goulding) b86eb763b1
fix: Tell buf to fetch to a depth of 100 commits when looking for breaking changes
Per <https://docs.buf.build/reference/inputs#other-options>, this is by
default 50 (and I have 51 commits on this PR 😰)

100 commits should be enough for anyone.
2022-11-23 17:24:50 -05:00
jaymebrd d41c1290cf
chore: copy iox images skopeo gar registry (#6172)
* chore: copy the iox images to the gar registry with skopeo

* chore: add quotes
2022-11-18 16:37:08 +00:00
Andrew Lamb 1eb0d64210
chore: remove unecessary doc exclude (#6018)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-11-01 20:17:27 +00:00
Luke Bond c39ae97dd4
chore: temporarily disable circle filter to build & push PRs (#5973)
* chore: temporarily disable circle filter to build & push PRs

* chore: allow build & push of container image for branches using param

* chore: indentation fix in circle config

* chore: rename build_perf to release_branch
2022-10-25 15:40:13 +00:00
Marco Neumann adeacf416c
ci: fix (#5569)
* ci: use same feature set in `build_dev` and `build_release`

* ci: also enable unstable tokio for `build_dev`

* chore: update tokio to 1.21 (to fix console-subscriber 0.1.8

* fix: "must use"
2022-09-06 14:13:28 +00:00
Andrew Lamb f9d0e37144
chore: reduce h2 and hyper logging level in tests (#5332)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-08 09:39:26 +00:00
Andrew Lamb 38a0cdbb4a
fix: Install cargo deny in ci image (#5317)
* fix: install cargo deny in ci image

* fix: Update docker/Dockerfile.ci

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>

* fix: Apply suggestions from code review

Co-authored-by: Marco Neumann <marco@crepererum.net>

Co-authored-by: Carol (Nichols || Goulding) <193874+carols10cents@users.noreply.github.com>
Co-authored-by: Marco Neumann <marco@crepererum.net>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-05 15:51:35 +00:00
Dom Dwyer 36d36c507c ci: bump redpanda version 2022-08-04 16:57:28 +02:00
Raphael Taylor-Davies 5f9ade103f
chore: tweak rustc flags (#5293)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-08-04 09:12:21 +00:00
Andrew Lamb e2d871b00b
chore: Update datafusion and arrow/parquet/arrow-flight to `18.0.0` (#5079)
* chore: Update datafusion to 10.0.0, arrow/parquet/arrow-flight to 18

* chore: Run cargo hakari tasks

* fix: update cargo pin

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-07-18 15:01:03 +00:00
Marco Neumann d33525055b
ci: update CI machine images (#5061) 2022-07-07 09:14:02 +00:00
Marco Neumann e84e1f3de2
ci: clean up and fixes (#5054)
* ci: remove unused helper script

* ci: update CI machine images

Try to fix error a la:

```text
error: failed to solve: failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://docker-images-prod.s3.dualstack.us-east-1.amazonaws.com/registry-v2/docker/registry/v2/blobs/sha256/...": dial tcp 52.216.28.112:443: i/o timeout
```

See https://discuss.circleci.com/t/increased-rate-of-errors-when-pulling-docker-images-on-machine-executor/42094/9
2022-07-06 14:47:50 +00:00
Marko Mikulicic 015bba8589
chore: c2updater to acs transition (#5045)
1. disable c2updater call
2. use full sha tags (required by `acs`)
2022-07-05 21:47:05 +00:00
Jacob Marble 1ef9c7ceda
chore: remove perf.py (#5012)
Closes #4485
Closes #5010

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-30 22:32:47 +00:00
Marco Neumann fba58dbc5f
ci: IOx still needs to push its image tags (#4977)
This will be replaced by a pull-based approach soon, but for the time
being we still need perform the final push.
2022-06-29 09:52:44 +00:00
Marco Neumann 847c84a6b4
ci: fix `docker load` paths (#4961)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-29 08:03:02 +00:00
Marco Neumann cede296eb1 ci: fix `docker save` paths 2022-06-28 10:41:04 +02:00
Marco Neumann 2ebb7b195b
ci: fix image deploy (#4953)
The Influx deployment pipeline was changed so the an image push is used
as a signal for deployment (instead of a magic script that was used
before). So we need to adopt our CI to only push images when all tests
pass.

Old workflow:
- build release: builds docker images and push commit-based tags to
  registry
- deploy release: pulls built images from registry, adds+pushes branch
  tags, calls magic deploy script

New workflow:
- build release: builds docker image, saves them to disk
- deploy release: load image files, tags them, pushes tags

You may wonder why there are two steps if we could just use a single
one. The reason is: time-to-deploy. We can already build the image while we
are waiting for the tests. If the tests fail, the image will just not be
published.

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-28 07:48:41 +00:00
Andrew Lamb 394c84f3e8
chore: Update CI checks to verify data generator build (#4857)
* chore: Update CI checks to verify data generator build

* fix: bench verify test

* docs: Update .circleci/config.yml

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-06-15 10:31:14 +00:00
Marco Neumann 9accb5912e
chore: lower job count in `build_dev` (#4815) 2022-06-09 13:26:35 +00:00
Carol (Nichols || Goulding) b52a3586a7
fix: Turn cargo doc warnings into errors (#4710)
* fix: Correct intra-doc links

* fix: Turn cargo doc warnings into errors

Co-authored-by: Jake Goulding <jake.goulding@integer32.com>
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-28 11:24:22 +00:00
Dom Dwyer 6b6dbb0286 build: remove iox_gitops_adapter from build
Broken release builds since:

    https://github.com/influxdata/influxdb_iox/pull/4675
2022-05-24 16:30:19 +01:00
Andrew Lamb 52a50c4a14
fix: use large circleci executor for docs job (#4680) 2022-05-24 14:26:49 +00:00
Jake Goulding 359046f3f2 ci: give the doc builder more memory 2022-05-20 10:44:06 -04:00
Marco Neumann 7d16f57c85
ci: simplify cargo deny (#4640)
Taken from https://github.com/influxdata/object_store_rs/pull/5

Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
2022-05-19 09:51:15 +00:00
Marco Neumann 52346642a0
ci: fix cargo deny (#4629)
* ci: fix cargo deny

* chore: downgrade `socket2`, version 0.4.5 was yanked

* chore: rename `query` to `iox_query`

`query` is already taken on crates.io and yanked and I am getting tired
of working around that.
2022-05-18 09:38:35 +00:00
Raphael Taylor-Davies f2bb0fdf77
feat: update to crates.io object_store version (#4595)
* feat: update to crates.io object_store version

* chore: Run cargo hakari tasks

* fix: tests

* chore: remove object store integration test plumbing

Co-authored-by: CircleCI[bot] <circleci@influxdata.com>
2022-05-13 16:26:07 +00:00