* fix: bind to correct port for e2e tests
This also fixes up some log messages on server start for naming
* chore: do notpass value in TEST_LOG env var to CI tests
* feat(circleci): add inclusivity checks
* chore(circleci): adjust package-validation for inclusive language
* chore: update tests for inclusive language
This changes our CI to use cargo-nextest which is faster and does not
have issues around global statics. Since it runs each test in it's
own process we don't have to worry about tests stepping on each other's
toes in this regard. It also updates the CI to ignore the current
cargo deny failure as we can't do anything until the arrow crates are
upgraded.
* chore: clean up heappy, pprof, and jemalloc
Setup the use of jemalloc as default allocator using tikv-jemallocator
crate instead of tikv-jemalloc-sys.
Removed heappy and pprof, and also cleaned up all the mutually exclusive
compiler flags for using heappy as the allocator.
* chore: remove heappy from ci
For releases we need to have Docker images and binary images available for the
user to actually run influxdb3. These CI changes will build the binaries on a
release tag and the Docker image as well, test, sign, and publish them and make
them available for download.
Co-Authored-By: Brandon Pfeifer <bpfeifer@influxdata.com>
* feat: /ping API to serve version
The /ping API was added, which is served at GET and
POST methods. The API responds with a JSON body
containing the version and revision of the build.
A new crate was added, influxdb3_process, which
takes the process_info.rs module from the influxdb3
crate, and puts it in a separate crate so that other
crates (influxdb3_server) can depend on it. This was
needed in order to have access to the version and
revision values, which are generated at build time,
in the HTTP API code of influxdb3_server.
A E2E test was added to check that /ping works.
E2E TestServer can now have logs emitted using the
TEST_LOG environment variable.
* chore: Update to Rust 1.77.0
This is a fairly quiet upgrade. The only changes are some lints around
`OpenOptions` that were added to clippy between 1.75 and this version
and they're small changes that either remove unecessary function calls
or add a needed function call.
* fix: cargo-deny by using the --locked flag
This commit re-enables the limits test after making a fix that has it
run <1 second on my laptop vs the old behavior of >=30 seconds. It does
so by constructing one single write_lp request to create 1995 tables
rather than 1995 individual requests that make a table. This is far more
efficient.
This commit is a major refactor for the code base. It mainly does four
things:
1. Splits code shared between the internal IOx repository and this one
into it's own repo over at https://github.com/influxdata/influxdb3_core
2. Removes any docs or anything else that did not relate to this project
3. Reorganizes the Cargo.toml files to use the top level Cargo.toml to
declare dependencies and versions to keep all crates in sync and sets
all others to use `<dep>.workspace = true` unless it's an optional
dependency
4. Set the top level Cargo.toml to point to the core crates as git
dependencies
With this any changes specific to Edge will be contained here, updating
deps will be a PR over in `influxdata/influxdb3_core`, and we can prove
out the viability for this model to use for IOx.
Prior to this change we've had CI fail nightly because we can't push the
image to CI due to permissions issues. The problem is that
influxdata/influxdb_iox is the one that actually has access to push that
data to quay.
This commit removes the nightly build and references to it as this image
is built nightly by the IOx team. If things break we have access to fix
it, but I don't think it'll be an issue.
* chore(deps): Update arrow and datafusion to 49.0.0
This commit copies in our dependency code from influxdb_iox in order for
us to be able to upgrade from a forked version of 46.0.0 to 49.0.0 of
both arrow and datafusion. Most of the important changes were around how
we consumed the crates in influxdb3(_server/_write). Those diffs are
particularly worth looking at as the rest was a straight copy and we
don't touch those crates in our development currently for influxdb3
edge.
* fix: regenerate workspace hack crate
* fix: Protobuf issues with incompatibility labels
* fix: Broken CI yaml
* fix: buf version
* fix: Only check IOx repo
* fix: Remove protobuf lint
* fix: Comment out call to protobuf-lint
We currently don't need or want to deploy influxdb as we're still
building out the Edge product. Maybe later for a demo, but for now it
just breaks CI and so this removes it.
This commit changes the circle-ci config to use influxdb3 rather than
iox in our ci config script as the repo is influxdb not influxdb_iox.
While we could probably strip out a lot more here as a first attempt to
get this to build release images and push them on main this will do just
fine.
* fix: build, upgrade rustc, and deps
This commit upgrades Rust to 1.75.0, the latest release. We also
upgraded our dependencies to stay up to date and to clear out any
uneeded deps from the lockfile. In order to make sure everything works
this also fixes the build by upgrading the workspace-hack crate using
cargo hikari and removing the `workspace.lint` that was in
influxdb3_write that didn't need to be there, probably from a merge
issue.
With this we can build influxdb3 as our default on main, but this alone
is not enough to fix CI and will be addressed in future commits.
* fix: warnings for influxdb3 build
This commit fixes the warnings emitted by `cargo build` when compiling
influxdb3. Mainly it adds needed lifetimes and removes uneccesary
imports and functions calls.
* fix: all of the clippy lints
This for the most part just applies suggested fixes by clippy with a few
exceptions:
- Generated type crates had additional allows added since we can't
control what code gets made
- Things that couldn't be automatically fixed were done so manually in
particular adding a Send bound for traits that created a Future that
should be Send
We also had to fix a build issue by adding a feature for tokio-compat
due to the upgrade of deps. The workspace crate was updated accordingly.
* fix: failing test due to rust panic message change
Inbetween rustc 1.72 and rustc 1.75 the way that error messages were
displayed when panicing changed. One of our tests depended on the output
of that behavior and this commit updates the error message to the new
form so that tests will pass.
* fix: broken cargo doc link
* fix: cargo formatting run
* fix: add workspace-hack to influxdb3 crates
This was the last change needed to make sure that the workspace-hack
crate CI lint would pass.
* fix: remove tests that can not run anymore
We removed iox code from this code base and as a result some tests
cannot be run anymore and so this commit removes them from the code base
so that we can get a green build.
* refactor: isolate docker build to script
* chore: add labels to docker image
* chore: export image as OCI
* chore: print image digest
* fix: convert to OCI BEFORE calculating digest
* fix: use digest of uploaded image, not of the local archive
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
As we've learned in #8048 and #8052, rustflags do NOT stack. Since we
only want to change one specific parameter (the debug feature), use the
env variable that cargo provides us.
**In contrast to the linked PRs, this only changes the test excution. Prod
builds remain untouched.**
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
- move `tokio_unstable` to cargo config, so all we can use it within
our code (e.g. for #7982)
- disable incremental builds for prod docker builds. this was tried
before but got lost at some point because build params weren't passed
to docker correclty
- fix `CARGO_NET_GIT_FETCH_WITH_CLI` for docker builds (env wasn't
passed through)
This updates the job logic so that workflow condition is evaluated
by CircleCI rather than the shell. This also uses the "aws-s3" orb
for uploading to S3 (rather than awscli).
The terraform shipped with snap (in the older version of Ubuntu)
only supported public key encryption with ssh-rsa. New versions
of Linux started deprecating ssh-rsa, so this version bump
is required.
This allows changelogs to be built from "non-release" tags. These
changelogs use "UNRELEASED" as the first section header. Commits
from these sections are eventually rolled into a proper "release"
tag (e.g v2.7.0).
* fix: racing JDBC tests
The JDBC tests have been flaky since adding additional tests. Use
the makefile to build the client to avoid the clients racing.
* chore: pre-download JDBC drive in integration test
* fix: remove stray lockfile
* chore: Run end to end integration tests as part of a separate circle CI Job
* fix: but
* feat: update job name
* chore: Update resources
---------
Co-authored-by: Dom <dom@itsallbroken.com>
Jackson Newhouse
Trevor Hilton
Paul Dix
Brandon Pfeifer
Jamie Strandboge
praveen-influx
Michael Gattozzi
Marco Neumann
alespour
Carol (Nichols || Goulding)
Andrew Lamb
Martin Hilton