Commit Graph

49707 Commits (fix/docker-python-deps-earlier)

Author SHA1 Message Date
Adrian Thurston 5703da545b fix: docker: fetch python deps ahead of copying in full source tree
The current build order:

 1. Copy in full source tree
 2. Fetch python build dependencies
 3. Build

The issue with this is any source tree change causes the python dependencies to
be re-fetched, then the rust components of those deps rebuilt.

Instead, copy in just the .circleci directory, which informs the python build
dependency fetch. After we have the python deps, then copy in the full source
tree and build.
2025-05-26 12:06:02 -07:00
Trevor Hilton 4dc61df77f
chore: update to latest influxdb3_core (#26429)
* chore: update to latest core

* chore: allow CDLA permissive 2 license

* chore: update insta snapshot for new internal df tables

* test: update assertion in flightsql test

* fix: object store size hinting workaround in clap_blocks

Applied a workaround from upstream to strip size hinting from the object
store get request options. See:

https://github.com/influxdata/influxdb_iox/issues/13771

* fix: query_executor tests use object store size hinting workaround

* fix: insta snapshot test for show system summary command

* chore: update windows- crates for advisories

* chore: update to latest sha on influxdb3_core branch

* chore: update to latest influxdb3_core rev

* refactor: pr feedback

* refactor: do not use object store size hint layer

Instead of using the ObjectStoreStripSizeHint layer, just provide the
configuration to datafusion to disable the use of size hinting from
iox_query.

This is used in IOx and not relevant to Monolith.

* fix: use parquet cache for get_opts requests

* test: that the parquet cache is being hit from write buffer
2025-05-26 14:11:06 -04:00
Trevor Hilton 9ed1af7d7a
fix: use correct ENV vars in Dockerfile (#26461)
The `INFLUXDB3_OBJECT_STORE` and `INFLUXDB3_DB_DIR` env vars were not
being used in the Dockerfile.
2025-05-26 14:09:59 -04:00
Stuart Carnie c5ed113c5b
chore: Update rust toolchain to 1.87.0 (#26456)
Changes were due to a number of clippy improvements
2025-05-26 09:22:32 -04:00
Stuart Carnie 1abbb525db
fix: Ensure series key metadata is persisted to Parquet snapshots (#26449)
* chore: Ensure Parquet sort key is serialised with snapshots

* chore: PR feedback, rename state variable to match intent

* chore: Use `Default` trait to implement `TableBuffer::new`

* chore: Fix change in file size with extra metadata

* chore: Add rustdoc for `sort_key` field
2025-05-26 09:27:07 +10:00
Trevor Hilton 760c89873d
fix: add key variant back to wal to fix bitcode deserialization (#26453) 2025-05-23 13:04:18 -04:00
Trevor Hilton 6e9446a8bb
test: reproduce problem with NULL backfill of omitted tag cols (#26448)
* test: reproduce problem with NULL backfill of omitted tag cols

* fix: do not fill tag columns with empty string on persist

* chore: clippy
2025-05-22 12:31:53 -04:00
Trevor Hilton 56df158afd
docs: add rust docs to catalog types (#26447) 2025-05-22 11:18:42 -04:00
Trevor Hilton d1c10f4b29
fix: backfill new tags with NULL instead of empty string (#26446)
* fix: backfill new tags with NULL instead of empty string

* refactor: use helper for append_null

* test: add a test to check null back/forward fill
2025-05-21 17:23:51 -04:00
Trevor Hilton 4a917c5a9f
refactor: remove variants for unused series key type (#26443)
* refactor: remove unused Key type from write buffer

The write buffer had a Key variant for handling the experimental v3
write API that was phased out and removed from an earlier iteration
of influxdb3.

* refactor: remove key column type from last cache
2025-05-21 15:45:23 -04:00
praveen-influx 1ec063b0c4
feat: support named admin tokens (#26434)
* feat: support named admin tokens

- `--name` and `--expiry` are now allowed for `--admin` subcommand
- `--regenerate` is restricted to operator token only
- the endpoint is not allowed if auth is disabled

closes: https://github.com/influxdata/influxdb_pro/issues/854

This is a port of https://github.com/influxdata/influxdb_pro/pull/850 (hash:156981e4a1)

* refactor: address PR feedback
2025-05-20 15:30:19 +01:00
Stuart Carnie bf83e7fbb3
feat: `/ping` API contains versioning headers (#26433)
* feat: `/ping` API contains versioning headers

Further, the product version can be modified by updating the metadata in
the `influxdb3_process` `Cargo.toml`.

* chore: PR feedback

* chore: placate linter
2025-05-20 08:50:27 +10:00
peterbarnett03 b615e5c370
chore: update install_influxdb.sh (#26428) 2025-05-19 11:27:34 -04:00
Stuart Carnie a967e23171
chore: Export APIs necessary for future work in Pro (#26424) 2025-05-19 09:27:44 +10:00
Trevor Hilton 4822886495
docs: update release.md with additional details (#26427) 2025-05-16 13:38:04 -04:00
praveen-influx 1f076b69c8
feat: add trigger count to telemetry (#26426)
* feat: add trigger count to telemetry

closes: https://github.com/influxdata/influxdb/issues/26285

* refactor: do trigger counts by type
2025-05-16 17:18:26 +01:00
praveen-influx b404e8475c
fix: do not allow operator token from being deleted (#26418)
* fix: do not allow operator token from being deleted

closes: https://github.com/influxdata/influxdb_pro/issues/819

* refactor: address PR feedback

* fix: add a word and clarifying colon

* fix: failing test

---------

Co-authored-by: Peter Barnett <peter.barnett03@gmail.com>
2025-05-15 09:10:37 +01:00
Trevor Hilton 2a94f4232b
feat: query duration metrics in lvc (#26388) 2025-05-14 10:25:13 -04:00
praveen-influx 8aab3cc607
feat: allow health,ping,metrics to opt out of auth (#26406)
* feat: allow health,ping,metrics to opt out of auth

This commit introduces `--disable-authz <DISABLE_AUTHZ_RESOURCES>`. The
options for `DISABLE_AUTHZ_RESOURCES` are health, ping and metrics. By
default all these resources will be guarded

closes: https://github.com/influxdata/influxdb_pro/issues/774

* chore: update influxdb3/src/commands/helpers.rs

space after comma in help text

Co-authored-by: Trevor Hilton <thilton@influxdata.com>

* chore: update influxdb3/src/help/serve.txt

space after comma in help text

Co-authored-by: Trevor Hilton <thilton@influxdata.com>

* chore: update influxdb3/src/help/serve_all.txt

space after comma in help text

Co-authored-by: Trevor Hilton <thilton@influxdata.com>

* refactor: use statics to reduce clones/copies

---------

Co-authored-by: Trevor Hilton <thilton@influxdata.com>
2025-05-13 15:47:53 +01:00
Trevor Hilton ed80c852c2
refactor: use truncate in lvc to ensure elements removed (#26401)
This removes the `pop_back` methods from the lvc and uses truncate instead
so that it ensures that the cache is at its desired size.

Adjusted logic in the eviction of LVC to try to be a bit more efficient
2025-05-12 15:14:10 -04:00
Stuart Carnie 510325fff1
chore: Increase the visibility of some APIs to the crate (#26385)
These will be used in Enterprise.
2025-05-12 08:52:22 +10:00
praveen-influx 8a3d98a273
feat: support `Basic $TOKEN` for all apis (#26363)
* feat: support `Basic $TOKEN` for all apis

closes: https://github.com/influxdata/influxdb/issues/25833

* refactor: address PR feedback to return MalformedRequest error when `:` is used more than once in user-pass pair

* refactor: change the message sent back for malformed auth header
2025-05-09 18:11:37 +01:00
Trevor Hilton 9f66852590
feat: track object store metrics (#26365) 2025-05-08 12:32:12 -04:00
Trevor Hilton 0e946718a6
docs: add Building from source section to CONTRIBUTING.md (#26364) 2025-05-07 09:47:18 -04:00
Trevor Hilton 1cc03ed937
feat: catalog metrics to count operation types (#26357) 2025-05-06 20:08:03 -04:00
Trevor Hilton a2dc597d54
chore: update rust toolchain to 1.86.0 (#26355) 2025-05-05 17:33:26 -04:00
Trevor Hilton db27c69fb0
docs: remaining release process for docker/apt/yum (#26354) 2025-05-05 13:36:10 -04:00
peterbarnett03 403791e69a
chore: update install_influxdb.sh (#26352) 2025-05-03 12:25:38 -04:00
Trevor Hilton f34d99c61f
docs: update RELEASE.md for post-GA process (#26351) 2025-05-02 16:21:41 -04:00
Trevor Hilton d544073db2
chore: add -nightly to cargo version for snapshot releases (#26349)
This will have revision info displayed with `-nightly` added to the version, which
leads the most recent released version by 1 minor release.

For example,
* Using the `/ping` API:
```
❯ curl localhost:8181/ping
{"version":"3.1.0-nightly","revision":"f6aa0c1b29","process_id":"ccbd5879-a83b-4091-b68c-92c6b704cf8a"}
```
* Checking the binary version:
```
❯ influxdb3 --version
influxdb3 3.1.0-nightly, revision f6aa0c1b29
```
2025-05-02 12:11:50 -04:00
peterbarnett03 f6aa0c1b29
fix: update token creation output (#26336)
* fix: clarify token generation output for easier readibility

* fix: linter error

* fix: additional linter error

* fix: linter error on output

* fix: switch to owo_colorize
2025-04-30 12:11:33 -05:00
praveen-influx 08f2f857da
feat: additional info when started --without-auth (#26338)
- return 405 message body to indicate the endpoints are disabled
- extra log to say server has been started without auth
2025-04-29 08:06:44 -05:00
peterbarnett03 6a6743412a
feat: Remove Last Cache Size Limitation (#26333)
* feat: remove limit on LVC size

* fix: bad test case and incorrect info

* fix: more clarity and default value

* fix: light CLI polishes

* fix: bad snapshot
2025-04-28 13:40:18 -04:00
praveen-influx a3428261a3
feat: add `cluser_uuid`/`catalog_uuid` to telemetry (#26335)
closes: https://github.com/influxdata/influxdb_pro/issues/764
2025-04-28 12:37:37 -05:00
Trevor Hilton d30f26618c
fix: distinct cache counting bug (#26332)
* test: reproducer for #26318

* fix: distinct cache counting bug

Fixed an issue where the distinct cache was not counting rows correctly
for scenarios where a projection and predicate were used in conjunction
2025-04-25 16:40:25 -04:00
praveen-influx 9c8c7fe105
refactor: port changes from enterprise (#26329) 2025-04-25 18:08:07 +01:00
Trevor Hilton e977d263f8
chore: back-port changes from influxdb_pro#762 (#26323) 2025-04-25 10:02:08 -04:00
Trevor Hilton 85850cd394
feat: improve control of logs on test harness (#26325) 2025-04-24 21:09:00 -04:00
Trevor Hilton 0f52ebb90d
fix: group by tag columns with escape quotes (#26326)
* test: reproducer for #26216
* fix: group by tag columns with escape quotes
2025-04-24 20:31:14 -04:00
Michael Gattozzi fb8daa7860
feat: Add more TLS tests to the test suite (#26324)
* feat: Add a negative cert test

This adds a test that will panic on server startup because connections
to said server are invalid. We add a bad expired cert to our cert
generation for usage in our tests.

Note that this test is only really valid if other tests pass as it
depends on waiting for the server start checks to fail. If other
tests run then their server started fine and so did this one, the
only difference being that connections will error due to a bad tls cert.

Closes #26256

* feat: Add minimum TLS version test

This is a follow on to #26307. In this commit we add a test where we
check that connections only pass if TLS is set to v1.3. The default is
1.2 and other tests connect with that just fine. In this test we spin
up a server using only v1.3 as the minimum and try to connect with v1.2
which we expect to fail and then v1.3 which should pass.

Closes #26308
2025-04-24 18:11:39 -04:00
Michael Gattozzi e684fc1a24
feat: Add support for CORS requests from a browser (#26314)
This commit adds support for CORS by modifying our requests to make
preflight checks valid and to handle responses containing the necessary
headers for browsers to access the data they need. We keep what we
accept as open as this is essentially what requests to the server are
normally like and we gate the requests with an auth token.

Closes #26313
2025-04-24 15:16:18 -04:00
Stuart Carnie 8507bdc766
fix: Ensure `iox` table schema is sorted for SHOW TABLES (#26302)
* fix: Ensure `iox` table schema is sorted for SHOW TABLES

Closes #25860

* chore: Add insta snapshot

* chore: placate clippy
2025-04-23 09:05:06 +10:00
Stuart Carnie b0532dce87
fix: Consistent thread arguments with influxdb3_pro (#26304)
If the old environment variables are used, a warning will be displayed
on the console with the recommended replacement.
2025-04-23 09:04:32 +10:00
Michael Gattozzi af57abd6a9
feat: Allow setting a minimum TLS version (#26307)
This commit allows users to set a minimum TLS version. The default is
1.2. The choices are TLS 1.2 or TLS 1.3 which can be set via env var:

INFLUXDB3_TLS_MINIMUM_VERSION="tls-1.2"
or
INFLUXDB3_TLS_MINIMUM_VERSION="tls-1.3"

and for the command line flag for the serve command:

--tls-minimum-version tls-1.2

or

--tls-minimum-version tls-1.3

With this users have more fine grained control over what tls version
they require.

Closes #26255
2025-04-22 13:57:23 -04:00
Trevor Hilton 59c87aee7a
test: distinct cache ignores rows with nulls (#26306) 2025-04-22 11:43:49 -04:00
Michael Gattozzi f9721d3355
feat: Generate testing TLS certs on the fly (#26288)
This commit is a follow up to #26246 and generates test certs on the fly
for our test suite. In practice this will only need to be done once with
a fresh repo check out as the certs will expire long after anyone would
reasonably be working on this code in the year 4096! This could be
extended in the future to generate negative tls tests where the file
should be expired.

Closes #26254
2025-04-21 11:43:33 -04:00
peterbarnett03 c894451900
fix: update install script to use latest version and DockerHub (#26300)
* chore: update install script to use DockerHub

* fix: ensure script pulls latest images and uses DockerHub
2025-04-21 08:37:36 -05:00
praveen-influx b41a2d9bc3
feat: expose `--format json` for token creation cmd (#26286)
* feat: expose `--format json` for token creation cmd

closes: https://github.com/influxdata/influxdb/issues/25913

* refactor: address feedback
2025-04-18 08:06:02 +01:00
Paul Dix e9a27f1aab
chore: version bump for 3.0.1 (#26282) (#26284) 2025-04-17 10:04:16 -04:00
peterbarnett03 2ceed952b8
docs: clarify default maxes for distinct cache (#26281) 2025-04-16 15:29:30 -04:00