Commit Graph

36 Commits (fix_build-docker-arm64)

Author SHA1 Message Date
Stuart Carnie 58b0725cf9
fix: Existing soft-deleted schema can be hard-deleted (#26574)
* feat: Allow hard_deleted date of deleted schema to be updated

* feat: Include hard_deletion_date in `_internal` `databases` and `tables`

* feat: Unit tests for testing deleted databases

* chore: Default is now to hard-delete with default duration

* test: Update test names and assertions for new default hard deletion behavior

- Renamed delete_table_defaults_to_hard_delete_never to delete_table_defaults_to_hard_delete_default
- Renamed delete_database_defaults_to_hard_delete_never to delete_database_defaults_to_hard_delete_default
- Updated assertions to expect default deletion duration instead of None/Never
- Aligns with the change of HardDeletionTime default from Never to Default

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: Remove TODO

* chore: PR feedback and other improvements

* Ensure system databases and tables schema specify a timezone for the
  `hard_deletion_time` Timestamp columns (otherwise they display without
  a timezone)
* `DELETE` using `default` delay is idempotent, so multiple requests
  will not continue to update the `hard_deletion_time`
* Improved test coverage for these behaviours

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-07-03 07:26:13 +10:00
praveen-influx 4694c83c86
fix: --object-store is explicitly marked required (#26575)
Before this commit, although `--object-store` is mandatory it is not
reflected in the error messages. The examples are listed in the issue
976.

This commit makes `object_store` explicitly required which means error
messages include `--object-store` listed as mandatory

closes: https://github.com/influxdata/influxdb_pro/issues/976
2025-07-01 17:23:04 +01:00
Stuart Carnie d396921aa0
chore: Update to Rust 1.88 (#26567)
* chore: Update to Rust 1.88

* chore: Fixes missed by Claude
2025-06-27 07:38:45 +10:00
Michael Gattozzi d07d2f75a2
feat: Add retention period to cli commands (#26520)
This commit touches quite a few things, but the main changes that need
to be taken into account are:

- An update command has been added to the CLI. This could be further
  extended in the future to update more than just Database retention
  periods. The API call for that has been written in such a
  way as to allow other qualities of the database to be updated
  at runtime from one API call. For now it only allows the retention
  period to be updated, but it could in theory allow us to rename a
  database without needing to wipe things, especially with a stable ID
  underlying everything.
- The create database command has been extended to allow
  its creation with a retention period. In tandem with the update
  command users can now assign or delete retention periods at will
- The ability to query catalog data about both databases and tables has
  been added as well. This has been used in tests added in this commit,
  but is also a fairly useful query when wanting to look at things such
  as the series key. This could be extended to a CLI command as well if
  we want to allow users to look at this data, but for now it's in the
  _internal table.

With these changes a nice UX has been created to allow our customers to
work with retention periods.
2025-06-16 10:22:45 -04:00
Trevor Hilton 419d80ddbb
fix: do not allow _ in db names on API-based create (#26507)
Prevents database names that start with `_` when created explicitly
through the Create Database or Create Table APIs.
2025-06-09 12:52:05 -04:00
Trevor Hilton d1c10f4b29
fix: backfill new tags with NULL instead of empty string (#26446)
* fix: backfill new tags with NULL instead of empty string

* refactor: use helper for append_null

* test: add a test to check null back/forward fill
2025-05-21 17:23:51 -04:00
praveen-influx 1ec063b0c4
feat: support named admin tokens (#26434)
* feat: support named admin tokens

- `--name` and `--expiry` are now allowed for `--admin` subcommand
- `--regenerate` is restricted to operator token only
- the endpoint is not allowed if auth is disabled

closes: https://github.com/influxdata/influxdb_pro/issues/854

This is a port of https://github.com/influxdata/influxdb_pro/pull/850 (hash:156981e4a1)

* refactor: address PR feedback
2025-05-20 15:30:19 +01:00
praveen-influx b404e8475c
fix: do not allow operator token from being deleted (#26418)
* fix: do not allow operator token from being deleted

closes: https://github.com/influxdata/influxdb_pro/issues/819

* refactor: address PR feedback

* fix: add a word and clarifying colon

* fix: failing test

---------

Co-authored-by: Peter Barnett <peter.barnett03@gmail.com>
2025-05-15 09:10:37 +01:00
peterbarnett03 f6aa0c1b29
fix: update token creation output (#26336)
* fix: clarify token generation output for easier readibility

* fix: linter error

* fix: additional linter error

* fix: linter error on output

* fix: switch to owo_colorize
2025-04-30 12:11:33 -05:00
praveen-influx 08f2f857da
feat: additional info when started --without-auth (#26338)
- return 405 message body to indicate the endpoints are disabled
- extra log to say server has been started without auth
2025-04-29 08:06:44 -05:00
Trevor Hilton e977d263f8
chore: back-port changes from influxdb_pro#762 (#26323) 2025-04-25 10:02:08 -04:00
praveen-influx b41a2d9bc3
feat: expose `--format json` for token creation cmd (#26286)
* feat: expose `--format json` for token creation cmd

closes: https://github.com/influxdata/influxdb/issues/25913

* refactor: address feedback
2025-04-18 08:06:02 +01:00
Stuart Carnie 5da3850319
fix: `--tags` argument is optional and now requires at least one tag (#26275)
* fix: `--tags` argument is optional and now requires at least one tag

Closes #25824

* chore: Added CLI unit tests
2025-04-16 11:46:13 -04:00
praveen-influx 0cb5f9077c
feat: ask for confirmation when regenerating token (#26258) 2025-04-13 09:07:09 +01:00
praveen-influx e5af9f95d2
feat: enable auth by default (#26243)
* feat: enable auth by default

- Removes `--bearer-token` support and starts the server with auth by
  default.
- Adds `--without-auth` switch to start the server without any auth

* feat: changes for auth being turned off

when auth is turned off,
- disallow token endpoints (returns 405)
- remove hash column when querying tokens system table

* refactor: address PR feedback
2025-04-11 16:44:27 +01:00
Trevor Hilton a1923d9304
fix: build after some merge regressions (#26248) 2025-04-10 16:20:11 -04:00
praveen-influx 64eeb09687
feat: allow token deletes (#26247)
This commit allows deletion of tokens by name. Below is an example,

`influxdb3 delete token --token-name _admin --token $CURRENT_ADMIN_TOKEN`

It needs user confirmation before proceeding with the delete
2025-04-10 20:31:19 +01:00
Michael Gattozzi fe69793418
feat: Add TLS support for influxdb3 (#26246)
This commit adds TLS support to influxdb3 and allows users to pass in a
path to a key and cert file with the --tls-key and --tls-cert flags in
the serve command. It also adds the ability for every command to specify
a certificate authority for requests. This is mostly needed when the
cert is self signed, but there are other use cases for this.

The big thing is that most of our tests now use TLS by default. Included
are self signed certs for localhost and the the CA cert included in the
commit. Since these are *only* used for testing this should be fine to
include as they are not used in nor are they intended to be used in any
production system. The expiry has been set for 365 days and the file
perms are set to o600 like the original issue mentioned. The tests pass
with this restriction.

I've verified that the API works via curl with the self signed certs as
I did *not* need to pass in the -k option to bypass checking the certs
were valid. The same goes for our tests. They use the rootCA.pem file
to verify the self signed cert when connecting and reject it otherwise.

With this users can be confident that their queries are safely encrypted
during transport.

Note that TLS works for both FlightSQL and our normal APIs.

Closes #25774
2025-04-10 13:45:47 -04:00
praveen-influx 1983818e36
feat: porting token work from enterprise (#26239)
* feat: generate persistable admin token

- this commit allows admin token creation using `influxdb3 create token
  --admin` and also allows regeneration of admin token by `influxdb3
  create token --admin --regenerate`
- `influxdb3_authz` crate hosts all low level token types and behaviour
- catalog log and snapshot types updated to use the token repo
- tests that relied on auth have been updated to use the new token
  generation mechanism and new admin token generation/regeneration tests
  have been added

* feat: list admin tokens

- allows listing admin tokens
- uses _internal db for token system table
- mostly test fixes due to _internal db
2025-04-09 16:31:59 +01:00
Trevor Hilton c7854363c4
chore: back-port changes to shutdown code from enterprise (#26206)
* refactor: make ShutdownManager Clone

ShutdownManager can be clone since its underlying types from tokio are
all shareable via clone.

* refactor: make ShutdownToken not Clone

Alters the API so that the ShutdownToken is not cloneable. This will help
ensure that the Drop implementation is invoked from the correct place.
2025-04-01 11:32:23 -04:00
Jackson Newhouse d245b2e886
fix(processing_engine): properly extract data from DictionaryArrays (#26204) 2025-03-31 11:38:23 -07:00
Trevor Hilton 24887770ef
feat: shutdown on WAL overwritten (#26203)
* feat: trigger shutdown if wal has been overwritten

WAL persist uses PutMode::Create in order to invoke shutdown if another
process writes to the WAL ahead of it.

A test was added to check that it works from CLI test suite.

* chore: clippy
2025-03-31 12:58:18 -04:00
Jackson Newhouse 7c02593fef
chore: Overhaul CLI tests for better ergonomics. (#26168)
* chore: Overhaul CLI tests for better ergonomics.

* chore: switch with_query() to with_sql().
2025-03-20 09:19:12 -07:00
Trevor Hilton 3ad0670452
test: small change to distinct cache cli test (#26152)
Added one check to the distinct cache create test that checks duplicate
creation behaviour while investigating influxdb_pro#601
2025-03-17 11:05:25 -04:00
Jackson Newhouse e49f0e89fa
fix(processing_engine): Change how temporary plugin files are managed. (#26141) 2025-03-14 10:52:38 -07:00
Jackson Newhouse e1015f15f7
feat(processing_engine): proper escaping of LineBuilder arguments. (#26134) 2025-03-14 09:31:58 -07:00
Michael Gattozzi 97b8c471f8
fix: Make sure JSON fully closes on empty records (#26130) 2025-03-12 14:01:40 -04:00
Jackson Newhouse a455598be6
feat(processing_engine): Implement Processing Engine Cache. (#26111) 2025-03-11 15:48:47 -07:00
Trevor Hilton 72dc4458fd
chore: backport changes to catalog from enterprise (#26116)
* chore: backport changes to influxdb3_catalog crate

* chore: backport changes to influxdb3_cache crate

* chore: backport changes to influxdb3_write crate

* chore: backport changes to influxdb3_proc_eng crate

* chore: backport influxdb3 crate changes for catalog

* chore: backport changes to influxdb3_id crate

* chore: backport changes to influxdb3_wal crate

* chore: backport changes to influxdb3_clap_blocks crate

* chore: backport changes to influxdb3_client crate

* chore: backport influxdb3_server crate changes

* chore: fix after full backport

* fix: ordering of catalog broadcast
2025-03-11 12:11:51 -04:00
Jackson Newhouse 5fa417c3f0
feat: remove system-py (#26087)
* feat: remove system-py

* chore: allow Apache-2.0 WITH LLVM-exception license.
2025-03-10 11:10:33 -07:00
Jackson Newhouse 357c05f3d0
feat(processing_engine): Support Flask semantics for responses from request plugins. (#26095) 2025-03-04 09:00:59 -08:00
wayne e2c3e7c8c8
feat: improve CLI list arg consistency (always use commas) (#26088)
* chore: eliminate SeparateList, use comma-delimited flag list options everywhere
* fix: remove num_args attribute from comma-delimited args
2025-03-03 12:52:13 -07:00
Jackson Newhouse 208b82a32d
fix(processing_engine): Use default globals so that builtins are automatically included. (#26072) 2025-02-27 16:11:08 -08:00
Michael Gattozzi 1f72bfcc33
feat: Update to Rust 1.85 and 2024 Edition (#26046) 2025-02-20 14:58:07 -05:00
Jackson Newhouse d9dd8a32a2
fix(processing_engine): Use the configured request path for Request plugins. (#25945) 2025-01-31 10:36:47 -08:00
Michael Gattozzi b7d2475ea6
chore: move separate cli and server tests (#25934)
This commit restructures our tests to look like Enterprise in their
layout. We break cli.rs into it's own module, combine the server tests
and cli tests under one lib.rs file and handle the changes for
visibility and import paths needed to make things work. the packages
tests have been cfged out as a module so that it would not need to be
added on a per test basis. Note that those tests fail locally for me
currently, but it seems like we weren't testing these in CI at the
moment.

There is no issue for this.
2025-01-31 11:41:44 -05:00