Commit Graph

11 Commits (bugfix/run_triggers_on_start)

Author SHA1 Message Date
Paul Dix daa5cbd270
feat: better plugin errors for query and line building (#25850) 2025-01-16 16:57:30 -05:00
Paul Dix 1f71750bfe
feat: return better plugin execution errors (#25842)
* feat: return better plugin execution errors

This sets up the framework for fleshing out more useful plugin execution errors that get returned to the user during testing. We'll also want to capture these for logging in system tables.

Also fixes a test that was broken in previous commit on time limits. Didn't show up because of the feature flag.

* fix: compile errors without system-py feature
2025-01-16 15:58:05 -05:00
Trevor Hilton b8a94488b5
feat: support v1 query API GROUP BY semantics (#25845)
This updates the v1 /query API hanlder to handle InfluxDB v1's unique
query response structure when GROUP BY clauses are provided.

The distinction is in the addition of a "tags" field to the emitted series
data that contains a map of the GROUP BY tags along with their distinct
values associated with the data in the "values" field.

This required splitting the QueryExecutor into two query paths for InfluxQL
and SQL, as this allowed for handling InfluxQL query parsing in advance
of query planning.

A set of snapshot tests were added to check that it all works.
2025-01-16 11:59:01 -05:00
Paul Dix 4bfd95d068
feat: print plugin logs to the server (#25812) 2025-01-12 12:33:09 -05:00
Paul Dix ebf78aa6c9
feat: Updates to plugin API (#25799)
* Add uint64_field to LineBuilder
* Add bool_field to LineBuilder
* Change query_rows to query in Python API
* Update error output for test wal_plugin
2025-01-11 18:03:34 -05:00
Paul Dix e8422a240a
feat: Wire up arguments to wal plugin trigger (#25783)
This allows the user to specify arguments that will be passed to each execution of a wal plugin trigger. The CLI test was updated to check this end to end.

Closes #25655
2025-01-10 16:58:18 -05:00
Paul Dix 7230148b58
feat: Update WAL plugin for new structure (#25777)
* feat: Update WAL plugin for new structure

This ended up being a very large change set. In order to get around circular dependencies, the processing engine had to be moved into its own crate, which I think is ultimately much cleaner.

Unfortunately, this required changing a ton of things. There's more testing and things to add on to this, but I think it's important to get this through and build on it.

Importantly, the processing engine no longer resides inside the write buffer. Instead, it is attached to the HTTP server. It is now able to take a query executor, write buffer, and WAL so that the full range of functionality of the server can be exposed to the plugin API.

There are a bunch of system-py feature flags littered everywhere, which I'm hoping we can remove soon.

* refactor: PR feedback
2025-01-10 05:52:33 -05:00
Paul Dix 2d18a61949
feat: Add query API to Python plugins (#25766)
This ended up being a couple things rolled into one. In order to add a query API to the Python plugin, I had to pull the QueryExecutor trait out of server into a place so that the python crate could use it.

This implements the query API, but also fixes up the WAL plugin test CLI a bit. I've added a test in the CLI section so that it shows end-to-end operation of the WAL plugin test API and exercise of the entire Plugin API.

Closes #25757
2025-01-09 20:13:20 -05:00
Paul Dix 1ce6a24c3f
feat: Implement WAL plugin test API (#25704)
* feat: Implement WAL plugin test API

This implements the WAL plugin test API. It also introduces a new API for the Python plugins to be called, get their data, and call back into the database server.

There are some things that I'll want to address in follow on work:
* CLI tests, but will wait on #25737 to land for a refactor of the CLI here
* Would be better to hook the Python logging to call back into the plugin return state like here: https://pyo3.rs/v0.23.3/ecosystem/logging.html#the-python-to-rust-direction
* We should only load the LineBuilder interface once in a module, rather than on every execution of a WAL plugin
* More tests all around

But I want to get this in so that the actual plugin and trigger system can get udated to build around this model.

* refactor: PR feedback
2025-01-06 17:32:17 -05:00
Jackson Newhouse 8bfccb74ab
feat(processing_engine): Runtime and write-back improvements (#25672)
* Move processing engine invocation to a seperate tokio task.
* Support writing back line protocol from python via insert_line_protocol().
* Update structs to work with bincode.
2024-12-17 16:38:12 -08:00
Jackson Newhouse 486d79d801
feat(processing_engine): initial implementation of Processing Engine plugins and triggers (#25639) 2024-12-13 14:11:38 -08:00