Commit Graph

8 Commits (f01382f578703c3203936e13014e1827607ca826)

Author SHA1 Message Date
Dom Dwyer 7c5ba34d44 refactor: enable gRPC handler
Plumbs the gRPC write handler into the existing router2 server.
2022-03-04 14:51:43 +00:00
Dom Dwyer bb9b140f4b refactor: sequencer metrics
Records per-sequencer (kafka partition) enqueue latency / counts broken
down by operation success/error.
2022-03-03 23:40:13 +00:00
Carol (Nichols || Goulding) 3f2a58b47f
refactor: pub use data_types from data_types2
So it's clearer which parts of data_types the NG design is using, and
which types can be cleaned up eventually.
2022-03-02 13:55:31 -05:00
Carol (Nichols || Goulding) 8f3e44bf76
refactor: Extract a crate for shared data types in the new design 2022-03-02 12:16:15 -05:00
Dom Dwyer bd64f55658 feat: http ingest metrics
Records LP line count, field count & request body size (decompressed,
byte size) for writes, and request body byte size for deletes.
2022-03-02 13:05:55 +00:00
Marco Neumann 48722783f9
feat: offer metrics for in-mem catalog (#3876)
This can be quite helpful to test certain caching behavior w/o writing
yet-another abstraction layer.
2022-03-01 11:33:54 +00:00
Dom Dwyer d7eda88581 refactor: early schema validation
Changes the configuration of the router request pipeline to move schema
validation before partitioning.

This reduces the concurrency of callsm into the schema validator when a
single write is split into one or more partitions, reducing contention
and cash thrashing. It also ensures we don't bother partitioning the
writes if the request will fail.
2022-02-23 18:59:14 +00:00
Dom Dwyer 497615d715 test: router handler stack integration test
Adds an integration test covering the router's HTTP handler stack.

Given a well-formed HTTP write, the test asserts:

    * Write passes through the stack without error
    * Response code sent to client
    * Write buffer message is enqueued
    * Catalog namespace record is created
    * Metric handler is invoked and the hit is recorded
2022-02-18 14:39:03 +00:00