1bb4c08067
Changes the default ingester configuration to assign half the logical cores to datafusion for persist execution. Prior to this commit, datafusion always used 4 threads by default. In situations where the ingesters are configured with 4 logical cores or less, the periodic persist can start enough persist jobs to keep the 4 threads assigned to datafusion busy. Because there are enough threads to saturate all CPU cores, these CPU-heavy persist threads can impact write latency by stealing CPU time from the tokio runtime threads. This change assigns exactly half the threads to DF by default, ensuring there's always N/2 cores to service I/O heavy API requests. |
||
|---|---|---|
| .. | ||
| src | ||
| .gitattributes | ||
| Cargo.toml | ||
| README.md | ||
| build.rs | ||
README.md
workspace-hack
This crate is a "workspace hack" crate managed by cargo hakari.
Its purpose is to unify the features used by all crates in the workspace so that the crates share more dependencies and rebuild crates less. There are more details in hakari's documentation.
CI failures
If the workspace_hack_checks CI job is failing, there are two possible reasons and solutions:
- If
cargo hakari generate --difffails, that means a crate has started or stopped using a feature of some crate and that feature isn't up-to-date in theworkspace-hackcrate. To fix this, runcargo hakari generateand commit the changes. - If
cargo hakari manage-deps --dry-runfails, that means a crate in the workspace isn't depending on theworkspace-hackcrate. To fix this, runcargo hakari manage-depsand commit the changes.