Commit Graph

3 Commits (master)

Author SHA1 Message Date
yihao.dai 142bd2fc05
enhance: Pooling for data tasks (#41256)
1. Add global scheduler for datacoord.
2. Define and implement new CreateTask, QueryTask, DropTask interfaces.
3. Refine Import, Compaction, Stats, Index task.

issue: https://github.com/milvus-io/milvus/issues/41123

Co-authored-by: Cai Zhang <cai.zhang@zilliz.com>
2025-05-20 21:06:24 +08:00
XuanYang-cn e6c46a25ea
enhance: Use correct counter metrics for overall wa calculation (#40394)
- Use CounterVec to calculate sum of increase during a time period.
- Use entries number instead of binlog size

---------

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-03-10 16:34:06 +08:00
XuanYang-cn 837ac295fa
enhance: Remove iterators in datanode (#40301)
Iterators are long deprecated, but sort are still using it. This PR
unifies stats task with the latest compaction common functions and
remove the usage of iterators.

1. Rename `datanode/compaction` to `datanode/compactor`
2. Add `internal/compaction` and move some compaction commons into it.
3. Replace `DeltalogIterators` with `ComposeDeleteFromDeltalogs`
4. Remove `datanode/iterators`

See also: #39242

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2025-03-04 12:14:00 +08:00