Commit Graph

10 Commits (master)

Author SHA1 Message Date
yihao.dai 192521c6bd
enhance: Fix unbalanced task scheduling (#43581)
Make scheduler always pick the node with the most available slots.

issue: https://github.com/milvus-io/milvus/issues/43580

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-07-28 12:58:55 +08:00
Zhen Ye e9ab73e93d
enhance: add schema version at recovery storage (#43500)
issue: #43072, #43289

- manage the schema version at recovery storage.
- update the schema when creating collection or alter schema.
- get schema at write buffer based on version.
- recover the schema when upgrading from 2.5.

---------

Signed-off-by: chyezh <chyezh@outlook.com>
2025-07-23 21:38:54 +08:00
cai.zhang f19e0ef6e4
fix: Ensure task execution order by using a priority queue (#43271)
issue: #43260

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-07-22 17:42:53 +08:00
cai.zhang 5566a85bcc
enhance: Add proxy task queue metrics (#42156)
issue: #42155

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2025-06-04 11:26:32 +08:00
wei liu 5a355d1e57
fix: Fix data race in global scheduler test using atomic counters (#42454)
issue: #42457

Replace unsafe ExpectedCalls modification with atomic.Int32 state
tracking to avoid race conditions in concurrent test execution. Changes
include:
- Use atomic counters instead of direct mock ExpectedCalls manipulation
- Add RunAndReturn with atomic state transitions for thread safety
- Remove github.com/samber/lo dependency

This prevents data race when mock framework and test goroutines access
ExpectedCalls concurrently.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-06-03 14:18:30 +08:00
yihao.dai 297331b2cc
enhance: Add slot and tasks num metrics (#42141)
issue: https://github.com/milvus-io/milvus/issues/41123

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-05-30 21:52:30 +08:00
yihao.dai 79b51cbb73
fix: Fix task getting stuck after recovery (#42114)
Submit tasks into the global scheduler after recovery.

issue: https://github.com/milvus-io/milvus/issues/42046

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-05-28 12:46:28 +08:00
yihao.dai 83c9527e70
enhance: Use QuerySlot interface for tasks (#41989)
Use `QuerySlot` rpc instead of `QueryTask` for querying slot.

issue: https://github.com/milvus-io/milvus/issues/41123

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-05-23 10:30:28 +08:00
yihao.dai e04e5b41ca
enhance: Add task version monitoring (#42023)
issue: https://github.com/milvus-io/milvus/issues/41123

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2025-05-22 23:24:28 +08:00
yihao.dai 142bd2fc05
enhance: Pooling for data tasks (#41256)
1. Add global scheduler for datacoord.
2. Define and implement new CreateTask, QueryTask, DropTask interfaces.
3. Refine Import, Compaction, Stats, Index task.

issue: https://github.com/milvus-io/milvus/issues/41123

Co-authored-by: Cai Zhang <cai.zhang@zilliz.com>
2025-05-20 21:06:24 +08:00