milvus/internal/datanode/importv2
wei liu b08d9efe69
fix: Prevent delegator unserviceable due to shard leader change (#42689) (#43309)
issue: #42098 #42404
pr: #42689
Fix critical issue where concurrent balance segment and balance channel
operations cause delegator view inconsistency. When shard leader
switches between load and release phases of segment balance, it results
in loading segments on old delegator but releasing on new delegator,
making the new delegator unserviceable.

The root cause is that balance segment modifies delegator views, and if
these modifications happen on different delegators due to leader change,
it corrupts the delegator state and affects query availability.

Changes include:
- Add shardLeaderID field to SegmentTask to track delegator for load
- Record shard leader ID during segment loading in move operations
- Skip release if shard leader changed from the one used for loading
- Add comprehensive unit tests for leader change scenarios

This ensures balance segment operations are atomic on single delegator,
preventing view corruption and maintaining delegator serviceability.

---------

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2025-07-15 17:46:51 +08:00
..
hash.go enhance: [2.5] bulkinsert handles nullable/default (#42072) 2025-06-10 11:50:35 +08:00
hash_test.go fix: [2.5] Fix no candidate segments error for small import (#41772) 2025-05-14 10:36:22 +08:00
mock_task.go fix: [2.5] Fix no candidate segments error for small import (#41772) 2025-05-14 10:36:22 +08:00
pool.go enhance: [2.5] Use v2 package name for pkg module (#40117) 2025-02-23 00:46:01 +08:00
pool_test.go enhance: [2.5] Use v2 package name for pkg module (#40117) 2025-02-23 00:46:01 +08:00
scheduler.go enhance: [2.5] Use v2 package name for pkg module (#40117) 2025-02-23 00:46:01 +08:00
scheduler_test.go enhance: Add Size interface to FileReader to eliminate the StatObject call during Read (#42911) 2025-06-25 15:26:42 +08:00
task.go enhance: [2.5] Use v2 package name for pkg module (#40117) 2025-02-23 00:46:01 +08:00
task_import.go enhance: [2.5] Enhance import integration tests and logs (#42696) 2025-06-16 20:06:38 +08:00
task_l0_import.go enhance: [2.5] Enhance import integration tests and logs (#42696) 2025-06-16 20:06:38 +08:00
task_l0_import_test.go enhance: [2.5] Use v2 package name for pkg module (#40117) 2025-02-23 00:46:01 +08:00
task_l0_preimport.go enhance: [2.5] Enhance import integration tests and logs (#42696) 2025-06-16 20:06:38 +08:00
task_manager.go enhance: [2.5] Use v2 package name for pkg module (#40117) 2025-02-23 00:46:01 +08:00
task_manager_test.go enhance: [2.5] Use v2 package name for pkg module (#40117) 2025-02-23 00:46:01 +08:00
task_preimport.go enhance: [2.5] Enhance import integration tests and logs (#42696) 2025-06-16 20:06:38 +08:00
util.go fix: Prevent delegator unserviceable due to shard leader change (#42689) (#43309) 2025-07-15 17:46:51 +08:00
util_test.go enhance: [2.5] bulkinsert handles nullable/default (#42072) 2025-06-10 11:50:35 +08:00