issue: #31063
Increase the graceful time to ensure that bounded query requests can be
executed normally in the event of RootCoord failure.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
This PR introduces novel managerial roles for importv2:
1. ImportMeta: To manage all the import tasks;
2. ImportScheduler: To process tasks and modify their states;
3. ImportChecker: To ascertain the completion of all tasks and instigate
relevant operations.
issue: https://github.com/milvus-io/milvus/issues/28521
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
If DC restarted, those unkonwn compaction tasks
will never get call back in DN, so that the segments in the compaction
task will be locked, unable to sync and compaction again, blocking cp
advance and compaction executing.
See also: #30137
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
issue: #30715
- Bug: Set nil struct pointer to describe nil interface.
Panic with segment violation when calling method on this nil struct
pointer.
Signed-off-by: chyezh <chyezh@outlook.com>
This PR mainly improve two items:
1. Target observer should refresh loading status during init time. An
uninitialized loading status blocks search/query. Currently, the target
observer refreshes every 10 seconds, i.e. we'd need to wait for 10s for
no reason. That's also the reason why we constantly see false log
"collection unloaded" upon mixcoord restarts.
2. Delete session when service is stopped. So that the new service
doesn't need to wait for the previous session to expire (~10s).
Item 1 is the major improvement of this PR, which should speed up init
time by 10s.
Item 2 is not a big concern in most cases as coordinators usually shut
down after stop(). In those cases, coordinator restart triggers serverID
change which further triggers an existing logic that deletes expired
session. This PR only fixes rare cases where serverID doesn't change.
integration test:
`go test -tags dynamic -v -coverprofile=profile.out -covermode=atomic
tests/integration/coordrecovery/coord_recovery_test.go -timeout=20m`
Performance after the change:
Average init time of coordinators: 10s
Hardware: M2 Pro
Test setup: 1000 collections with 1000 rows (dim=128) per collection.
issue: #29409
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
This pr decoups importing segment from flush process by:
1. Exclude the importing segment from the flush policy, this approch
avoids notifying the datanode to flush the importing segment, which may
not exist.
2. When RootCoord call Flush, DataCoord directly set the importing
segment state to `Flushed`.
issue: https://github.com/milvus-io/milvus/issues/30359
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
don't store logPath in meta to reduce memory, when service get
segmentinfo, generate logpath from logid.
#28885
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
currently integration test may timeout if any case run time is above 3
minutes. This duration was hard coded.
This PR change this duration into a customized parameter and could be
passed via test running commands.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
This pull request simplifies the integration test for cross-cluster
routing by reusing `integration.MiniClusterSuite`, instead of defining
custom Milvus clients, servers, and etcd client.
issue: https://github.com/milvus-io/milvus/issues/29874
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
See also: #29113
Add a new utitliy function in `pkg/util/typetuil` to pre-allocate field
data slice capacity acoording to search limit. This shall avoid copying
the data during `AppendFieldData` when previous slice is out of space.
And shall also save CPU time during high paylog.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Memory management is broken due caused by chunkmanager not getting
cleaned up.
Fixing the following error:
```
SIGSEGV: segmentation violation
PC=0x10fc4aa0c m=6 sigcode=2
```
relate: https://github.com/milvus-io/milvus/issues/29507
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
cai.zhang
jaime
yihao.dai
chyezh
XuanYang-cn
Jiquan Long
yiwangdr
aoiasd
smellthemoon
congqixia
wayblink
Xu Tong
xige-16
Filip Haltmayer
SimFG
Enwei Jiao
Gao
Cai Yudong
wei liu