Cherry-pick from master
pr: #29916
Delete detail log will be large and hard to read when log level is
debug. This PR change the log to stringer and print only pk range,
number.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
`flushInsertData` & `flushDeleteData` prints WARNING log even there is
no error returned. So move error branch into if block.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
pr: #29901
when apply dynamic config changes, we should format the value to proper
unit
This PR fix update rate limit config with wrong value.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Cherry-pick from master
pr: #29806
See also #29803
This PR:
- Add trace span for collection/partition load
- Use TraceSpan to generate Segment/ChannelTasks when loading
- Refine BaseTask trace tag usage
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Cherry-pick from master
pr: #29726
See also: #29113
Add a new utitliy function in `pkg/util/typetuil` to pre-allocate field
data slice capacity acoording to search limit. This shall avoid copying
the data during `AppendFieldData` when previous slice is out of space.
And shall also save CPU time during high paylog.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
pr: #29856
If segment has more than 128 log fils, drop segment will exceed etcd txn
ops limit, which will failed the drop segment request
This PR drop segment meta info with prefix, to avoid drop segment meta
failed
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #29709#291712
pr: #29836
to avoid concurrent recursive RLock and Lock cause deadlock, This PR
remove the unnecessary lock in config manager
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
When the TimeTravel functionality was previously removed, it
inadvertently affected the MVCC functionality within the system. This PR
aims to reintroduce the internal MVCC functionality as follows:
1. Add MvccTimestamp to the requests of Search/Query and the results of
Search internally.
2. When the delegator receives a Query/Search request and there is no
MVCC timestamp set in the request, set the delegator's current tsafe as
the MVCC timestamp of the request. If the request already has an MVCC
timestamp, do not modify it.
3. When the Proxy handles Search and triggers the second phase ReQuery,
divide the ReQuery into different shards and pass the MVCC timestamp to
the corresponding Query requests.
issue: #29656
pr: #29749
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
the growing segments contribute to this metric while inserting and
putting into the manager, but the current impl inserts data before
putting the segments into manager, which leads to double contributions
fix: #29766
pr: #29767
Signed-off-by: yah01 <yah2er0ne@outlook.com>
PR: https://github.com/milvus-io/milvus/pull/29797
enhance: We aim to cancel the previously initiated job or workflow
exclusively during the Pull Request (PR) stage
Signed-off-by: Sammy Huang <sammy.huang@zilliz.com>
- this much improve the performance for GPU index
- this also reduce 1x copy while parsing index meta
pr: #29678
Signed-off-by: yah01 <yang.cen@zilliz.com>
Once a role is granted to a user, the user should automatically possess
the privilege information associated with that role.
issue: #29710
pr: #29747
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Cherry-pick from master
pr: #29668
See also #29113
The collection schema is crucial when performing search/query but some
of the information is calculated for every request.
This PR change schema field of cached collection info into a utility
`schemaInfo` type to store some stable result, say pk field,
partitionKeyEnabled, etc. And provided field name to id map for
search/query services.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
issue: #29494
pr: #29496
1. link with install path's libblob-chunk-manager
2. performance of `ShouldBindWith` is better than `ShouldBindBodyWith`
3. the middleware shouldn't read the unrefreshed parameter repeatly
4. diff from old restful, named HandlersV1
Signed-off-by: PowderLi <min.li@zilliz.com>
this protect the loading index from failure, and speed up the loading
progress
pr: #29715
Signed-off-by: yah01 <yang.cen@zilliz.com>
Signed-off-by: yah01 <yah2er0ne@outlook.com>
issue: #29582
pr: #29574
This PR rewrite gen segment plan logic based on assign segment in
`score_based_balancer`
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
issue: #29672
master pr: #29703
the storage account need privileges of actions
`Microsoft.Storage/storageAccounts/blobServices/containers/blobs/*` at
least
Signed-off-by: PowderLi <min.li@zilliz.com>
zhenshan.cao
jaime
congqixia
aoiasd
wei liu
SimFG
wayblink
chyezh
yah01
sre-ci-robot
sammy.huang
XuanYang-cn
Jiquan Long
PowderLi
smellthemoon
cai.zhang
zhuwenxing
nico
foxspy
MrPresent-Han