0dd9af8f13
fix: [2.4] Check legacy guarantee ts when skipping alloc ts ( #34982 )
...
Cherry-pick from master
pr: #34981
See also #34980
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-25 10:27:47 +08:00
4e95f3835a
fix: [2.4] `execute` method should return an error when result is a failure ( #34872 )
...
- issue: #34812
- pr: #34813
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-22 12:13:42 +08:00
ffee6b9c27
enhance: Optimize ConsistencyLevel and GuaranteeTimestamp log ( #34738 ) ( #34824 )
...
/kind improvement
pr: https://github.com/milvus-io/milvus/pull/34738
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-22 11:32:04 +08:00
d1e0d4ea4a
enhance: [2.4] Add dynamic cgo pool for proxy CGO call ( #34768 ) ( #34842 )
...
Cherry-pick from master
pr: #34768
Related to #34705
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-21 18:05:41 +08:00
34873a7f76
feat: record the duration waiting in the proxy queue ( #34744 ) ( #34788 )
...
fix: https://github.com/milvus-io/milvus/issues/34743
pr: #34744
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-07-19 15:37:44 +08:00
3cf2e1831a
feat: [cherry-pick][2.4] support partition key isolation ( #34336 ) ( #34613 )
...
issue: #34332
pr: #34336
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 104d0966b7
2024-07-17 14:03:43 +08:00
9cf1f03e89
fix: [2.4] the lint-fix error ( #34717 )
...
- issue: #31293
- pr: #34033
- related pr: #34699
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-16 17:37:39 +08:00
d607a43ea6
fix: [2.4] metaCache cleanup issue when listPolicy failed ( #34697 )
...
- issue:#34667
- pr: #34449
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-16 14:13:45 +08:00
0fdb288de7
enhance: upsert support autoid( #30342 ) ( #34633 )
...
pr: #30342
issue: #29258
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
2024-07-15 20:53:39 +08:00
9bcab2ebee
enhance: [2.4] use the insert rate limit for the upsert request ( #34616 )
...
- issue: #34614
- pr: #34615
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-12 16:19:36 +08:00
d6fc6a9f41
fix: [Cherry-pick] Restful API use deprecate error code cause access log panic. ( #34579 )
...
Avoid panic when use deprecate error code and use merr code replace
deprecate common code.
relate: https://github.com/milvus-io/milvus/issues/34578
pr： https://github.com/milvus-io/milvus/pull/34576
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-11 11:58:51 +08:00
f6cd84161c
enhance: ensure autoindex default metric type compatibility ( #34479 )
...
issue: #34304
pr: #34261
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-08 19:48:14 +08:00
326370c1be
enhance: add disk quota and max collections into db properties ( #34386 )
...
issue: https://github.com/milvus-io/milvus/issues/34385
pr: #34368
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-05 18:22:17 +08:00
ba2c232331
fix: [cherry-pick] [restful v2] count(*) & hook ( #34433 )
...
issue: #31224 #34374
pr: #34369
for query api:
1. param filter is not requried
2. param limit is useless while outputFields = [count(*)]
add hook about grpc call
---------
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-07-05 09:52:10 +08:00
9087b6f42e
enhance: [Cherry-Pick] support mark error as user error ( #33498 ) ( #34396 )
...
relate: https://github.com/milvus-io/milvus/issues/33492
pr: https://github.com/milvus-io/milvus/pull/33498
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-04 10:08:10 +08:00
014cb7b071
enhance: use configed max topk for iterator when input topk exceeds( #34292 ) ( #34293 )
...
related: #34292
pr: https://github.com/milvus-io/milvus/pull/34290
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-04 09:58:09 +08:00
b3c5eb29ed
enhance: [2.4] the proxy metric in the query request ( #34356 )
...
/kind improvement
- issue: #33306
- pr: #33307
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-03 10:00:09 +08:00
c62bf8a0b0
fix: [Cherry-pick]Pick major compaction fixs and optimizations ( #34360 )
...
This PR cherry-picks the following commits:
- fix: sync partitiion stats blocking balance task #33742
- fix: Fix meta prefix overlap bug #33830
- fix: Small fixs of major compaction #33929
- fix: Fix memory buffer error & some renaming #33850
- fix: sync part stats task cannot be finished #34027
- Add an option to enable/disable vector field clustering key #34097
- fix: fix error ignore in compactor #34169
- fix:load major compaction partial result #34052
- Use new stream segment reader in clustering compaction #34232
issue: #30633
pr: #33742 #33830 #33929 #33850 #34027 #34097 #34169 #34052 #34232
---------
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: Chun Han <116052805+MrPresent-Han@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-03 09:53:37 +08:00
99586066f5
feat: [cherry-pick] Major compaction ( #34326 )
...
This PR cherry-picks the following commits:
fix: speed up segment lookup via channel name in datacoord (#33530 )
needed by the next commit
feat: Major compaction (#33620 )
issue: #30633
pr: #33620
---------
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: yiwangdr <80064917+yiwangdr@users.noreply.github.com>
Co-authored-by: MrPresent-Han <chun.han@zilliz.com>
2024-07-02 18:29:01 +08:00
975f3bbeab
enhance: [2.4] Refine max length exceeded error message ( #34300 ) ( #34323 )
...
Cherry-pick from master
pr: #34300
This PR make varchar & string array field max length exceeded error
message clearer. Also fixed a minor issue that error string format and
argument number not match.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-02 15:52:09 +08:00
bc8ca593bd
enhance: [Cherry-Pick] Update access log ( #34295 )
...
Support restful api,print with consistency level and add new write
cache.
relate: https://github.com/milvus-io/milvus/issues/31823
pr: https://github.com/milvus-io/milvus/pull/33155
https://github.com/milvus-io/milvus/pull/33503
https://github.com/milvus-io/milvus/pull/32213
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-07-02 15:30:15 +08:00
c344083f22
enhance: Optimize grow slice cost during query ( #34254 )
...
issue: #32252
pr: #34253
This PR try to pre-allocate FieldData for Reduce operations in the Query
chain using typeutil.PrepareResultFieldData to avoid the overhead of
dynamically growing the slice during appendFieldData process.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-01 15:18:11 +08:00
0bd93de627
enhance: [cherry-pick]remove CheckVecIndexWithDataTypeExist function in pkg ( #34104 )
...
issue: https://github.com/milvus-io/milvus/issues/22837
related pr: https://github.com/milvus-io/milvus/pull/34102
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-26 21:14:26 +08:00
7f636dbdf6
enhance: refine error message for invalid limit parameter ( #34179 )
...
pr: #34178
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-26 20:46:06 +08:00
6423b6c718
enhance: move rocksmq from internal to pkg ( #34165 )
...
pr: https://github.com/milvus-io/milvus/pull/33881
issue: https://github.com/milvus-io/milvus/issues/33956
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-26 13:36:05 +08:00
1168e8c107
enhance: [2.4] support integral type for MV and skip MV if there is only one category ( #34005 )
...
pr: #33161
issue: #29892
---------
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
2024-06-24 19:12:02 +08:00
061a00c58f
enhance: Enable database level replica num and resource groups for loading collection ( #33052 ) ( #33981 )
...
pr: #33052
issue: #30040
This PR introduce two database level props:
1. database.replica.number
2. database.resource_groups
User can set those two database props by AlterDatabase API, then can
load collection without specified replica_num and resource groups. then
it will use database level load param when try to load collections.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-21 16:56:02 +08:00
2d3c5191b8
enhance: Refine name rule check error msg ( #33815 ) ( #33900 )
...
pr: #33815
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-19 14:26:01 +08:00
4513569207
enhance: add restful api to trigger component stop ( #32076 ) ( #33799 )
...
issue: #32698
pr: #32076
This PR add two rest api for component stop and status check:
1. `/management/stop?role=querynode` can stop the specified component
2. `/management/check/ready?role=rootcoord` can check whether the target
component is serviceable
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-06-17 21:53:59 +08:00
5fc1370f6f
enhance: [2.4] autoindex for multi data type ( #33867 )
...
issue: #22837
pr: https://github.com/milvus-io/milvus/pull/33868
- opensource autoindex support
- metric type check for different data types
- autoindex data type for search param
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-14 23:26:00 +08:00
562369627d
enhance: [cherry-pick]check index with data type in knowhere api ( #33878 )
...
issue: https://github.com/milvus-io/milvus/issues/22837
related: https://github.com/milvus-io/milvus/pull/33880
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
2024-06-14 19:45:58 +08:00
fd1c7b1a1c
enhance: enable flush rate limiter of collection level ( #33864 )
...
pr: #33837
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-06-14 16:27:57 +08:00
7ef28927e6
fix: [2.4] fix binary vector data size ( #33751 )
...
issue #22837
pr: #33750
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-06-12 20:11:55 +08:00
c331aa4ad3
enhance: [2.4] add the includeCurrentMsg param for the Seek method ( #33743 )
...
/kind improvement
- issue: #33325
- pr: #33326
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-06-11 15:01:55 +08:00
ed1dee9e38
enhance: Support L0 import ( #33514 ) ( #33712 )
...
issue: https://github.com/milvus-io/milvus/issues/33157
pr: https://github.com/milvus-io/milvus/pull/33514
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-06-08 11:17:52 +08:00
39e341e83a
fix: [2.4] update check for sparse hnsw index ( #33714 )
...
issue: #29419
pr: #33713
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-06-07 18:36:07 +08:00
7203ff8898
enhance: [2.4] Print `UseDefaultConsistency` param in read requests ( #33617 ) ( #33644 )
...
Cherry-pick from master
pr: #33617
`UseDefaultConsistency` param is crucial for debugging slow query
problems. It could be confusing when guarantee timestamp is 1 while this
param is not logged
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-06-06 14:19:56 +08:00
58a7111599
enhance: [cherry-pick] add autoindex mapping for binary/sparse datatype ( #33625 )
...
issue: #22837
pr: #33624
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-06-06 10:33:52 +08:00
627b787aed
fix: query iterator lack results( #33137 ) ( #33422 ) ( #33506 )
...
related: #33137
pr: https://github.com/milvus-io/milvus/pull/33422
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2024-06-05 18:51:52 +08:00
b205d7e29f
fix: Fix frequent 'failed to get request info' log ( #33334 ) ( #33337 )
...
issue: issue: https://github.com/milvus-io/milvus/issues/33333
pr: https://github.com/milvus-io/milvus/pull/33334
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-27 01:37:41 +08:00
e3f1331c47
fix: check array field data is nil before getting the type ( #33114 ) ( #33311 )
...
issue: https://github.com/milvus-io/milvus/issues/33074
pr: #33114
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-05-24 09:07:11 +08:00
e8c9bdfa5c
fix: Fix global rate limit is not working ( #33335 ) ( #33336 )
...
If the request is limited by rate limiter, limiter should not "Cancel".
This is because, if limited, tokens are not deducted; instead, "Cancel"
operation would increase the token count.
issue: https://github.com/milvus-io/milvus/issues/31705
pr: https://github.com/milvus-io/milvus/pull/33335
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-24 09:05:47 +08:00
62663ac2e2
fix: Fix printing type of request ( #33289 ) ( #33319 )
...
issue: https://github.com/milvus-io/milvus/issues/31705
pr: https://github.com/milvus-io/milvus/pull/33289
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-23 21:14:31 +08:00
37b2f90c90
enhance: [2.4] the panic when db isn't existed in the rate limit interceptor ( #33308 )
...
issue: #33243
pr: #33244
1. fix: the panic when db isn't existed in the rate limit interceptor
#33244
2. enhance: check the auth in some rest v2 api #33256
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-23 17:25:41 +08:00
ad4c1975bd
fix: Fix filtering by partition key fails for importing data ( #33274 ) ( #33277 )
...
Before executing the import, partition IDs should be reordered according
to partition names. Otherwise, the data might be hashed to the wrong
partition during import. This PR corrects this error.
issue: https://github.com/milvus-io/milvus/issues/33237
pr: https://github.com/milvus-io/milvus/pull/33274
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-23 11:25:40 +08:00
e5db2f7e77
fix: [2.4] Check schema without vector field in proxy ( #33211 ) ( #33218 )
...
Cherry-pick from master
pr: #33211
Related to #33199
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-21 16:37:44 +08:00
a27a2e8021
fix: try best to get enough query results ( #33178 ) ( #33179 )
...
issue: https://github.com/milvus-io/milvus/issues/33137
pr: #33178
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-21 14:07:45 +08:00
6b02c085f6
fix: [2.4] reset the quota value when init the limiter ( #33152 )
...
issue: #33107
pr: #33111
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-05-20 15:17:38 +08:00
6fc00e42d7
fix: Fix binlog import ( #33027 )
...
binlog import should only import into one partition.
issue: https://github.com/milvus-io/milvus/issues/32807
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-05-14 21:45:32 +08:00
4fc7915c70
enhance: unify data generation test APIs ( #32955 )
...
Issue: #22837
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
2024-05-14 14:33:33 +08:00
congqixia
SimFG
yihao.dai
Jiquan Long
Patrick Weizhi Xu
smellthemoon
aoiasd
Gao
jaime
PowderLi
Chun Han
wayblink
wei liu
cqy123456
Buqian Zheng
foxspy
Bingyi Sun
Cai Yudong