e486442ca6
enhance: Disallow the keywords as a field name or dynamic field name ( #36101 )
...
issue: #35873
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-15 15:19:14 +08:00
a61668c77e
feat: Introduce stats task for import ( #35868 )
...
This PR introduce stats task for import:
1. Define new `Stats` and `IndexBuilding` states for importJob
2. Add new stats step to the import process: trigger the stats task and
wait for its completion
3. Abort stats task if import job failed
issue: https://github.com/milvus-io/milvus/issues/33744
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-15 15:17:08 +08:00
5ce99e2535
[automated] Update Pytest image changes ( #36250 )
...
Update Pytest image changes
See changes:
2e434b2358sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-09-14 17:59:09 +08:00
e3e88ffbe6
test: update pymilvus version and skip some cases ( #36281 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-14 17:57:14 +08:00
2ff3765058
enhance: catch std::stoi exception and improve error msg ( #36267 )
...
issue: #36255
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-14 16:17:08 +08:00
3352030a84
enhance: Graceful stop flowgraph manager when stopping datanode ( #36229 )
...
Flowgraph manager is not stopped durong datanode stopping procedure
which may lead to unexpect flowgraph behavior during/after datanode stop
progress.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-14 15:55:08 +08:00
b8b4aea4f5
enhance: restrict max group size( #33544 ) ( #36223 )
...
related: #33544
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-09-14 15:45:08 +08:00
fc1bdd4c84
fix: to forbid bulk insert with nullable field in numpy files ( #36246 )
...
#36241
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-14 15:35:07 +08:00
329fb421cd
fix: fix search/query/count may access same growing and sealed segment ( #36258 )
...
issue: #36257
during syncTargetVersion, sealed segment should be excluded, to avoid
it's growing segment be conusmed from stream again.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-14 14:21:07 +08:00
d07338b4b9
enhance: Ignore index check for non-loaded vector field ( #36170 )
...
Related to #35996
For `Field Partial Load` feature, Milvus shall ignore index check for
non-loaded vector field.
Also, this PR unifies the logic of index check for load collection and
load partitions tasks.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-14 11:01:08 +08:00
2d05b7f219
fix: to add check in partition key field when set nullable==true ( #36218 )
...
#36213
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-14 10:53:08 +08:00
3bc7d63be9
fix: overwrite correct selection when pk duplicated ( #35826 )
...
Related to #35505
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-14 10:27:08 +08:00
ef8289ea9b
enhance: git clone with retry mechanism ( #36256 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-09-14 09:44:46 +08:00
9d8d332c88
fix: Fix improper use of offset in HybridSearch ( #36244 )
...
issue :https://github.com/milvus-io/milvus/issues/36243
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-09-13 22:05:15 +08:00
489087d18b
enhance: refactor executor framework V2 ( #35251 )
...
#32636
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-13 20:57:09 +08:00
22cce44afc
fix: metrics stored_index_files_size is never cleared ( #36160 )
...
issue: #36159
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-09-13 20:09:15 +08:00
58d3200986
enhance: Filter out non-hit delete records during load delta ( #36207 )
...
Related to #35303
This PR utilizes pk index in segment to exclude non-hit delete record
during load delete records. This ability is crucial when l0/delete
forward policy only replies on segment itself(without BF filtering).
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-13 19:05:08 +08:00
f7d950d465
fix: [skip e2e] Fix unstable ut TestCollectionObserver ( #36231 )
...
issue: #36237
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 19:01:09 +08:00
fb2a41a94c
fix: Clean dirty segment/channel on querynode ( #36202 )
...
issue: #36201
after querynode has been remove from replica, all dirty segment/channel
on it should be released.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 18:15:08 +08:00
b4c1603a57
fix: skip recover flusher on streaming node when channel is dropped ( #36149 )
...
issue: #36016,#36230
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-13 17:19:08 +08:00
bd658a6510
enhance: Enable dynamic update replica selection policy ( #35860 )
...
issue: #35859
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 17:05:15 +08:00
c03eb6f664
fix: streaming node consume blocks if recv message is too large ( #36151 )
...
issue: #36081
Signed-off-by: chyezh <chyezh@outlook.com>
2024-09-13 16:41:08 +08:00
2e434b2358
test: Add group size tests ( #36240 )
...
related issue: #36146
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-09-13 16:13:08 +08:00
c22a2cebb6
fix: split stream query result to avoid grpc response too large error ( #36090 )
...
relate: https://github.com/milvus-io/milvus/issues/36089
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-13 15:07:09 +08:00
f0f2fb4cf0
enhance: span tracing of c++ part ( #36205 )
...
fix: https://github.com/milvus-io/milvus/issues/36204
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-13 11:19:09 +08:00
06a706e5f0
fix: fix ReadWrite privilege group deny all global API ( #36144 )
...
issue: #35471
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-09-13 10:33:08 +08:00
c6eb6c7cb2
enhance: Add error handler for write buffer ( #36216 )
...
Related to #36215
This PR add error handler setting option providing the possibility to
change error handling behavior other than panicking.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-13 10:11:09 +08:00
b9c037f558
feat: adding cache to expression parse ( #36185 )
...
See #36122
This PR improves the proxy node performance by adding cache to
expression parse.
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-09-13 10:03:09 +08:00
1833f2d516
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-09-13 00:01:28 +00:00
8395c8a8db
enhance: Update stats task to optional ( #35947 )
...
issue: #33744
---------
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-09-12 20:37:08 +08:00
e8840a1b41
enhance: Add metrics for Delete entries num of L0seg ( #36175 )
...
- Add metrics *DataCoordL0DeleteEntriesNum*
- Remove metrics *DataCoordRateStoredL0Segment*
See also: #36147
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-09-12 18:07:08 +08:00
b5ff348906
test: Add part of test cases for default and null support ( #36186 )
...
issue : #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-12 17:29:13 +08:00
5e5e87cc2f
enhance: rename some params and reduce default bitmapCardinalityLimit… ( #36138 )
...
#32900
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-09-12 12:09:08 +08:00
11dbe1e755
enhance: Add L0 forward policy to support remote load ( #36189 )
...
Related to #35303
This PR add a param item to support change l0 forward behavior from bf
filtering and forward to remote load.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-12 12:01:08 +08:00
da227ff9a1
feat: Support create collection with functions ( #35973 )
...
relate: https://github.com/milvus-io/milvus/issues/35853
Support create collection with functions. Prepare for support bm25
function.
---------
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-09-12 10:43:06 +08:00
08e681174a
fix: [GoSDK] Sync empty result set handling ( #36177 )
...
Related to #33952
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-12 10:27:08 +08:00
700a443e85
enhance: Check ANN field loaded in proxy ( #36172 )
...
Related to #35415
Currently ANN field is loaded in Parital load is not performed in proxy,
this cause error message not clear and no error returned when collection
is empty
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-09-11 17:43:06 +08:00
e7ea1d7a04
enhance: improve log encoding performance on proxy nodes ( #36123 )
...
See #36122
This PR is designed to enhance log performance through two improvements:
1. Optimize JSON encoding by switching JSON serializer to
`json-iterator`.
2. Adding support of lazy initialization `WithLazy`.
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-09-11 14:51:07 +08:00
3f75bf1f20
fix: clustering compact not support null ( #36152 )
...
#36055
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-09-11 14:49:06 +08:00
f31264c02c
fix: Change l0SegmentsRowCount limits to a reasonable value ( #36014 )
...
See also: #36028
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-09-11 11:55:08 +08:00
d9c8d1ea90
test: Add part of test cases for default and null support ( #36130 )
...
issue: #36129
Signed-off-by: binbin lv <binbin.lv@zilliz.com>
2024-09-11 10:59:07 +08:00
dbe03a6151
test: add check after altering cardinality limit ( #36158 )
...
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
2024-09-11 10:49:06 +08:00
42eef4900d
test: update test cases and sdk version ( #36148 )
...
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-09-11 10:45:07 +08:00
229f969560
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-09-10 12:00:39 +00:00
2694551757
enhance: fix a security vulnerability ( #36154 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-09-10 19:09:27 +08:00
1d22de140a
enhance: [pipeline] list all status before uninstall helm release ( #36040 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-09-10 19:07:08 +08:00
29ddb10e8c
enhance: timer instead of ticker ( #36133 )
...
https://github.com/milvus-io/milvus/issues/36132
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-09-10 15:23:06 +08:00
9d10f378e1
enhance: [skip e2e] [mergify]replace branch by reference ( #36139 )
...
https://github.com/milvus-io/milvus/issues/36140
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-09-10 15:21:07 +08:00
9aa01c69f8
enhance: add a env ( #36142 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-09-10 15:19:13 +08:00
89bf226f0b
feat: support keyword text match ( #35923 )
...
fix : #35922
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-09-10 15:11:08 +08:00
cai.zhang
yihao.dai
sre-ci-robot
nico
jaime
congqixia
Chun Han
smellthemoon
wei liu
yellow-shine
zhenshan.cao
zhagnlu
Zhen Ye
yanliang567
aoiasd
Jiquan Long
Ted Xu
sre-ci-robot
XuanYang-cn
binbin
wt
CharlesFeng