9dc1311f38
enhance: add the param to control whether to include the current msg ( #35656 )
...
/kind improvement
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-23 15:56:57 +08:00
59387f0b3a
fix: Use correct release func when datanode unwatch a channel ( #35655 )
...
See also #35654
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-23 14:09:03 +08:00
e8e3544a11
enhance: add hit segment num metrics for queryHook ( #35577 )
...
issue: #35576
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-08-23 12:49:02 +08:00
a773836b89
enhance: optimize milvus core building ( #35610 )
...
issue: #35549,#35611,#35633
- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 12:35:02 +08:00
ab532ae199
enhance: Add back BF lazy load logic for datanode watch channel ( #35646 )
...
Add back lazy loading statslog when watch dml channel on datanode.
Related to #22994 #27675
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 19:42:57 +08:00
5c245d51c4
enhance: Refresh proxy cache after restore rbac meta ( #35635 )
...
issue: #35443
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
9fff07936a
fix: Re-read value after `once` initialization ( #35642 )
...
Related to #35641
See also #35271
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 17:24:58 +08:00
582d2eec79
enhance: Move datanode/indexnode manager to session pkg ( #35634 )
...
Related to #28861
Move session manager, worker manager to session package. Also renaming
each manager to corresponding node name(datanode, indexnode).
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 16:02:56 +08:00
3107701fe8
enhance: optimize retrieve on dynamic field ( #35580 )
...
#35514
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-22 14:24:56 +08:00
570a8879e7
test: update gosdk case for valid empty sparse vector ( #35621 )
...
/kind improvement
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-22 10:54:56 +08:00
c992a61a23
enhance: Separate allocator pkg in datacoord ( #35622 )
...
Related to #28861
Move allocator interface and implementation into separate package. Also
update some unittest logic.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 10:06:56 +08:00
024eccbde0
enhance: add pkg-config for knowhere ( #35433 )
...
Signed-off-by: yusheng.ma <yusheng.ma@zilliz.com>
2024-08-22 09:56:56 +08:00
6fa6e2f978
Update all contributors
...
Signed-off-by: sre-ci-robot <sre-ci-robot@zilliz.com>
2024-08-21 12:00:50 +00:00
2b6e6f4040
feat: [skip e2e] effective way to use cache ( #35598 )
...
issue: https://github.com/milvus-io/milvus/issues/34876
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-21 18:46:55 +08:00
ed4eaffc9d
enhance: add csv support for bulkinsert ( #34938 )
...
See this issue for details: #34937
---------
Signed-off-by: OxalisCu <2127298698@qq.com>
2024-08-21 17:47:01 +08:00
ba6db117e3
enhance: add some integration test in null ( #35599 )
...
#31728
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-21 17:44:56 +08:00
cf242f9e09
fix: fix dynamic update config doesn't works for some param ( #35572 )
...
issue: #35570
milvus support config cache to spped up config access, but only evict
param's cache when param has been updated. but milvus's param may rely
on other param's value, let's say ParamsA relys on paramsB, when paramsB
updated, it will evict paramB's cache, but the paramA's cache still keep
the old value.
This PR evict all config cache to solve the above issue, cause dynamic
update config won't be much frequetly.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-21 11:02:56 +08:00
7ac339ac64
test: Update init collection method ( #35596 )
...
Related issue: #32653
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-08-21 09:22:56 +08:00
731d45abbe
enhance: provide more general configuration to control mmap behavior ( #35359 )
...
- issue: #35273
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00
99f8c64c31
enhance: [GoSDK] support Bitmap Index ( #35585 )
...
See also #32900
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 19:44:55 +08:00
41646c8439
feat: integrate new deltalog format ( #35522 )
...
See #34123
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-08-20 19:06:56 +08:00
bc2bbdc82f
enhance: Forbid access log print error message with line break ( #34491 )
...
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2024-08-20 17:36:55 +08:00
80a7c78f28
enhance: import supports null in parquet and json formats ( #35558 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-20 16:50:55 +08:00
2fbc628994
feat: Support field partial load collection ( #35416 )
...
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:49:02 +08:00
031ee6f155
enhance: support httpv1/v2 throttle and add it for httpV2( #35350 ) ( #35470 )
...
related: #35350
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-20 16:16:55 +08:00
3491608256
fix: Match int8_t and int16_t in Array::get_data ( #35579 )
...
Related to #35578
Previously int16/int8 bitmap index may read int32 array as int16, which
may cause build index with half of the data(if array is full) and half
zeros. This causes BITMAP index lost information.
This PR matches int8_t & int16_t while `get_data` when building index.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-20 16:10:56 +08:00
249dc4d9eb
test: Add tests for upsert with auto id ( #35556 )
...
Related issue: #34668
---------
Signed-off-by: yanliang567 <yanliang.qiao@zilliz.com>
2024-08-20 14:20:56 +08:00
e09dc3be58
enhance: Mark query node as read only after suspend ( #35492 )
...
issue: #34985 #35493
after querynode has been suspended, it's not allow to load
segment/channel on it, which means the node is read only. to be
compatible with resource group design, after query node has been
suspend, we remove it from it's original resource group, make it a read
only query node in replica. then two things will happens:
1. it's original resource group will be lacking of query nodes, query
coord will assign new node to it.
2. querycoord will try to move out all segments/channels after querynode
has been suspended
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 14:02:54 +08:00
3948bd4e79
fix: Added check for validating varchar,array max length ( #35499 )
...
issue : https://github.com/milvus-io/milvus/issues/34150
This is for numpy,parquet,json readers.
---------
Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
2024-08-20 11:42:55 +08:00
22ced010cd
enhance: make configure load param feature be compatible with old sdk ( #35520 )
...
issue: #31570 #35521
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-20 10:30:55 +08:00
6e29d713b3
enhance: [skip e2e]dockerfile for milvus-sdk-go test ( #35426 )
...
https://github.com/milvus-io/milvus/issues/35567
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-20 09:48:55 +08:00
4d69898cb2
enhance: support single pchannel level transaction ( #35289 )
...
issue: #33285
- support transaction on single wal.
- last confirmed message id can still be used when enable transaction.
- add fence operation for segment allocation interceptor.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-19 21:22:56 +08:00
853d2f33e0
enhance: Trim space during parsing string list from string ( #35546 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-19 20:48:55 +08:00
227ecd3901
enhance: Remove the check for channel cp lag when generating compaction plan ( #35383 )
...
issue: https://github.com/milvus-io/milvus/issues/35382
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-19 19:40:55 +08:00
795339ecbc
enhance: using timer instead of ticker in client/index.go ( #35552 )
...
https://github.com/milvus-io/milvus/issues/35551
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-08-19 18:56:54 +08:00
b4743b4ca8
enhance: [GoSDK] Sync latest milvus proto and mockery ( #35511 )
...
Related to #35443
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-19 16:22:55 +08:00
b96db798b9
enhance: using timer instead of tickers for need to be executed successfully once ( #35534 )
...
https://github.com/milvus-io/milvus/issues/35533
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-08-19 16:16:55 +08:00
337e065902
fix: querynode hang when failing to allocate disk space for mmap( #35184 ) ( #35187 )
...
related: #35184
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-08-19 15:30:55 +08:00
14ef88f423
fix: Fix proxy panic after Import related api failed ( #35540 )
...
issue: #35539
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-19 15:22:55 +08:00
967f38672a
enhance: Add integration tests for l0 ( #35429 )
...
See also: #34796
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-19 10:56:54 +08:00
719c5f793a
[automated] Update cpu Builder image changes ( #35508 )
...
Update cpu Builder image changes
See changes:
1275005fc1sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-16 20:34:58 +08:00
635e9ddb0e
test: fix go sdk cases parallel context deadline ( #35519 )
...
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2024-08-16 20:32:53 +08:00
2736a8b88c
enhance: Update Makefile to generate mockery ( #35517 )
...
Some mockery cmd is out-of-date and fail to work. This PR update these
commands to match current pkg.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-16 18:44:53 +08:00
ead37326f0
enhance: Add backup/restore privilege to admin group ( #35515 )
...
issue: #35443
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-16 18:42:54 +08:00
80dbe87759
enhance: support null value in index ( #35238 )
...
#31728
---------
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-08-16 15:30:54 +08:00
f87af9bc54
enhance: Exclude L0 segment from readable snapshot ( #35507 )
...
L0 segments now do not contain insert data and may cause confusion for
query hook optimizer if counted as sealed segment number.
This PR add segment level flag in segment entry and exclude L0 segments
while get readable segment snaphsot
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-16 15:28:53 +08:00
5e9097c2dd
fix: err degenerated to a new variable ( #35413 )
...
https://github.com/milvus-io/milvus/issues/35412
Signed-off-by: fengjun2016 <jornfeng@gmail.com>
2024-08-16 14:24:54 +08:00
a570567644
enhance: Enable ReadOnly/ReadWrite/Admin Privilege Group to simplify RBAC grant progress ( #35472 )
...
issue: #35471
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-16 14:18:54 +08:00
f4a91e135b
enhance: Allow empty sparse row ( #34700 )
...
issue: #29419
* If a sparse vector with 0 non-zero value is inserted, no ANN search on
this sparse vector field will return it as a result. User may retrieve
this row via scalar query or ANN search on another vector field though.
* If the user uses an empty sparse vector as the query vector for a ANN
search, no neighbor will be returned.
Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
2024-08-16 14:14:54 +08:00
1d49358f82
enhance: Add BackupRBAC/RestoreRBAC API to enable rbac backup ( #35444 )
...
issue: #35443
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-16 10:10:53 +08:00
SimFG
congqixia
Gao
Zhen Ye
wei liu
zhagnlu
ThreadDao
presburger
sre-ci-robot
yellow-shine
OxalisCu
smellthemoon
yanliang567
Ted Xu
aoiasd
Chun Han
nish112022
yihao.dai
CharlesFeng
XuanYang-cn
sre-ci-robot
Buqian Zheng