7721a28584
fix: Fix unstable TestDispatchToVchannels ut ( #35163 )
...
issue: https://github.com/milvus-io/milvus/issues/35150
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-01 14:56:13 +08:00
5bbb1c201c
enhance:support l2 single compaction ( #34935 )
...
#34928
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-08-01 14:36:13 +08:00
f8c1b138a8
fix:fix get array error for int type ( #35154 )
...
#35055
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-01 14:30:12 +08:00
b4d0f4df0a
enhance: update param documents ( #35152 )
...
See: #35161
---------
Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2024-08-01 12:38:13 +08:00
aa247f192d
enhance: remove unused code for StorageV2 ( #35132 )
...
issue: https://github.com/milvus-io/milvus/issues/34168
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-01 12:08:13 +08:00
9412002d7d
fix: Fix data race for clustering buffer writer ( #35145 )
...
issue: #34495
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-01 11:20:13 +08:00
089fb2b5fb
enhance: Skip updating the level that are already at the specified level ( #35094 )
...
issue: #34495
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-08-01 11:18:12 +08:00
f229f244d2
enhance: add chunk basic impl ( #34634 )
...
https://github.com/milvus-io/milvus/issues/35112
This pr would not affect milvus functionality by now.
It implments a Chunk memory layout that looks like
```
VariableColumn
|offset|offset|offset|
|data|data|data|
```
We maybe move offsets to the beginning and add null bitmaps later but
not in this PR.
And mmap test will also be added in another PR.
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-08-01 10:29:51 +08:00
e9d61daa3f
enhance: Reduce delegator memory overloaded factor to 0.1 ( #35092 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-01 10:21:50 +08:00
dfda9f0478
enhance: Add depguard rules to ban deprecated proto lib ( #35140 )
...
See also #34394 #34252
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-01 10:01:49 +08:00
c15d074b21
[automated] Update Knowhere Commit ( #35157 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-01 01:17:49 +08:00
6695c6d0a3
enhance: add channel num for queryHook optimization ( #35104 )
...
At most cases, data in each channel is almost evenly distributed, we
could utilize the channel num info to optimize searh param in queryHook
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-07-31 18:59:49 +08:00
03912a8788
enhance: Avoid balance stuck after segment list become stable ( #34728 )
...
issue: #34715
if collection's segment list doesn't changes anymore, then the next
target will be empty at most time, and balance segment will check
whether segment exist in both current and next target, so the balance
cloud be blocked due to next target is empty.
This PR permit segment to be moved if next target is empty, to avoid
balance stuck.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-31 18:09:48 +08:00
a9744d62b1
enhance: improve error msg when expr invalid during isolation ( #35127 )
...
issue: #34336
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit b1d7b24972e9414ab9488c7611aca9975e674cd4)
2024-07-31 17:23:51 +08:00
6106a48acb
fix: upsert result use the previous pk ( #34672 )
...
#34668
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-07-31 15:25:51 +08:00
acaa78db58
fix: disable sparse and binary in MV ( #35126 )
...
issue: #35120
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 0d340ebe7412cd85963825bc8abd3c874e339679)
2024-07-31 15:02:01 +08:00
f7f9a729c9
enhance: Pre-allocate space for reduce data structure ( #35118 )
...
Grow slice & map.growWork may cause a lot when segment number is large
for big K query. This PR pre-allocate space for reduce methods to avoid
this cost.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-31 10:35:49 +08:00
48b14c56f6
fix: add prometheus dependency for milvus_monitor ( #35078 )
...
add prometheus dependency for monitor module. Or else Some compilers may
report a compilation failure.
issue: #35077
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
2024-07-31 10:15:48 +08:00
f919b6e1b8
enhance: refine error handle in clustering compaction task ( #34956 )
...
related: #30633 #34792
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-30 20:37:57 +08:00
ce3f836876
fix: compaction task not be cleaned correctly ( #34765 )
...
1.fix compaction task not be cleaned correctly
2.add a new parameter to control compaction gc loop interval
3.remove some useless configs of clustering compaction
bug: #34764
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-07-30 20:21:56 +08:00
39d25938c5
enhance: Avoid panic due to nil schema ( #35063 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/discussions/25620
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-30 20:18:32 +08:00
dd0c26cf58
enhance: redefine variable column block size ( #35040 )
...
#35013
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-30 19:23:50 +08:00
273e9203f4
feat: [skip e2e] effective way to use cache ( #34774 ) ( #35109 )
...
See also #34483
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-07-30 18:41:51 +08:00
b787bb4783
enhance: [skip e2e] Skip Amazon&Rockylinux instead of commenting job ( #35108 )
...
Related to #35084
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 16:23:49 +08:00
de8a266d8a
enhance: Enable linux code checker ( #35084 )
...
See also #34483
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 15:53:51 +08:00
4bb969ef8f
enhance: Skip manual stopped component during health check ( #34953 )
...
after manual stop component by management restful api, `healthz` may
return unhealthy state. k8s may restart the pod to save the unhealthy
sate, and the manual stop operation will got unexpected result.
to solve this, we make `healthz` API skip the manual stopped component.
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-30 14:37:51 +08:00
cabb200498
enhance: [GoSDK] Expose indexing progress in DescribeIndex result ( #35066 )
...
See also #31293
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-30 10:33:49 +08:00
a8a4779749
fix: fix parse plan proto failed for search type ( #34944 )
...
#25848
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 21:19:49 +08:00
d36cfe71e5
enhance: Refine protobuf dependency installation in Makefile ( #35072 )
...
Related to #34394
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-29 21:15:48 +08:00
86322e0468
fix: fix query count(*) concurrently ( #35007 )
...
#34778
#34849
fix two problems:
1. count(*) incorrect, if growing insert duplicated (pk, timestamp)
pairs that pk and timestamp all same, need to keep just one pair.
2. count(*) may core dump, if get_real_count interface get snapshot and
do mvcc at not consistency status, mainly happens under concurrency.
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 19:53:50 +08:00
972752258a
enhance: Support otlp http exporter ( #35053 )
...
See also #35052
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-29 17:43:49 +08:00
a495754034
enhance: the datacoord gc should fast quitable ( #35050 )
...
issue: #35049
Signed-off-by: chyezh <chyezh@outlook.com>
2024-07-29 17:03:48 +08:00
e62f6b2f4e
enhance: [skip e2e] enlarge timeout of UT for go up to 90 mins ( #35035 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-07-29 15:51:49 +08:00
3c7f73137e
fix: disallow expr when partition key isolation is enabled ( #35031 )
...
issue: #34336
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 2889481ce9a14230e9c7f1c8f9c3c1decde77e03)
2024-07-29 14:21:50 +08:00
0e41f104c5
enhance: use the collection id to group msg pack in the msg dispatcher ( #34871 )
...
- issue: #34870
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-07-29 11:45:49 +08:00
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 ( #34394 )
...
issue: #34252
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
e2e38e98df
fix: nil part stats without l2 compaction( #34923 ) ( #34992 )
...
related: #34923
Signed-off-by: MrPresent-Han <chun.han@gmail.com>
Co-authored-by: MrPresent-Han <chun.han@gmail.com>
2024-07-29 11:07:48 +08:00
9a07790215
enhance: Add ut for l0CompactionTask processExecuting ( #34800 )
...
See also: #34796
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-07-29 10:53:48 +08:00
4a3d98d88c
enhance: Fast close msgdispatcher target ( #34803 )
...
/kind improvement
issue: https://github.com/milvus-io/milvus/issues/34075
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-07-29 00:25:49 +08:00
f77f5364b2
fix: disable use_index when some array expr ( #34894 )
...
#34797
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-29 00:17:46 +08:00
5584ae2e14
enhance: add metrics for restful apis ( #34969 )
...
issue: #34968
Signed-off-by: PowderLi <min.li@zilliz.com>
2024-07-29 00:13:47 +08:00
497afcb897
enhance: Refine code for GetRecoveryInfo ( #34973 )
...
issue: #34495
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-29 00:01:46 +08:00
2372452fac
enhance: Optimized the GC logic to ensure that memory is released in time ( #34949 )
...
issue: #34703
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2024-07-28 23:53:47 +08:00
08fa51d4f4
fix: memory leak while parsing query plan ( #34931 )
...
issue: #34930
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-07-28 21:50:20 +08:00
9463eeef2b
fix: Avoided dereferencing NULL pointer ( #34836 )
...
issue : https://github.com/milvus-io/milvus/issues/34835
Signed-off-by: Ald392 <imagesai32@gmail.com>
2024-07-27 17:27:52 +08:00
804ec24c02
fix:fix retrieve raw data from bitmap array index ( #34848 )
...
#34795
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-07-27 01:53:47 +08:00
e93687cdd3
[automated] Update Knowhere Commit ( #35039 )
...
Update Knowhere Commit
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-07-27 00:59:46 +08:00
b86d795d6d
enhance: [GoSDK]Support specifying field schema to be cluster key ( #35027 )
...
See also #32939 milvus-io/milvus-sdk-go#794
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 20:01:45 +08:00
783f9d9c33
fix: Unify hook singleton implementation in proxy ( #34887 )
...
Related to #34885
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-07-26 18:07:53 +08:00
6e9fbd1630
enhance: Solve channel unbalance on datanode ( #34984 )
...
issue: #33583
the old policy permit datanode has at most 2 more channels than other
datanode. so if milvus has 2 datanode and 2 channels, both 2 channels
will be assign to 1 datanode, left another datanode empty.
This PR refine the balance policy to solve channel unbalance on datanode
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-26 17:35:47 +08:00
yihao.dai
wayblink
zhagnlu
Ted Xu
zhenshan.cao
cai.zhang
Bingyi Sun
wei liu
congqixia
sre-ci-robot
Gao
Patrick Weizhi Xu
smellthemoon
foxspy
yellow-shine
chyezh
SimFG
Chun Han
XuanYang-cn
PowderLi
jaime
Aldrin