8b706122a8
enhance: [2.4] support to drop the role which is related the privilege list ( #35863 )
...
- issue: #35545
- pr: #35727
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-31 21:57:02 +08:00
5698d1859e
[automated] Bump milvus version to v2.4.10 ( #35869 )
...
Bump milvus version to v2.4.10
Signed-off-by: sre-ci-robot sre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-30 17:13:01 +08:00
da026b1e28
enhance: Add depguard rules to ban deprecated proto lib ( #35140 ) ( #35818 )
...
See also #34394 #34252
pr: #35140
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: congqixia <congqi.xia@zilliz.com>
2024-08-30 14:13:01 +08:00
8cd75d627c
enhance: [ci] organize values file in the form of milvus deployment o… ( #35839 )
...
https://github.com/milvus-io/milvus/issues/35842
https://github.com/milvus-io/milvus/pull/35832
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-30 13:07:15 +08:00
41582a90ad
enhance: more friendly to docker cache ( #35792 )
...
https://github.com/milvus-io/milvus/pull/35789
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-30 13:05:34 +08:00
1916aec67a
enhance: [skip e2e] add make run-test-cpp gtest ( #35829 ) ( #35837 )
...
pr: https://github.com/milvus-io/milvus/pull/35829
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-29 21:15:02 +08:00
cfc99e63b1
fix: [2.4] Make sure querycoord observers started once ( #35811 ) ( #35817 )
...
Cherry-pick from master
pr: #35811
Related to #35809
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 19:15:01 +08:00
8d3685fadf
enhance: [2.4] Print log only when rate limit updates ( #35806 ) ( #35816 )
...
Cherry-pick from master
pr: #35806
The debug log for "RateLimiter register for rateType" is too frequent
and in e2e cases, the may print 18M times in one run.
This PR make the log be printed only when rate limit is updated.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 19:09:01 +08:00
a1d39326b8
fix: Fix arrow go client ( #35819 ) ( #35820 )
...
issue: https://github.com/milvus-io/milvus/issues/35662
pr: https://github.com/milvus-io/milvus/pull/35819
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-29 17:13:00 +08:00
fedf3a7fe1
enhance: Bump milvus & proto version to v2.4.10 ( #35798 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 14:55:01 +08:00
00202b0d7e
enhance: add skip 3rdparty env var ( #35748 )
...
issue: #35611
pr: #35736
- call `conan install` is too slow for developer, add option to skip it.
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-29 14:23:01 +08:00
5da64f6d5a
feat: [2.4] support range search pagination retains order ( #35739 )
...
issue: https://github.com/milvus-io/milvus/issues/35464
pr: https://github.com/milvus-io/milvus/pull/35738
2024-08-29 14:07:01 +08:00
50ec3dd4db
enhance: avoid the coexistence of the old coordinator and the new node/proxy ( #35760 )
...
issue: #35719
pr: #35720
In standalone mode, block the start process until the new coordinator is
active to avoid the coexistence of the old coordinator and the new
node/proxy
1. In the start/restart process, the new coordinator will become active
immediately and will not be blocked
2. In the rolling upgrade process, the new coordinator will not be
active until the old coordinator is down, and it will be blocked
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-29 12:33:01 +08:00
8928c9d570
enhance: [2.4] Change frequent balancer debug log to rated one ( #35749 ) ( #35796 )
...
Cherry-pick from master
pr: #35749
"skip balance" log is too frequent in debug level. This PR changes it
into rated on.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-29 12:31:00 +08:00
2fc111eb10
enhance: set database properties to restrict read access ( #35754 )
...
issue: #35744
pr: #35745
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-29 11:41:01 +08:00
46a8b07bc4
enhance: show pod status and delete pvc during pipeline dear down stage ( #35785 )
...
https://github.com/milvus-io/milvus/pull/35783
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-29 09:59:00 +08:00
2794ebdc45
enhance: [2.4] fix new nightly pipeline ( #35750 )
...
https://github.com/milvus-io/milvus/pull/35747
https://github.com/milvus-io/milvus/issues/35704
---------
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-28 21:43:00 +08:00
21454163bb
fix: [2.4] Check response size before add to counter ( #35779 )
...
Cherry-pick from master
pr: #35778
Related to #35767
prometheus counter cannot add negative value
when response is not written(say timeout/network broken) panicking may
happen if not check
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-28 20:01:01 +08:00
498c9470de
test: update test cases ( #35718 )
...
pr: #35640
Signed-off-by: nico <cheng.yuan@zilliz.com>
2024-08-28 19:23:01 +08:00
e25d1ef63c
fix: wrong construction in evalctx ( #35773 )
...
issue: #35771
pr: #35772
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-28 18:37:01 +08:00
12575885d3
enhance: [2.4] Add skip load validation for create collection task ( #35761 )
...
Cherry-pick from master
pr: #35737
Related to #35415
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-28 18:25:00 +08:00
e331288b78
enhance: [skip e2e] Fix test script always skip pip install ( #35775 )
...
https://github.com/milvus-io/milvus/pull/35766
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-28 17:05:06 +08:00
93b118946c
feat: [2.4] Encode traceID and spanID as hex string & upgrade knowhere version ( #35568 )
...
Cherry-pick and upgrade knowhere version
issue: https://github.com/zilliztech/knowhere/issues/714
pr: #34807
knowhere release notes:
https://github.com/zilliztech/knowhere/releases/tag/v2.3.8
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2024-08-28 17:03:00 +08:00
c75f556769
fix: inconsistent meta view causes rate limit invalid ( #35664 )
...
issue: #35663
pr: #35665
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 11:21:04 +08:00
8e6ec58652
fix: Fix rate wasn't limited to the expected value ( #35699 ) ( #35700 )
...
Each time the rate is reset, the token bucket is fully refilled, causing
the rate wasn't limited to the expected value. This PR addresses the
issue by preventing the token reset.
issue: https://github.com/milvus-io/milvus/issues/35675 ,
https://github.com/milvus-io/milvus/issues/35702
pr: https://github.com/milvus-io/milvus/pull/35699
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-28 10:29:00 +08:00
eecc598786
fix: mistaken deletions may occur during GC channel checkpoints ( #35708 )
...
issue: #35706
pr: #35707
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-28 10:11:05 +08:00
a4533f1b8a
enhance: optimize milvus core building ( #35660 )
...
issue: #35549,#35611,#35633
pr: #35610
- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-27 18:55:00 +08:00
be17cf6ff1
fix: Fix panic due to empty candidate import segments ( #35673 ) ( #35674 )
...
issue: https://github.com/milvus-io/milvus/issues/35662
pr: https://github.com/milvus-io/milvus/pull/35673
---------
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-08-27 17:34:59 +08:00
69e8d5ef46
fix: munmap deallocate too much memory ( #35726 )
...
issue: #35693
pr: #35725
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-27 17:22:59 +08:00
fc324b4254
feat: [2.4] add the rbac msg and send them to the replicate channel ( #35562 )
...
- issue: #35391
- pr: #35392
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-27 14:45:00 +08:00
03627150d3
enhance new jenkinsfile for nightly and e2e ( #35715 )
...
https://github.com/milvus-io/milvus/pull/35672
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-27 14:23:05 +08:00
ab261d0f8b
feat: [2.4] Support field partial load collection ( #35416 ) ( #35696 )
...
Cherry-pick from master
pr: #35416
Related to #35415
---------
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-27 14:07:00 +08:00
33fbc1fb96
enhance: turnoff USE_ASAN in ci ( #35723 )
...
Signed-off-by: Yellow Shine <sammy.huang@zilliz.com>
2024-08-27 14:01:11 +08:00
f2caecc577
fix: [cp]Fill nil schema so that Milvus can watch channel for those upgraded from 2.2 to 2.4 ( #35695 )
...
See also: #35701
pr: #35694
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-27 10:38:59 +08:00
ff68eb53d7
fix: [skip e2e]unstable l0 it ( #35613 )
...
See also: #35617
pr: #35612
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-26 18:53:04 +08:00
7062e6fcbf
enhance: enable asan for milvus ( #35687 )
...
issue: #35626
pr: #35627
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-26 09:54:58 +08:00
7269d5eda2
enhance: [2.4] reduce the log level of frequent log ( #35653 )
...
pr: #35651
Signed-off-by: xiaofanluan <xiaofan.luan@zilliz.com>
2024-08-25 17:48:57 +08:00
5c1a380d9d
fix: the incorrect number of partitions in rootcoord meta ( #35601 )
...
pr: #35600
Signed-off-by: jaime <yun.zhang@zilliz.com>
2024-08-25 16:30:59 +08:00
4b5bd07971
enhance: [2.4] enable MV on binary vectors ( #35669 )
...
pr: #35668
Signed-off-by: Patrick Weizhi Xu <weizhi.xu@zilliz.com>
(cherry picked from commit 5404228fca58d4ad4a788ec6d58d2693019132b6)
2024-08-25 16:27:08 +08:00
35d2f9b210
fix: Fix index memory estimation ( #35225 ) ( #35670 )
...
issue: https://github.com/milvus-io/milvus/issues/35229
pr: #32525
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Bingyi Sun <sunbingyi1992@gmail.com>
2024-08-24 10:28:57 +08:00
81c4cae2c1
enhance: Enable to write multiple segments in mix compactor ( #35648 )
...
Prevent segments to be written larger than maxSize * expansionRate
See also: #35584
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-24 10:00:58 +08:00
eb18e114a1
fix: Clear channelcp meta and metrics ASAP ( #35659 )
...
See also: #35588
pr: #35658
---------
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2024-08-24 09:28:58 +08:00
79d093257c
fix: Added check for validating varchar,array max length ( #35499 ) ( #35581 )
...
issue : https://github.com/milvus-io/milvus/issues/34150
pr: https://github.com/milvus-io/milvus/pull/35499
This is for numpy,parquet,json readers.
cherry-picked from master branch
Signed-off-by: Nischay Yadav <nischay.yadav@ibm.com>
2024-08-23 17:14:58 +08:00
f3fb37655b
fix: [2.4] Use correct release func when datanode unwatch a channel ( #35657 )
...
Cherry-pick from master
pr: #35655
See also #35654
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-23 14:09:03 +08:00
1687d64c46
enhance: [2.4] add hit segment num metrics for queryHook ( #35619 )
...
issue: #35576
pr: #35577
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2024-08-23 12:49:02 +08:00
5b5119a51f
feat: [2.4] provide more general configuration to control mmap behavior ( #35609 )
...
- issue: #35273
- pr: #35359
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-23 12:35:02 +08:00
90a5bafe17
fix:fix bitmap log error ( #35644 )
...
pr: #32902
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2024-08-23 08:56:58 +08:00
216f33c529
[automated] Update Pytest image changes ( #35639 )
...
Update Pytest image changes
See changes:
d7d4525c2esre-ci-robot@users.noreply.github.com
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-08-22 19:18:58 +08:00
421a00bfe8
enhance: Refresh proxy cache after restore rbac meta ( #35636 )
...
issue: #35443
pr: #35635
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-08-22 19:09:01 +08:00
9c67965d6c
fix: [2.4] Re-read value after `once` initialization ( #35643 )
...
Cherry-pick from master
pr: #35642
Related to #35641
See also #35271
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-08-22 16:30:57 +08:00
SimFG
sre-ci-robot
wei liu
yellow-shine
zhenshan.cao
congqixia
yihao.dai
Zhen Ye
Patrick Weizhi Xu
jaime
nico
foxspy
XuanYang-cn
Xiaofan
nish112022
Gao
zhagnlu