Commit Graph

376 Commits (hotfix-2.3.3)

Author SHA1 Message Date
SimFG 3787da8e7d
enhance: [hotfix-2.3.3] Add concurrency for datacoord segment GC (#29562)
issue: https://github.com/milvus-io/milvus/issues/29553
pr: #29561
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-01-03 14:22:49 +08:00
aoiasd 2d517039b5
fix: [Cherry-Pick] msgstream memory leak caused by config event (#29418)
relate: https://github.com/milvus-io/milvus/issues/28620
https://github.com/milvus-io/milvus/issues/28367
pr: https://github.com/milvus-io/milvus/pull/29266
https://github.com/milvus-io/milvus/pull/29377
https://github.com/milvus-io/milvus/pull/29403

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-25 11:18:44 +08:00
SimFG 38f50f6e4a
enhance: [hotfix-2.3.3] Support to get the param value in the runtime (#29396)
pr: https://github.com/milvus-io/milvus/pull/29297
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 19:12:43 +08:00
yah01 1ddb12ba92
enhance: enhance the accuracy of memory usage (#28554) (#28559) (#29381)
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then rss - shared would be the accurate memory usage

resolve https://github.com/milvus-io/milvus/issues/28553
pr: #28554

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 15:00:43 +08:00
Jiquan Long 747934eb2b
enhance: make consistency level used in delete configurable (#29280) (#29285)
pr: #29280 
issue: #29279

---------

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-12-20 14:14:05 +08:00
congqixia b9757f8433
enhance: [hotfix] Add channel checkpoint ts lag metric (#28767) (#29324)
Cherry-pick from master
pr: #28767
See also #28765

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-20 11:04:45 +08:00
Jiquan Long 90847fbcfd
fix: disable reset kafka connection timeout (#28681) (#29286)
pr: https://github.com/milvus-io/milvus/pull/28642 issue
https://github.com/milvus-io/milvus/issues/28588

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Co-authored-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-12-19 10:11:30 +08:00
yah01 8e19049b5c
enhance: remove merger for load segments (#29062) (#29078)
remove merger as now QueryNode could load segments concurrently
fix https://github.com/milvus-io/milvus/issues/29063
pr: #29062

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-08 22:31:58 +08:00
congqixia 7774d3545b
Bump Milvus & milvus-proto version to v2.3.3 (#28268)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-08 19:08:27 +08:00
wei liu 918333817e
Disable auto balance when old node exists (#28191) (#28224)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 07:10:17 +08:00
yah01 d10a82dba4
Fix getting incorrect CPU num (#28178)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-07 11:52:22 +08:00
Xiaofan bbcaf7a703
Fix coordinator fast restart (#28205)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-11-07 10:04:16 +08:00
PowderLi 43db873770
add a custom http header: Accept-Type-Allow-Int64 (#28125)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-06 16:02:18 +08:00
yah01 218a988941
Increase the ChunkManager request timeout (#28015) (#28154)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-04 10:26:17 +08:00
wei liu e2e2487f40
avoid sleep to ctx done during retry (#27992) (#28092)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-02 09:02:15 +08:00
yihao.dai 0f3e83a764
Construct plan directly when search with vector output (#27963)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-27 10:32:13 +08:00
congqixia 8ab01e5a14
Set ElementType for empty array FieldData (#27924)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-25 20:08:10 +08:00
Gao c8415453a1
Support prepare config for index (#27920)
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-10-25 19:54:12 +08:00
aoiasd 53246b1b38
Set accesslog default to close and use stdout (#27891)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-10-25 10:30:10 +08:00
wei liu 40723a292e
reduce compact parallel task num (#27899)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-25 09:40:12 +08:00
jaime 4640928280
Fix initialization and backward compatibility issue for GRPC compression (#27894)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-24 22:14:14 +08:00
congqixia ee184a709e
Bump Milvus Version & milvus-proto/go-api to v2.3.2 (#27893)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-24 21:02:11 +08:00
jaime 6749957e71
Refine RPC call in unwatch drop channel (#27864)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-24 17:46:15 +08:00
congqixia 93a877f55e
Make qcv2 target&leader observer execute in parallel (#27844)
- Add `taskDispatcher` to submit and run task async safely
- Change `LeaderObeserver` and `TargetObserver` schedule and manual check action to submitting task into dispatcher
- Fix logic problem in collection observer when manual check return false

See also #27494

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-24 10:14:11 +08:00
Xiaofan 2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-10-23 21:44:09 +08:00
zhagnlu 6060dd7ea8
Add chunk manager request timeout (#27692)
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-10-23 20:08:08 +08:00
congqixia 4a6790b63a
Make KeyLock generic (#27854)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-23 17:40:11 +08:00
SimFG 086220dd15
Add the mock factory of msg stream (#27834)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 20:14:08 +08:00
SimFG 9b0ecbdca7
Support to replicate the mq message (#27240)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
zhenshan.cao 020ad9a6bc
Rectify wrong exception messages associated with Array datatype (#27769)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-10-19 17:24:07 +08:00
smellthemoon 4b0ec156b3
Set channel work pool size in datanode (#27728)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-10-19 08:28:08 +08:00
yah01 9467de79c3
Use pool to block exceeded tasks (#27767)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-19 03:30:06 +08:00
zhenshan.cao 2b13078b14
Refine insert when send message (#27491)
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
Co-authored-by: Cai Zhang <cai.zhang@zilliz.com>
2023-10-18 21:12:08 +08:00
yihao.dai 49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-18 15:32:11 +08:00
congqixia 2f201c25e2
Remove deprecated io/ioutil usage (#27747)
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-17 20:32:09 +08:00
Futao Wei 599012a340
Implement SearchByPks path for Search (#25882)
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
2023-10-16 15:34:08 +08:00
wayblink e3f2122618
Expose metrics of stanby coordinators (#27698)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-10-16 15:04:09 +08:00
wei liu 43f2452f4b
speed up shard leader cache update frequency (#27640)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-16 10:52:10 +08:00
dependabot[bot] 98e5a7391c
Bump golang.org/x/net from 0.10.0 to 0.17.0 in /pkg (#27657)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.10.0 to 0.17.0.
- [Commits](https://github.com/golang/net/compare/v0.10.0...v0.17.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-14 02:53:35 +08:00
yah01 15746b5b5f
Refine KV errors (#27588)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-13 17:15:34 +08:00
yah01 be980fbc38
Refine state check (#27541)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
foxspy 2b46bd1f08
fix diskann searchCache param compatibility (#27557)
Signed-off-by: xianliang <xianliang.li@zilliz.com>
2023-10-11 16:57:34 +08:00
yah01 2df9908c6a
Map old error code to new error (#27616)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 12:43:33 +08:00
smellthemoon a9982aa7c6
Remove topic empty restriction when preCreated topic (#27351)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-10-10 18:53:32 +08:00
yah01 3759857bc5
Refine Proxy errors (#27499)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-09 10:09:33 +08:00
XuanYang-cn 56c94cdfa7
Add channel manager in DataNode (#27308)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-10-08 21:37:33 +08:00
yah01 41495ed266
Improve the error message for getting all indexes of collection (#27389)
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-08 21:23:32 +08:00
PowderLi 8d3069b1db
update openssl to 3.1.2 (#27399)
deal with root path's normalization

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-10-08 19:17:31 +08:00
congqixia 5d558623fe
Add revive sub-lints and fix existing problems (#27495)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-07 20:53:38 +08:00
congqixia 80eb5434d3
Fix var-naming caused by old PRs (#27501)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-07 14:43:32 +08:00