af54c3ba85
fix: Make datacoord client retry on index api ( #30656 )
...
pr: #30654
This PR add retry on all interface which belong to indexcoord in milvus
2.2 and. move to data coord in milvus 2.3, to prevent meet unimplemented
error during rolling upgrade from milvus 2.2 to 2.3.
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-03-04 11:37:09 +08:00
2f76303989
enhance: Support varchar autoid for bulkinsertV1 ( #30896 ) ( #30913 )
...
This PR is a supplement to PR
https://github.com/milvus-io/milvus/pull/30377 .
pr: https://github.com/milvus-io/milvus/pull/30896
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-02-29 12:11:00 +08:00
1c8d9fa686
fix: wrong context passing into NewClient, error handling lost in session_util ( #30818 )
...
issue: #30799
pr: #30817
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-02-28 10:41:00 +08:00
692dcebac6
enhance: support varchar autoid when bulkinsert（#30377） ( #30448 )
...
related pr: #30377
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2024-02-02 16:11:08 +08:00
21c944beaa
enhance: add basic information of milvus into metrics ( #29666 )
...
add basic build information and runtime component dependency into
metrics.
issue: #29664
pr: #29665
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2024-01-29 15:49:04 +08:00
14aa20b7f7
enhance: [cherry-pick] fix otel config param type & leak ( #30068 )
...
cherry pick from master
pr: #29810 #30055
`SampleFraction` shall be float and all `C.CString` shall be freed
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-18 12:43:05 +08:00
df9b3376dc
fix: Use determined order to lock in BlockAll to avoid deadlock ( #29972 )
...
issue: #29104
pr: #29246
Signed-off-by: chyezh <ye.zhen@zilliz.com>
2024-01-15 14:32:51 +08:00
e1446da83c
feat: [Cherry-pick] Implement DescribeAlias and ListAliases interfaces ( #29896 )
...
#22882
pr: #29641
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2024-01-12 16:30:51 +08:00
ba1d0556ba
fix: Import data from parquet file in streaming way ( #29528 )
...
issue: #29292
master pr: #29514
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-29 11:56:45 +08:00
85f17ada74
fix: [cherry-pick] grpc client check session skipped due to role not match ( #29358 )
...
Cherry pick from master
pr: #29356
Related to #28815
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-21 10:12:51 +08:00
7ef49adbd2
enhance: Use mockery to replace manual mock code ( #29074 ) ( #29182 )
...
issue: #29043
pr: #29074
This PR remove mannul mock code for proxy and data coord
---------
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-14 16:44:43 +08:00
7946c2a8dc
fix: [cherry-pick] bulkinsert binlog didn't consider ts order when processing delta data ( #29176 )
...
issue: #29162
pr: #29163
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-12-14 14:38:38 +08:00
bd46dc2d1b
enhance: [Pick]Support implicit type conversion for parquet ( #29119 )
...
issue: https://github.com/milvus-io/milvus/issues/29019
master pr: #29046
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-12 16:14:43 +08:00
0b532b42ea
enhance: [cherry-pick] Add client connect wrapper to keep connection alive ( #29061 )
...
Cherry-pick from master
pr: #29058
See also #29057
Add wrapper to maintain client&connection
When reset operation is needed, close method shall wait until all
on-going request return
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-08 21:12:35 +08:00
bba0e159ba
feat: [pick] Add bypassing logic for ttMsg in flowgraph of DataNode( #28756 ) ( #29036 )
...
In order to minimize the CPU usage of the coroutine and avoid frequent
execution of time-consuming operations in the flowgraph when the message
stream consists solely of "ttMsg," it is recommended to implement a
mechanism for quickly bypassing the subsequent flowgraph node processing
logic.
If "ttMsg" is continuously received for a certain period of time
(coldTime), the flowgraph enters skipMode. Once in skipMode, every
skipNum "ttMsg" messages are merged into one for processing. If a
non-"ttMsg" message is received while in skipMode, the flowgraph exits
skipMode.
pr: #28756
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: wayblink <anyang.wang@zilliz.com>
2023-12-08 12:10:39 +08:00
c21a1d59e4
fix: createCollection failed occasionally ( #28592 ) ( #28989 )
...
related: #28592
pr:#https://github.com/milvus-io/milvus/pull/28712
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-08 11:34:39 +08:00
7f59826089
enhance: [Pick]Make import-related error message clearer ( #28979 )
...
issue: #28976
master pr: #28978
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-08 10:16:35 +08:00
5f4ac437b2
enhance: [Cherry-pick] Moving etcd client into session ( #27069 ) ( #28996 )
...
relate: #26694
pr: https://github.com/milvus-io/milvus/pull/27069
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
Co-authored-by: Filip Haltmayer <81822489+filip-halt@users.noreply.github.com>
2023-12-07 16:22:34 +08:00
00461145b3
enhance: [Pick]Support importing data with parquet file ( #28820 )
...
issue: https://github.com/milvus-io/milvus/issues/28272
master pr: #28608
Numpy does not support array type import.
Array type data is imported through parquet.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-29 22:16:29 +08:00
fda452ea4d
enhance: [Pick] Revert import support csv format ( #28766 )
...
issue: #28778
master pr: #28760
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-28 18:26:28 +08:00
1ed92da414
enhance:[cherry-pick] Merge flowgraph goroutines into 1 ( #28728 )
...
see also: #28654 #28233
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-11-27 21:04:27 +08:00
ef115dd7df
enhance: Support Array DataType for bulk_insert ( #28578 )
...
issue: https://github.com/milvus-io/milvus/issues/28272
master pr: #28341
Support array DataType for bulk_insert with json, binlog files.
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-11-27 16:56:27 +08:00
288844e3cf
enhance: Reduce the goroutine in flowgraph to 2 ( #28233 ) ( #28545 )
...
each node in flow graph alloc a goroutine, but it is actually executed
sequentially and can be placed in one goroutine. InputNode will consume
msg form msgstream, alloc one goroutine.
issue: #24826
pr: #28233
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-24 15:00:26 +08:00
c6ba4fa102
enhance: enhance the accuracy of memory usage ( #28554 ) ( #28559 )
...
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.
we allocates the memory by private anon mmap,
then rss - shared would be the accurate memory usage
resolve https://github.com/milvus-io/milvus/issues/28553
pr: #28554
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-24 14:12:25 +08:00
92e5f96064
fix: symbol 'GetStorageMetrics' and 'enableDynamicField' ( #28581 )
...
issue: #28579 #28504
pr: #28580
1. replace enableDynamic with enableDynamicField
2. cgo directly link to milvus_storage
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-21 15:30:23 +08:00
1383680b3a
add internal storage metrics ( #28279 )
...
/kind improvement
issue: #28277
pr: #28278
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-19 17:22:25 +08:00
cbd3e100b8
[cherry-pick]Add CSV file import function ( #28161 )
...
issue: #27148
from pr: #27149
Signed-off-by: kuma <675613722@qq.com>
Co-authored-by: kuma <675613722@qq.com>
2023-11-15 18:32:20 +08:00
16dc26833b
Fix retry when proxy stopped ( #28263 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-09 10:10:19 +08:00
29e66ed46b
Fix bulkinsert bug that segments are compacted after import ( #28227 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-11-08 10:18:20 +08:00
918333817e
Disable auto balance when old node exists ( #28191 ) ( #28224 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-08 07:10:17 +08:00
62e585d8db
fix check grpc error logic ( #28182 ) ( #28218 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 14:22:16 +08:00
bbcaf7a703
Fix coordinator fast restart ( #28205 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-11-07 10:04:16 +08:00
e8950a529e
fix grpc client retry on node server not match error ( #28169 ) ( #28177 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-07 00:44:17 +08:00
f11e99efed
Bulkinsert support pure list json ( #28126 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2023-11-03 16:02:24 +08:00
4558af94d5
fix retry on offline node ( #28079 ) ( #28139 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-11-03 16:00:17 +08:00
6060dd7ea8
Add chunk manager request timeout ( #27692 )
...
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
2023-10-23 20:08:08 +08:00
9b0ecbdca7
Support to replicate the mq message ( #27240 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-10-20 14:26:09 +08:00
bcbe98aba1
Add querynode client wrapper and avoid grpc in standalone mode ( #27781 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-19 11:10:07 +08:00
ac2d1bb5c2
Support receive signals from parent process ( #27756 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-18 20:20:11 +08:00
49b3a12804
Return newly defined merr instead of grpc unimplemented err ( #27751 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-18 15:32:11 +08:00
2f201c25e2
Remove deprecated io/ioutil usage ( #27747 )
...
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
2023-10-17 20:32:09 +08:00
7aa862c0ea
fix retry on unimplemented rpc error ( #27639 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-16 10:50:09 +08:00
ec1fe3549e
Add a stop hook to clean session ( #27564 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-10-16 10:24:10 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
e4f73cc805
Add host & enable_disk to session ( #27507 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-08 20:05:31 +08:00
8394b3a1ec
Block creating new error from status reason ( #27426 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-07 11:29:32 +08:00
5c1abfa2cc
Print the server id when active-standby switch ( #27119 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-07 10:01:31 +08:00
0f14d18201
Optimize the codec code of session ( #27360 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-01 10:33:30 +08:00
106c17f304
Make read ahead policy in ChunkCache configurable ( #27291 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-28 15:47:27 +08:00
c9653b1683
Add some log and improve TestSessionProcessActiveStandBy test case ( #27403 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-28 09:35:27 +08:00
wei liu
yihao.dai
chyezh
smellthemoon
congqixia
chyezh
wayblink
cai.zhang
zhenshan.cao
MrPresent-Han
yah01
PowderLi
KumaJie
groot
Xiaofan
zhagnlu
SimFG
jaime
Jiquan Long