milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,794 Commits
136 Branches
115 Tags
568 MiB
Commit Graph

6625 Commits (bd5fab1e539958ed90c0ad7e05bc52b0705842e3)

Author SHA1 Message Date
aoiasd 872721e3ec
Refactor invalid unit test (#22347) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-27 14:27:47 +08:00
yah01 b758c305a7
Make Future generic (#22411) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-27 12:07:47 +08:00
Xiaofan 3d6bded115
Fix Row Number Mismatch (#22307) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-27 10:41:46 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
smellthemoon cf999b68dd
Fix range search can not get status from knowhere (#22384) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-24 17:59:45 +08:00
zhenshan.cao e768437681
Correct usage of Timer and Ticker (#22228) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-02-23 18:59:45 +08:00
SimFG bddda5bdb2
Only print the file content when log the hook.yaml (#22361) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-23 15:33:45 +08:00
smellthemoon 820d0bc060
Fix range search error (#22352) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-23 14:21:46 +08:00
wei liu a9a263d5a8
fix assign node to replica in nodeUp (#22323) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-23 14:15:45 +08:00
Enwei Jiao 0851e05014
Gernate milvus.yaml by code (#22003) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-23 11:37:46 +08:00
yah01 3e9d497157
Avoid allocating a zero-bytes vector each search time (#22219) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-23 10:37:46 +08:00
SimFG 7e19a924b7
Add the rbac for the GetLoadingProgress and GetLoadState api (#22320) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-22 20:37:45 +08:00
aoiasd cc5d72a9f2
add search option to ignore ignore segment (#21994) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-22 17:31:46 +08:00
smellthemoon 7a4dfcc72b
Add status error output (#22325) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-22 15:51:44 +08:00
Jiquan Long 3cb9a44ed8
Add license to rootcoord module (#22301) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-22 11:37:45 +08:00
aoiasd 27b456dd46
Fix data race of data node test (#22221) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-22 10:28:26 +08:00
wei liu c3e8ad3629
fix balance generate reduce task (#22236) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-21 19:06:27 +08:00
congqixia abbf1efa0e
[Cherry-Pick] Fix read wrong variable caused nil error (#22297) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-21 18:38:26 +08:00
yah01 6f374fa19d
Fix fetching nodeID through session before init it (#22290) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-21 18:08:27 +08:00
Cai Yudong d4e0b6e91b
Remove unused index param check (#22271) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-02-21 17:36:26 +08:00
jaime 5936723904
Refine Read method of MinioChunkManager (#22235) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-21 16:22:26 +08:00
wei liu 0ce70b4f10
fix compact task data race (#22299) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-21 16:10:26 +08:00
wayblink b7c0d12d82
Skip file with prefix '.' may exist in filesystem (#22138) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-21 10:54:29 +08:00
smellthemoon 9e0ec15436
Support range search (#21652) 
Signed-off-by: smellthemoon <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
 2023-02-21 09:48:32 +08:00
Enwei Jiao 0a9a9058b9
Fix UT timeout (#22261) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-20 21:14:25 +08:00
Bennu ac6e879811
[skip e2e] fix etcd image for healthcheck (#22294) 
Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>
 2023-02-20 17:08:25 +08:00
Enwei Jiao b25b3ef431
Integreation with Velox (#22102) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-16 17:26:35 +08:00
yah01 187788059b
Fix double copy varchar field while loading (#22114) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-16 17:16:35 +08:00
congqixia fef12b53ef
Fix 2.2.2 possible broken channel cp (#22205) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-16 16:46:35 +08:00
aoiasd 9167773eff
Remove needless kv at dataSyncService (#22206) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-16 16:36:39 +08:00
Jiquan Long d9ca06eb4e
Fix max partition number don't work (#22216) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-16 16:34:38 +08:00
wei liu 13b508103e
refine mock querycoord (#22198) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 15:38:34 +08:00
bigsheeper f4e7b246f2
Fix msgdispatcher concurrency issue and add prometheus metrics (#22174) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-16 14:58:34 +08:00
xige-16 8ec5ba13dc
Fix search failed on disk index when search_list equal to limit (#21113) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-02-16 11:02:34 +08:00
wei liu 87a4ddc7e2
fix rg e2e (#22187) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 10:48:34 +08:00
congqixia 9346f1752a
Convert msg pchannel to vchannel before check IsCloseMsg (#22182) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-15 17:26:34 +08:00
congqixia f2575e5fa8
Add unconvert & durationcheck linters and fix issues (#22161) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-15 17:22:34 +08:00
aoiasd 148a024e05
Add tickle for datacoord watch event (#21193) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-15 16:20:34 +08:00
aoiasd 1894766235
Update datacoord compaction plan after datanode update plan to ensure consistency (#22143) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-15 16:00:33 +08:00
smellthemoon e20d79a8a1
Fix data race (#22171) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-15 15:40:33 +08:00
Cai Yudong 68f146d5df
Show knowhere version when system up (#22172) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-02-14 19:04:34 +08:00
wayblink 77a27c6dfc
Support print log to file and console at the same time (#21946) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-14 16:56:34 +08:00
wei liu 7b4511b8f4
fix transfer node (#22120) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-14 16:16:34 +08:00
SimFG c2a49d5a0b
Add error log and metrics for the hook and privilege interceptor (#22111) (#22137) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-13 16:50:33 +08:00
bigsheeper d2667064bb
Add msgDispatcher to support sharing msgs for different vChannel (#21917) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-13 16:38:33 +08:00
jaime a2435cfc4f
[Cherry-Pick]Add a segment metrics for count number of binlog files (#22103) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-13 10:24:33 +08:00
cai.zhang 511265c68c
Modify lastSyncTime in advance to prevent multiple flush binlogs (#22048) (#22088) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-12 18:56:33 +08:00
Xiaofan 6004867853
Add bucket name in error message (#21887) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-10 17:44:31 +08:00
cai.zhang 66b3566ac1
Update component state to healthy after start (#22118) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-10 17:40:32 +08:00
presburger 9950cacd10
support knowhere 2.0 (#21857) 
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
 2023-02-10 14:24:32 +08:00
XuanYang-cn 2b982b5d8f
Fix DataNode ut never meet condition (#22093) 
See also: #22079

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-02-10 11:04:32 +08:00
wei liu d078441522
support rg api rbac (#22097) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-10 10:54:33 +08:00
Xiaofan d3f8948c17
Fix embedded milvus (#22077) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-09 22:06:32 +08:00
liliu-z 915503de0b
Upgrade knowhere to 1.3.8 (#22087) 
Signed-off-by: Li Liu <li.liu@zilliz.com>
 2023-02-09 18:58:32 +08:00
wei liu e9684ddb5a
refine rg capacity behavior (#22089) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-09 18:46:31 +08:00
wei liu d085abbd56
fix load collection with rg (#22083) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-09 16:24:31 +08:00
yah01 b1f31da77a
Fix activate standby server ignores all errors (#22073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-09 15:24:31 +08:00
Enwei Jiao 44154a828c
Forbid updating some configs (#22047) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-09 14:28:31 +08:00
Gao 120164b0a2
Remove unnecessary plan deserialization (#22056) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-02-08 19:40:31 +08:00
wei liu 137e620a8c
fix search failed msg (#22036) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-08 15:44:31 +08:00
groot c21e11eb8a
Fix datanode crash bug for bulkinsert (#22041) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-02-08 14:06:31 +08:00
congqixia 81fbb885eb
Remove redundant optimization code (#22044) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-07 20:12:30 +08:00
wei liu 6fb60629be
fix describe rg api (#21998) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-07 19:30:31 +08:00
congqixia 7adabe09f1
Fix Alloc timstamp failure blocks compaction queue forever (#22039) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-07 19:02:31 +08:00
wei liu aced41d5d8
add resource group name rule (#21992) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-07 17:52:31 +08:00
Cai Yudong fc1d2d18cf
Optimize knowhere CMakeLists.txt (#22000) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2023-02-07 16:12:04 +08:00
Jiquan Long a548ce1cf1
Change default max shards num to 64 (#22016) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-07 14:43:57 +08:00
yah01 5abb8a0853
Improve search task merge policy (#22006) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-07 14:27:54 +08:00
XuanYang-cn a8db19fbdb
Fix ut multi-remove the timer (#22021) 
See also: #22019

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-02-07 14:15:54 +08:00
Jiquan Long 004a8c25ca
Remove hardcode max shards num (#22004) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-07 10:17:53 +08:00
jaime baa035885d
Modify default value of some configurations (#22013) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-07 10:05:56 +08:00
smellthemoon ffdb652911
Fix kafka ut (#21943) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-06 16:33:54 +08:00
Bingyi Sun 4005508880
Fix load collection blocks when memory is limited. (#21990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-02-06 16:07:53 +08:00
wei liu 5da6aade02
fix default capacity of default rg (#21965) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-06 15:55:56 +08:00
congqixia 73b6c13245
Expand singleflight to Get and make writeEvent sync (#21993) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-06 15:27:54 +08:00
yah01 73ce87dfe5
Fix reduce decreasing recall (#21981) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-06 11:23:53 +08:00
cai.zhang e25f987a5c
Fix bug that calculated an extra DataNode of register policy (#21927) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-06 10:33:54 +08:00
cai.zhang ae305a54bd
Set scheduler duration of index task can be configurable (#21944) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-05 17:21:53 +08:00
Ten Thousand Leaves f9495fef00
Minor fix on segment metric (#21976) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-02-03 19:33:52 +08:00
zhenshan.cao 5316743a00
Reduce memory usage of scalar index (#21970) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-02-03 19:01:53 +08:00
Enwei Jiao 6d05f102c1
Upgrade boost to 1.81 (#21954) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-03 17:21:53 +08:00
jaime 748a935a11
Modify segment seal policy with number of statslog files (#21957) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-03 17:07:52 +08:00
congqixia 787ce7c121
Add Rated group for checkpoint gc skip case (#21949) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-03 14:23:56 +08:00
Ten Thousand Leaves 74e3cc64fb
Add refresh path to proxy (#21933) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-02-02 19:29:51 +08:00
wayblink d41cc0b21b
Revoke session to only delete session key created by this node (#21935) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-02 16:37:52 +08:00
wayblink 88293b8b79
Add log to analysis when etcd txn operation failed (#21558) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-02 15:57:51 +08:00
aoiasd 6a6818ac32
Fix rocksmq retention test is not stable (#21925) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-02 14:09:51 +08:00
Xiaofan 2213ed7bed
Fix bulkload segment can not be flushed (#21919) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-02 13:11:51 +08:00
yah01 bdcff5b921
Avoid counting all bits to improve query performance (#21909) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-01 18:49:51 +08:00
yah01 9b491858a5
Fix datanode unit test not stable (#21911) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-01 18:19:50 +08:00
Enwei Jiao 53ae40b8c6
Make ratelimiter's config refreshable (#21757) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-01 16:03:51 +08:00
Bingyi Sun b12740e108
Remove a goroutine when creating a proxy client (#21904) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-02-01 15:49:50 +08:00
Cai Yudong e3d1624e97
Handle distance Inf correctly (#21828) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2023-02-01 14:59:51 +08:00
congqixia ec282b1958
Apply lifetime control for indexnode (#21892) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 18:37:50 +08:00
congqixia 01e59f8ec2
Fix master semver value (#21873) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 17:41:50 +08:00
wayblink 45ff568470
Fix probable ut fail caused by session key confict (#21883) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-31 17:39:49 +08:00
congqixia 11a3d0a50b
Use embed ETCD for datacoord unit test (#21890) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 14:47:49 +08:00
yah01 61c2f29ab3
Fix miss to throw exception when failed to create dir (#21886) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-31 13:49:50 +08:00
congqixia d329cf33a0
Use embed etcd for ConnectManager unit test (#21884) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 13:35:49 +08:00
Xiaofan f1daef22a1
Fix failed bulkload segment marked as sealed (#21799) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-31 12:41:53 +08:00
cai.zhang a04238c25f
Keep the configuration of indexCoord (#21879) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-31 12:07:50 +08:00
congqixia 8af2aebb66
Fix msg channel prefix config priority for backward compatibility (#21871) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 10:09:51 +08:00
Jiquan Long 270c5b0502
Refine log when initialize dml channels (#21863) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-30 14:05:54 +08:00
jaime 5ed6ae2f27
Remove repeated etcd endpoint configuration (#21860) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-30 12:07:48 +08:00
jaime e14f96a8e4
[skip e2e]Fix missed logger with information (#21859) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-30 11:51:49 +08:00
cai.zhang bcae97b125
Register after start to prevent there are tow coordinator at the same time (#21641) (#21707) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-30 11:11:50 +08:00
smellthemoon bb086f47e4
Swap execution flow order in enqueue (#21845) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-30 10:27:50 +08:00
wei liu 73c44d4b29
resource group impl (#21609) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-30 10:19:48 +08:00
congqixia 66027790a2
Implement detailed lifetime control for querynode (#21851) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-29 17:45:49 +08:00
yah01 43d633cfed
Fix fill the string field twice when string index exists (#21852) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-29 17:43:48 +08:00
congqixia 0480719e1a
Fix useasan format not correct (#21829) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-28 11:43:47 +08:00
Xiaofan 949d5d078f
Fix memory calculation in dataCodec (#21800) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-28 11:09:52 +08:00
jaime d7bb908148
Fix rename collection data race (#21827) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-24 10:01:46 +08:00
Xiaofan 9258f7059d
Fix Bulkload ut print too much result (#21821) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-19 21:31:44 +08:00
congqixia 54ec22e879
Allow empty AK/SK for minio cfg (#21815) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 15:49:44 +08:00
congqixia 5986106037
Make paramtable init only once (#21782) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 14:53:44 +08:00
jaime 1dcd9eeb79
Support rename collection (#21693) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-19 14:13:43 +08:00
Bingyi Sun c82b5d15b6
Add unsubscribe dmchannel task (#21513) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-18 17:45:44 +08:00
Bingyi Sun b0b0929c6c
Change CreateAt to UpdatedAt when checking timeout (#21769) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-18 17:29:43 +08:00
Ten Thousand Leaves defb7660c8
Add refresh option for LoadCollection/LoadPartitions interfaces (#21776) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-18 16:41:44 +08:00
Enwei Jiao cecd59d940
Disable time-travel by default (#21775) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-18 13:57:43 +08:00
Xiaofan 3e21d25de5
Add slow gc warning log (#21734) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-18 11:41:43 +08:00
groot 0af53d1e2e
Fix bulkinsert report bug (#21778) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-01-18 10:33:43 +08:00
yah01 78fd1987eb
Fix data race caused by no clone while updating index (#21772) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-17 18:11:42 +08:00
smellthemoon ad6cbc990f
Combine similar function (#21752) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-17 17:53:42 +08:00
Bingyi Sun b91bb5a729
Fix load timeout after next target updates (#21759) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-17 15:15:46 +08:00
presburger e64f55cd27
downgrade curl for compiled on aarch64 platform (#21739) 
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
 2023-01-17 14:53:42 +08:00
wayblink de584b508e
Fix active-standby switch fail bug (#21755) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-17 11:43:43 +08:00
yah01 c8f89907b6
Fix current target may be updated to an invalid target (#21742) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-17 11:41:51 +08:00
jaime 81326ca0a1
[skip e2e]Fix cpp lint errors (#21750) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-17 09:43:42 +08:00
yah01 6e66c10114
Fix check time tick lag with no channel (#21737) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-16 18:49:43 +08:00
Xiaofan e7f947ed0a
Check ScalarIndex deserialization (#21724) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-16 17:29:43 +08:00
yah01 836773f1a0
Optimize performance of reducing segments (#21722) 
- Improve the performance of reducing from O(knlogn) to O(nlogk)

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-16 15:25:42 +08:00
congqixia 5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-16 13:59:42 +08:00
yah01 837e3162d7
Fix ready notifiers leak when collection released (#21712) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-14 21:55:41 +08:00
yah01 32fb409e57
Fix may update the current target to an unavailable target when node down (#21698) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 17:11:41 +08:00
yah01 c66ce4aeba
Add warning comments (#21697) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 16:43:41 +08:00
groot c0e7a122d4
Fix bulkinsert progress bug (#21680) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-13 16:25:40 +08:00
Enwei Jiao 90d9e165d4
Fix some configs not shown (#21653) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-13 15:31:41 +08:00
smellthemoon 4620aebd46
Fix data race (#21706) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-13 14:29:41 +08:00
congqixia 782f942b6f
Add MLogger.WithRateGroup for logger (#21703) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-13 14:11:40 +08:00
Enwei Jiao 564173dcbc
Upgrade protobuf to 3.21.9 (#21679) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-13 14:05:41 +08:00
yah01 f4d80b4211
Fix reading config from etcd may block forever (#21688) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 11:29:41 +08:00
Xiaofan f8e1566b24
Support zstd compression in grpc (#21689) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-13 10:55:41 +08:00
congqixia 204db183f3
Prevent panicking when index segment info not exists (#21664) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-12 21:39:40 +08:00
wayblink 6a722396bd
Integration test framework (#21283) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-12 19:49:40 +08:00
Enwei Jiao 49f53003e9
upgrade libcurl to fix mac build failed (#21684) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 19:03:40 +08:00
wei liu c67070ab1f
add resource group interface (#21676) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-12 18:31:41 +08:00
Enwei Jiao fb42466c65
Use opentelemetry (#21509) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 16:09:39 +08:00
Enwei Jiao 9fccbbb92b
Remove factory in querycoord (#21659) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 15:59:39 +08:00
SimFG b307ee8c07
Make the rbac error message clearer (#21645) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-01-12 13:55:42 +08:00
Xiaofan 8f202274a7
Fix lag behind error with 0 servicetime (#21637) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-12 10:53:39 +08:00
Jiquan Long d0b974de07
Make rootcoord client compatible with older version (#21651) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-12 10:09:39 +08:00
Jiquan Long 9fd9bed2b9
Remove collection meta after GC finished (#21595) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-12 09:55:42 +08:00
Jiquan Long 6d09bbed68
[skip e2e] Fix load meta migration (#21584) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-11 19:31:39 +08:00
Ten Thousand Leaves 8c2c04517e
Fix an issue where flushed segments can turn into sealed (#21566) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-11 18:57:43 +08:00
groot 9f8095996b
Report bulkinsert progress (#21612) (#21638) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-11 17:37:44 +08:00
cai.zhang f02ccfb422
Set GetIndexBuildProgress is compatible with SDK version 2.1 (#21622) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 16:59:42 +08:00
cai.zhang e127cf7b99
Reset indexpb for upgrade (#21620) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 14:35:40 +08:00
groot df0ffd08ce
Update segment id for import task (#21621) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-11 11:55:39 +08:00
liliu-z cc5ecabac6
Skip search GRPC call for standalon (#21383) 
Signed-off-by: Li Liu <li.liu@zilliz.com>
 2023-01-10 21:09:38 +08:00
bigsheeper 2146af1fb2
Return insufficient memory error when load failed (#21574) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-01-10 20:35:39 +08:00
congqixia c550427e81
Use singleflight in localCache.load (#21606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-10 16:21:39 +08:00
yah01 3b41a6931d
Add QueryCoord OWNERS file (#21552) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-10 16:15:37 +08:00
groot aab20a4031
Streaming read numpy (#21579) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-01-10 14:53:38 +08:00
cai.zhang 5d660f16d6
Forbid IndexCoord start with version<2.3 (#21597) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-10 14:33:38 +08:00
Ten Thousand Leaves 23ceb9c638
Remove index checking phase for bulk insert (#21578) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-09 21:15:38 +08:00
congqixia 42ab1b0a48
Add segment gc restraint of dml position before channel cp (#21564) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-06 21:33:36 +08:00
Xiaofan dc2bfb0901
Add stack trace for core dump (#21555) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-06 16:53:35 +08:00
Xiaofan 4b4944ecee
Support sync all segments while close (#21421) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-06 14:49:36 +08:00
MrPresent-Han 6fb3542f2a
enable auto balance paramter(#21504) (#21507) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-01-06 14:45:35 +08:00
jaime 73d8509600
Improve ListSegments performance for datacoord catalog (#21477) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-06 14:33:36 +08:00
bigsheeper 79535931ab
Return error code when deny to read/write (#21561) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-01-06 14:31:37 +08:00
cai.zhang aa203acfb3
Low IndexCoord weight (#21548) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-06 14:21:37 +08:00
yah01 e8e2696430
avoid checking search params many times (#21516) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-06 13:15:35 +08:00
Ten Thousand Leaves b3da554705
Increase bulk insert file size limit (#21526) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-06 13:09:36 +08:00
jaime 6d53a99171
Improve LoadWithPrefix performance for SuffixSnapshot (#21524) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-06 10:11:36 +08:00
jaime 09e1d4fc83
Remove redundant LoadPrefix requests for Catalog ListCollections (#21551) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-06 10:09:36 +08:00
Enwei Jiao 24d5faf7aa
Fix show configuration bug (#21544) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-06 09:55:36 +08:00
congqixia 3d0f516fd7
Add With method for MLogger (#21530) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-05 13:13:34 +08:00
cai.zhang efefba4da4
Fix checkpoint will not update after delete and compact (#21495) (#21499) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-05 11:31:35 +08:00
cai.zhang 9dd2e1dcd8
Use tt msg stream for consume delete msg (#21478) (#21500) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-05 11:03:35 +08:00
jaime 58b79eb74c
Add based on timetravel GC for snapshot KV (#21417) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-04 21:37:35 +08:00
Jiquan Long 6e521dd4b3
Optimize GetLoadingProgress to save RPC (#21493) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-04 20:33:35 +08:00
cai.zhang e5f408dceb
Merge IndexCoord and DataCoord (#21267) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-04 19:37:36 +08:00
yah01 7b39873ae0
limit the frequency of GetMetrics() log (#21514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-04 17:39:35 +08:00
smellthemoon bf3c02155a
Support update (#20875) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-04 17:21:36 +08:00
Jiquan Long ff2a68e65a
Fix collection not exist when tried to do recovery (#21471) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-04 16:37:35 +08:00
Ten Thousand Leaves 3f6d139ad2
Fix bad import task expiration default (#21235) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-04 14:11:33 +08:00
congqixia 2844f9b1e9
Setup first tsafe to avoid false maxLag error (#21490) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-03 20:41:33 +08:00
Bingyi Sun 72184eaeaf
Upgrade pulsar client (#21456) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-03 16:27:34 +08:00
wei liu e735485bb8
fix empty auth config unmarshal (#21473) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-03 10:11:34 +08:00
SimFG 289df1cc3d
Make the `UnsubDmChannel` rpc unsuccessful when the releaseCollectionTask fails (#21469) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-01-02 16:39:32 +08:00
bigsheeper c187de7754
Show trigger reason when deny to read/write (#21368) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-12-30 18:35:32 +08:00
bigsheeper cf40bb9b5c
Improve datacoord log (#21453) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-12-30 15:37:31 +08:00
Jiquan Long de9ed42623
Forbid deleting entities by non-pk field (#21459) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-12-30 14:17:32 +08:00
SimFG df6ccfa104
Fix wrong bool param in the `getDistribution` method (#21463) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-30 14:15:31 +08:00
congqixia daa4492e95
Invalidate cache first when shard leader list contains error (#21451) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-29 23:29:32 +08:00
SimFG e6d2849c9a
Check whether the node is stopping when using `load balance` api (#21438) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-29 15:47:31 +08:00
Jiquan Long d7156812c1
Try using ASAN in ci ut (#21089) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-12-29 15:29:31 +08:00