milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,855 Commits
135 Branches
115 Tags
565 MiB
Commit Graph

6666 Commits (383915cfcd88fd7246d1a1e19b4bf218ba2521a8)

Author SHA1 Message Date
bigsheeper f994052099
Reduce printing frequency of rate limit log (#22636) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-09 14:15:55 +08:00
wayblink aa6212d5b0
[Cherry-pick] Fix channel checkpoint issues and fix GC (#22559) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: Ten Thousand Leaves <69466447+soothing-rain@users.noreply.github.com>
 2023-03-09 14:13:52 +08:00
XuanYang-cn b3287ca5ec
Fix the misleading log messages (#22606) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-09 10:39:51 +08:00
wei liu 11f1f4226a
support replica observer assign node (#22604) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-08 18:57:51 +08:00
yah01 efdc77bce6
Fix not fill timestamp when use collection created ts (#22629) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-08 18:53:51 +08:00
wei liu 1a1a5caafa
update shard leader cache periodically (#22595) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-08 15:13:51 +08:00
Enwei Jiao f1ec78d3c1
Ignore connection error when closing pulsar consumer (#22607) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-08 14:57:51 +08:00
XuanYang-cn 955bc06165
Fix search binary pagination failure (#22477) 
See also: #22168

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-08 11:03:51 +08:00
Jiquan Long e3c11069eb
Fix ddl tt go back (#22617) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-08 10:13:51 +08:00
Enwei Jiao f1a60b295c
Support multi yaml files (#22583) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-07 18:23:51 +08:00
congqixia f3e2d4a39a
Fix querynode stop meet unexpected GetComponentStates (#22590) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-07 09:51:50 +08:00
smellthemoon 29b3690c83
Upsert raise error (#22533) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-06 14:29:51 +08:00
smellthemoon c196c55381
Improve the assert information (#22578) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-06 14:25:50 +08:00
yah01 d4fccdd135
Add reason for spawning task (#22549) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-06 10:13:50 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-03-04 23:21:50 +08:00
Enwei Jiao b21395f487
Forbid dmlChannelNum auto refresh (#22550) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-03 15:21:48 +08:00
smellthemoon 0dae20c938
Layer range search results (#22539) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-03 15:19:48 +08:00
congqixia ea4ab9e4a4
Replace ErrorList with multiErrors (#22532) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-03 14:59:49 +08:00
yah01 319f1773af
Fix segment not found if forward delete to empty segment (#22528) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-03 14:13:49 +08:00
Gao aa19243c22
Fix diskann param parse bug (#22498) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-03-02 19:59:50 +08:00
congqixia 51cb8590a1
Use errors.Wrap and Combine so retry error could check `errors.Is` (#22520) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-02 17:25:48 +08:00
yah01 bdd6bc7695
Re-format cpp code (#22513) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-02 15:55:49 +08:00
congqixia fa86de530d
Restrain gc files by collection (#22504) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-02 11:49:47 +08:00
yah01 4d4bf99098
Fix DiskANN can't recovery after querynode reboots (#22488) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-02 10:17:48 +08:00
yah01 1ee10f3224
Refine the information while assertion failed (#22434) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-02 09:07:48 +08:00
Ten Thousand Leaves 74d158e149
Fix enableAutoCompaction default value (#22254) 
/kind bug

issue: #22253

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-03-02 07:21:48 +08:00
cai.zhang 5cae49e28d
Store index files in slices to reduce memory peaks (#22369) (#22455) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-03-01 19:43:47 +08:00
yah01 7478e44911
Support using mmap to load data (#22052) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-01 18:07:49 +08:00
congqixia fa5c33975e
Fix reset connection data race (#22483) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-01 11:21:47 +08:00
wei liu c162c6ecc0
fix assign node err (#22479) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-01 11:11:47 +08:00
yah01 21eea923cc
Add util to combine multiple errors so everyone checkable (#22473) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-28 19:35:47 +08:00
yah01 60cd548bf5
Rename concurrency package to conc (#22453) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-28 14:19:47 +08:00
congqixia 608615e5bd
Add background health check if ctx err returned (#22439) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-28 14:11:47 +08:00
Xiaofan 256aaa1944
Change Etcd session timeout to 20s (#22401) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-28 11:13:47 +08:00
Xiaofan fa101e9751
Change RockMQ page size to 256MB (#22432) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-28 10:45:47 +08:00
cai.zhang eefa29a5e8
Fix bug for get index state (#22396) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-28 10:29:50 +08:00
yah01 1779d25e9c
Fix ignoring error while subscribing channel (#22429) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-28 08:17:49 +08:00
wei liu d433e95ce0
fix transfer node meta err (#22420) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-27 19:29:47 +08:00
yah01 167581c08a
Refine Pool to improve the ease of use (#22412) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-27 18:07:48 +08:00
wayblink 416866e42f
Add memory usage too large sync policy (#22244) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-27 17:47:51 +08:00
smellthemoon 8d05874ff9
Fix get index mode error (#22364) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-27 14:45:47 +08:00
aoiasd 872721e3ec
Refactor invalid unit test (#22347) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-27 14:27:47 +08:00
yah01 b758c305a7
Make Future generic (#22411) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-27 12:07:47 +08:00
Xiaofan 3d6bded115
Fix Row Number Mismatch (#22307) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-27 10:41:46 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
smellthemoon cf999b68dd
Fix range search can not get status from knowhere (#22384) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-24 17:59:45 +08:00
zhenshan.cao e768437681
Correct usage of Timer and Ticker (#22228) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-02-23 18:59:45 +08:00
SimFG bddda5bdb2
Only print the file content when log the hook.yaml (#22361) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-23 15:33:45 +08:00
smellthemoon 820d0bc060
Fix range search error (#22352) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-23 14:21:46 +08:00
wei liu a9a263d5a8
fix assign node to replica in nodeUp (#22323) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-23 14:15:45 +08:00
Enwei Jiao 0851e05014
Gernate milvus.yaml by code (#22003) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-23 11:37:46 +08:00
yah01 3e9d497157
Avoid allocating a zero-bytes vector each search time (#22219) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-23 10:37:46 +08:00
SimFG 7e19a924b7
Add the rbac for the GetLoadingProgress and GetLoadState api (#22320) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-22 20:37:45 +08:00
aoiasd cc5d72a9f2
add search option to ignore ignore segment (#21994) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-22 17:31:46 +08:00
smellthemoon 7a4dfcc72b
Add status error output (#22325) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-22 15:51:44 +08:00
Jiquan Long 3cb9a44ed8
Add license to rootcoord module (#22301) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-22 11:37:45 +08:00
aoiasd 27b456dd46
Fix data race of data node test (#22221) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-22 10:28:26 +08:00
wei liu c3e8ad3629
fix balance generate reduce task (#22236) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-21 19:06:27 +08:00
congqixia abbf1efa0e
[Cherry-Pick] Fix read wrong variable caused nil error (#22297) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-21 18:38:26 +08:00
yah01 6f374fa19d
Fix fetching nodeID through session before init it (#22290) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-21 18:08:27 +08:00
Cai Yudong d4e0b6e91b
Remove unused index param check (#22271) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-02-21 17:36:26 +08:00
jaime 5936723904
Refine Read method of MinioChunkManager (#22235) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-21 16:22:26 +08:00
wei liu 0ce70b4f10
fix compact task data race (#22299) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-21 16:10:26 +08:00
wayblink b7c0d12d82
Skip file with prefix '.' may exist in filesystem (#22138) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-21 10:54:29 +08:00
smellthemoon 9e0ec15436
Support range search (#21652) 
Signed-off-by: smellthemoon <xinguo.li@zilliz.com>
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: jaime <yun.zhang@zilliz.com>
 2023-02-21 09:48:32 +08:00
Enwei Jiao 0a9a9058b9
Fix UT timeout (#22261) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-20 21:14:25 +08:00
Bennu ac6e879811
[skip e2e] fix etcd image for healthcheck (#22294) 
Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>
 2023-02-20 17:08:25 +08:00
Enwei Jiao b25b3ef431
Integreation with Velox (#22102) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-16 17:26:35 +08:00
yah01 187788059b
Fix double copy varchar field while loading (#22114) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-16 17:16:35 +08:00
congqixia fef12b53ef
Fix 2.2.2 possible broken channel cp (#22205) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-16 16:46:35 +08:00
aoiasd 9167773eff
Remove needless kv at dataSyncService (#22206) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-16 16:36:39 +08:00
Jiquan Long d9ca06eb4e
Fix max partition number don't work (#22216) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-16 16:34:38 +08:00
wei liu 13b508103e
refine mock querycoord (#22198) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 15:38:34 +08:00
bigsheeper f4e7b246f2
Fix msgdispatcher concurrency issue and add prometheus metrics (#22174) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-16 14:58:34 +08:00
xige-16 8ec5ba13dc
Fix search failed on disk index when search_list equal to limit (#21113) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-02-16 11:02:34 +08:00
wei liu 87a4ddc7e2
fix rg e2e (#22187) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 10:48:34 +08:00
congqixia 9346f1752a
Convert msg pchannel to vchannel before check IsCloseMsg (#22182) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-15 17:26:34 +08:00
congqixia f2575e5fa8
Add unconvert & durationcheck linters and fix issues (#22161) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-15 17:22:34 +08:00
aoiasd 148a024e05
Add tickle for datacoord watch event (#21193) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-15 16:20:34 +08:00
aoiasd 1894766235
Update datacoord compaction plan after datanode update plan to ensure consistency (#22143) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-15 16:00:33 +08:00
smellthemoon e20d79a8a1
Fix data race (#22171) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-15 15:40:33 +08:00
Cai Yudong 68f146d5df
Show knowhere version when system up (#22172) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-02-14 19:04:34 +08:00
wayblink 77a27c6dfc
Support print log to file and console at the same time (#21946) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-14 16:56:34 +08:00
wei liu 7b4511b8f4
fix transfer node (#22120) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-14 16:16:34 +08:00
SimFG c2a49d5a0b
Add error log and metrics for the hook and privilege interceptor (#22111) (#22137) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-13 16:50:33 +08:00
bigsheeper d2667064bb
Add msgDispatcher to support sharing msgs for different vChannel (#21917) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-13 16:38:33 +08:00
jaime a2435cfc4f
[Cherry-Pick]Add a segment metrics for count number of binlog files (#22103) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-13 10:24:33 +08:00
cai.zhang 511265c68c
Modify lastSyncTime in advance to prevent multiple flush binlogs (#22048) (#22088) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-12 18:56:33 +08:00
Xiaofan 6004867853
Add bucket name in error message (#21887) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-10 17:44:31 +08:00
cai.zhang 66b3566ac1
Update component state to healthy after start (#22118) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-10 17:40:32 +08:00
presburger 9950cacd10
support knowhere 2.0 (#21857) 
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
 2023-02-10 14:24:32 +08:00
XuanYang-cn 2b982b5d8f
Fix DataNode ut never meet condition (#22093) 
See also: #22079

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-02-10 11:04:32 +08:00
wei liu d078441522
support rg api rbac (#22097) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-10 10:54:33 +08:00
Xiaofan d3f8948c17
Fix embedded milvus (#22077) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-09 22:06:32 +08:00
liliu-z 915503de0b
Upgrade knowhere to 1.3.8 (#22087) 
Signed-off-by: Li Liu <li.liu@zilliz.com>
 2023-02-09 18:58:32 +08:00
wei liu e9684ddb5a
refine rg capacity behavior (#22089) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-09 18:46:31 +08:00
wei liu d085abbd56
fix load collection with rg (#22083) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-09 16:24:31 +08:00
yah01 b1f31da77a
Fix activate standby server ignores all errors (#22073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-09 15:24:31 +08:00
Enwei Jiao 44154a828c
Forbid updating some configs (#22047) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-09 14:28:31 +08:00
Gao 120164b0a2
Remove unnecessary plan deserialization (#22056) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-02-08 19:40:31 +08:00
wei liu 137e620a8c
fix search failed msg (#22036) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-08 15:44:31 +08:00
groot c21e11eb8a
Fix datanode crash bug for bulkinsert (#22041) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-02-08 14:06:31 +08:00
congqixia 81fbb885eb
Remove redundant optimization code (#22044) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-07 20:12:30 +08:00
wei liu 6fb60629be
fix describe rg api (#21998) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-07 19:30:31 +08:00
congqixia 7adabe09f1
Fix Alloc timstamp failure blocks compaction queue forever (#22039) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-07 19:02:31 +08:00
wei liu aced41d5d8
add resource group name rule (#21992) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-07 17:52:31 +08:00
Cai Yudong fc1d2d18cf
Optimize knowhere CMakeLists.txt (#22000) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2023-02-07 16:12:04 +08:00
Jiquan Long a548ce1cf1
Change default max shards num to 64 (#22016) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-07 14:43:57 +08:00
yah01 5abb8a0853
Improve search task merge policy (#22006) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-07 14:27:54 +08:00
XuanYang-cn a8db19fbdb
Fix ut multi-remove the timer (#22021) 
See also: #22019

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-02-07 14:15:54 +08:00
Jiquan Long 004a8c25ca
Remove hardcode max shards num (#22004) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-07 10:17:53 +08:00
jaime baa035885d
Modify default value of some configurations (#22013) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-07 10:05:56 +08:00
smellthemoon ffdb652911
Fix kafka ut (#21943) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-02-06 16:33:54 +08:00
Bingyi Sun 4005508880
Fix load collection blocks when memory is limited. (#21990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-02-06 16:07:53 +08:00
wei liu 5da6aade02
fix default capacity of default rg (#21965) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-06 15:55:56 +08:00
congqixia 73b6c13245
Expand singleflight to Get and make writeEvent sync (#21993) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-06 15:27:54 +08:00
yah01 73ce87dfe5
Fix reduce decreasing recall (#21981) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-06 11:23:53 +08:00
cai.zhang e25f987a5c
Fix bug that calculated an extra DataNode of register policy (#21927) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-06 10:33:54 +08:00
cai.zhang ae305a54bd
Set scheduler duration of index task can be configurable (#21944) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-05 17:21:53 +08:00
Ten Thousand Leaves f9495fef00
Minor fix on segment metric (#21976) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-02-03 19:33:52 +08:00
zhenshan.cao 5316743a00
Reduce memory usage of scalar index (#21970) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-02-03 19:01:53 +08:00
Enwei Jiao 6d05f102c1
Upgrade boost to 1.81 (#21954) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-03 17:21:53 +08:00
jaime 748a935a11
Modify segment seal policy with number of statslog files (#21957) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-03 17:07:52 +08:00
congqixia 787ce7c121
Add Rated group for checkpoint gc skip case (#21949) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-03 14:23:56 +08:00
Ten Thousand Leaves 74e3cc64fb
Add refresh path to proxy (#21933) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-02-02 19:29:51 +08:00
wayblink d41cc0b21b
Revoke session to only delete session key created by this node (#21935) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-02 16:37:52 +08:00
wayblink 88293b8b79
Add log to analysis when etcd txn operation failed (#21558) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-02 15:57:51 +08:00
aoiasd 6a6818ac32
Fix rocksmq retention test is not stable (#21925) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-02 14:09:51 +08:00
Xiaofan 2213ed7bed
Fix bulkload segment can not be flushed (#21919) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-02 13:11:51 +08:00
yah01 bdcff5b921
Avoid counting all bits to improve query performance (#21909) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-01 18:49:51 +08:00
yah01 9b491858a5
Fix datanode unit test not stable (#21911) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-01 18:19:50 +08:00
Enwei Jiao 53ae40b8c6
Make ratelimiter's config refreshable (#21757) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-01 16:03:51 +08:00
Bingyi Sun b12740e108
Remove a goroutine when creating a proxy client (#21904) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-02-01 15:49:50 +08:00
Cai Yudong e3d1624e97
Handle distance Inf correctly (#21828) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2023-02-01 14:59:51 +08:00
congqixia ec282b1958
Apply lifetime control for indexnode (#21892) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 18:37:50 +08:00
congqixia 01e59f8ec2
Fix master semver value (#21873) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 17:41:50 +08:00
wayblink 45ff568470
Fix probable ut fail caused by session key confict (#21883) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-31 17:39:49 +08:00
congqixia 11a3d0a50b
Use embed ETCD for datacoord unit test (#21890) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 14:47:49 +08:00
yah01 61c2f29ab3
Fix miss to throw exception when failed to create dir (#21886) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-31 13:49:50 +08:00
congqixia d329cf33a0
Use embed etcd for ConnectManager unit test (#21884) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 13:35:49 +08:00
Xiaofan f1daef22a1
Fix failed bulkload segment marked as sealed (#21799) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-31 12:41:53 +08:00
cai.zhang a04238c25f
Keep the configuration of indexCoord (#21879) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-31 12:07:50 +08:00
congqixia 8af2aebb66
Fix msg channel prefix config priority for backward compatibility (#21871) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 10:09:51 +08:00
Jiquan Long 270c5b0502
Refine log when initialize dml channels (#21863) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-30 14:05:54 +08:00
jaime 5ed6ae2f27
Remove repeated etcd endpoint configuration (#21860) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-30 12:07:48 +08:00
jaime e14f96a8e4
[skip e2e]Fix missed logger with information (#21859) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-30 11:51:49 +08:00
cai.zhang bcae97b125
Register after start to prevent there are tow coordinator at the same time (#21641) (#21707) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-30 11:11:50 +08:00
smellthemoon bb086f47e4
Swap execution flow order in enqueue (#21845) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-30 10:27:50 +08:00
wei liu 73c44d4b29
resource group impl (#21609) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-30 10:19:48 +08:00
congqixia 66027790a2
Implement detailed lifetime control for querynode (#21851) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-29 17:45:49 +08:00
yah01 43d633cfed
Fix fill the string field twice when string index exists (#21852) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-29 17:43:48 +08:00
congqixia 0480719e1a
Fix useasan format not correct (#21829) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-28 11:43:47 +08:00
Xiaofan 949d5d078f
Fix memory calculation in dataCodec (#21800) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-28 11:09:52 +08:00
jaime d7bb908148
Fix rename collection data race (#21827) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-24 10:01:46 +08:00
Xiaofan 9258f7059d
Fix Bulkload ut print too much result (#21821) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-19 21:31:44 +08:00
congqixia 54ec22e879
Allow empty AK/SK for minio cfg (#21815) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 15:49:44 +08:00
congqixia 5986106037
Make paramtable init only once (#21782) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 14:53:44 +08:00
jaime 1dcd9eeb79
Support rename collection (#21693) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-19 14:13:43 +08:00
Bingyi Sun c82b5d15b6
Add unsubscribe dmchannel task (#21513) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-18 17:45:44 +08:00
Bingyi Sun b0b0929c6c
Change CreateAt to UpdatedAt when checking timeout (#21769) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-18 17:29:43 +08:00
Ten Thousand Leaves defb7660c8
Add refresh option for LoadCollection/LoadPartitions interfaces (#21776) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-18 16:41:44 +08:00
Enwei Jiao cecd59d940
Disable time-travel by default (#21775) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-18 13:57:43 +08:00
Xiaofan 3e21d25de5
Add slow gc warning log (#21734) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-18 11:41:43 +08:00
groot 0af53d1e2e
Fix bulkinsert report bug (#21778) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-01-18 10:33:43 +08:00
yah01 78fd1987eb
Fix data race caused by no clone while updating index (#21772) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-17 18:11:42 +08:00
smellthemoon ad6cbc990f
Combine similar function (#21752) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-17 17:53:42 +08:00
Bingyi Sun b91bb5a729
Fix load timeout after next target updates (#21759) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-01-17 15:15:46 +08:00
presburger e64f55cd27
downgrade curl for compiled on aarch64 platform (#21739) 
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
 2023-01-17 14:53:42 +08:00
wayblink de584b508e
Fix active-standby switch fail bug (#21755) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-17 11:43:43 +08:00
yah01 c8f89907b6
Fix current target may be updated to an invalid target (#21742) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-17 11:41:51 +08:00
jaime 81326ca0a1
[skip e2e]Fix cpp lint errors (#21750) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-17 09:43:42 +08:00
yah01 6e66c10114
Fix check time tick lag with no channel (#21737) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-16 18:49:43 +08:00
Xiaofan e7f947ed0a
Check ScalarIndex deserialization (#21724) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-16 17:29:43 +08:00
yah01 836773f1a0
Optimize performance of reducing segments (#21722) 
- Improve the performance of reducing from O(knlogn) to O(nlogk)

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-16 15:25:42 +08:00
congqixia 5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-16 13:59:42 +08:00
yah01 837e3162d7
Fix ready notifiers leak when collection released (#21712) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-14 21:55:41 +08:00
yah01 32fb409e57
Fix may update the current target to an unavailable target when node down (#21698) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 17:11:41 +08:00
yah01 c66ce4aeba
Add warning comments (#21697) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 16:43:41 +08:00
groot c0e7a122d4
Fix bulkinsert progress bug (#21680) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-13 16:25:40 +08:00
Enwei Jiao 90d9e165d4
Fix some configs not shown (#21653) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-13 15:31:41 +08:00
smellthemoon 4620aebd46
Fix data race (#21706) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-01-13 14:29:41 +08:00
congqixia 782f942b6f
Add MLogger.WithRateGroup for logger (#21703) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-13 14:11:40 +08:00
Enwei Jiao 564173dcbc
Upgrade protobuf to 3.21.9 (#21679) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-13 14:05:41 +08:00
yah01 f4d80b4211
Fix reading config from etcd may block forever (#21688) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-13 11:29:41 +08:00
Xiaofan f8e1566b24
Support zstd compression in grpc (#21689) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-13 10:55:41 +08:00
congqixia 204db183f3
Prevent panicking when index segment info not exists (#21664) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-12 21:39:40 +08:00
wayblink 6a722396bd
Integration test framework (#21283) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-12 19:49:40 +08:00
Enwei Jiao 49f53003e9
upgrade libcurl to fix mac build failed (#21684) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 19:03:40 +08:00
wei liu c67070ab1f
add resource group interface (#21676) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-12 18:31:41 +08:00
Enwei Jiao fb42466c65
Use opentelemetry (#21509) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 16:09:39 +08:00
Enwei Jiao 9fccbbb92b
Remove factory in querycoord (#21659) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 15:59:39 +08:00
SimFG b307ee8c07
Make the rbac error message clearer (#21645) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-01-12 13:55:42 +08:00
Xiaofan 8f202274a7
Fix lag behind error with 0 servicetime (#21637) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-12 10:53:39 +08:00
Jiquan Long d0b974de07
Make rootcoord client compatible with older version (#21651) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-12 10:09:39 +08:00
Jiquan Long 9fd9bed2b9
Remove collection meta after GC finished (#21595) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-12 09:55:42 +08:00
Jiquan Long 6d09bbed68
[skip e2e] Fix load meta migration (#21584) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-11 19:31:39 +08:00
Ten Thousand Leaves 8c2c04517e
Fix an issue where flushed segments can turn into sealed (#21566) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-11 18:57:43 +08:00
groot 9f8095996b
Report bulkinsert progress (#21612) (#21638) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-11 17:37:44 +08:00
cai.zhang f02ccfb422
Set GetIndexBuildProgress is compatible with SDK version 2.1 (#21622) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 16:59:42 +08:00
cai.zhang e127cf7b99
Reset indexpb for upgrade (#21620) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 14:35:40 +08:00