milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,855 Commits
136 Branches
115 Tags
566 MiB
Commit Graph

6666 Commits (383915cfcd88fd7246d1a1e19b4bf218ba2521a8)

Author SHA1 Message Date
Xiaofan d16b7c3c2d
Add more logs for debugging load balance (#21389) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-27 15:57:30 +08:00
jaime a8a1b0b3e8
Remove logs that print insert results (#21394) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2022-12-27 12:49:29 +08:00
wayblink 28b44b781f
Update healthz to support standby state and fix rootcoord active-standby (#21078) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-12-27 12:47:34 +08:00
smellthemoon cded2ce77c
Fix offset panic (#21292) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-27 11:11:29 +08:00
Enwei Jiao 264f08953d
Refactor Rocksmq's config (#21304) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-26 19:11:30 +08:00
congqixia 3c9840d6d3
Fix some datanode instance not stopped in unit tests (#21385) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-26 16:59:30 +08:00
Xiaofan 5d044c4f29
Add check for stats log and bin log number (#21264) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-26 16:35:30 +08:00
smellthemoon 1e5cd9dc6e
Modify comment (#21362) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-26 15:27:30 +08:00
Enwei Jiao 77de8fc746
Fix boost compile error (#21374) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-26 09:57:29 +08:00
yah01 396a85c926
Fix the number of executing tasks may break the limit (#21318) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-20 20:33:27 +08:00
jaime d401608899
Add a segment seal policy by number of binlog files (#21263) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-20 14:09:25 +08:00
Enwei Jiao ae41a6583f
Fix compile on M1 (#21298) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-19 15:05:24 +08:00
jaime e05ac56283
Revert build arrow with Conan (#21258) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-16 18:27:24 +08:00
congqixia 74db56dece
Fix cache refetch logic not applied to Query (#21281) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-16 16:49:23 +08:00
yah01 9fa58968b0
Fix proxy doesn't update shard leaders cache for some types of error (#21185) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-16 16:05:24 +08:00
Enwei Jiao 166e9f0da5
Refactor GrpcConfig (#21142) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-16 15:59:23 +08:00
SimFG 63cd4132a6
Implement the `GetLoadState` api (#21257) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-16 14:39:24 +08:00
SimFG 4df91ddf60
Support the graceful stop for the indexnode (#21021) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-16 10:03:27 +08:00
Ten Thousand Leaves ef3fcdf65b
Small candidate compaction should only happen with >1 segments (#21249) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-16 10:01:23 +08:00
jaime 0954014c6d
[skip e2e] Fix broken default parameters for seal segment policy (#21207) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-15 18:31:23 +08:00
Enwei Jiao e6374901f9
Make TopK Limit configurable (#21151) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-15 14:03:22 +08:00
aoiasd 0011b65338
Fix unit test of local cache cause data race (#21237) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-15 13:41:22 +08:00
liliu-z 6188960e85
Upgrade Knowhere to 1.3.6 to fix bugs (#21240) 
Signed-off-by: Li Liu <li.liu@zilliz.com>

Signed-off-by: Li Liu <li.liu@zilliz.com>
 2022-12-15 12:15:22 +08:00
zhenshan.cao ce3e0fcf9e
Fixbug: proxy meta_cache partially update (#21232) (#21242) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-12-14 20:45:23 +08:00
congqixia 1518baacd1
Make sure querynode stop only once (#21203) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-14 18:21:23 +08:00
yah01 cc56d58052
Fix panic while concurrent releasing payload writer/reader (#21218) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-14 16:23:22 +08:00
Ten Thousand Leaves 9d8f422cb8
Reduce maxFieldNum from 256 to 64 (#21104) 
issue: #21077

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-14 10:39:22 +08:00
SimFG f31d5facff
Fix the unsafe casbin `Model` (#21129) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-14 10:29:22 +08:00
SimFG 655db658c8
Empty implement of the `GetLoadState` api (#21205) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-14 10:23:22 +08:00
Ten Thousand Leaves b01546edd3
Guard bad distance/score from knowhere (#21034) 
/kind bug

issue: #21138
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-13 17:03:21 +08:00
bigsheeper fc10c74005
Use channel-cp as seekPosition when FromDmlCPLoadDelete (#21110) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-12-13 16:17:22 +08:00
bigsheeper f595500383
Move sync delete policy from deleteNode to insertBufferNode (#21152) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-12-13 16:15:21 +08:00
Ten Thousand Leaves a81c9cc11c
Recalculate segment max size during compaction (#21077) 
issue: #21077

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-13 15:39:21 +08:00
Enwei Jiao d553b5108e
Fix jemalloc.so not found (#21106) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-13 11:27:21 +08:00
zhenshan.cao 14e737ee2d
Fix metric with wrong label (#21127) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-12-13 09:51:21 +08:00
XuanYang-cn 71cab36e37
Fix channelmanager ut (#21140) 
See also: #17199

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-12-12 19:31:22 +08:00
congqixia 53782ab83e
Fill collection schema after DataCoord restarts (#21161) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-12 19:05:21 +08:00
smellthemoon b04eb17088
Fix load timeout (#21156) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-12 18:43:23 +08:00
congqixia b0afdb2180
Fix access log panicking for nil response (#21149) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-12 17:23:22 +08:00
yah01 5ba1a94858
Fix observers may update current target to a unfinished next target (#21107) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-12 15:15:22 +08:00
XuanYang-cn b850c4ae2c
Refine search logs in querynode (#21134) 
- Make sure all logs logging with ctx
- Add an info log for searching on not-indexed segments
- Separate log prefix of shardleader and non-shardleaders

See also: #20975

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-12-12 14:33:22 +08:00
XuanYang-cn a7b24cbc53
Move APIs of types.DataNode to services.go (#21042) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-12-12 10:57:22 +08:00
Xiaofan e977e014a9
Fix flush didn't respect binaryvector and other schemas (#21120) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-12 10:33:26 +08:00
cai.zhang 0faf24cc54
Forbid gc index meta when creating index (#21061) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-12-12 09:51:20 +08:00
zhenshan.cao abc2adb1f8
Add indexCountLabel for metric QueryNodeNumEntities (#21081) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-12-10 23:45:20 +08:00
Ten Thousand Leaves 699b63fb42
Fix metric with wrong label (#21108) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-10 10:55:20 +08:00
liliu-z d326c385bb
Upgrade Knowhere to 1.3.5 to fix DiskANN aio issue (#21109) 
Signed-off-by: Li Liu <li.liu@zilliz.com>

Signed-off-by: Li Liu <li.liu@zilliz.com>
 2022-12-09 22:53:20 +08:00
aoiasd 9079b1d5a4
Rename default accesslog dir name (#21086) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-09 18:09:19 +08:00
SimFG 4a5c282b62
Add the superuser config (#21090) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-09 16:13:20 +08:00
Ten Thousand Leaves d67db2ed9b
Make compaction target able to exceed segment max size by a bit (#21068) 
issue: #21053
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-09 16:03:20 +08:00
aoiasd de0ab9e2cf
Refactor showConfigurations to allow return global config rather than only return config of this component (#21063) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-09 14:31:21 +08:00
Xiaofan 908023a06e
Add more test on file reader (#21082) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-09 11:55:19 +08:00
SimFG d67e878f96
Fix the concurrent write issue when init privilege policy (#21071) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-08 19:43:23 +08:00
aoiasd 60abe08f85
Add retention for access log on minio (#20681) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-08 19:39:19 +08:00
yah01 9ebaa10dec
Add more logs for GetShardLeaders (#21046) 
Also increase the heartbeatAvailableInterval from 2.5s to 10s

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-08 19:09:18 +08:00
Bingyi Sun 27219ce9c0
Fix load progress bug (#21069) 
issue: #21023

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-12-08 18:41:20 +08:00
smellthemoon 18cad3a1fb
Optimization of delete and insert (#20990) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-08 18:37:19 +08:00
cai.zhang eb7ef01b9a
Fix bug for ListWithPrefix from minio with recursive is false (#21059) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-12-08 15:47:18 +08:00
Jiquan Long 6709ccf690
Fix empty retrieve result (#21032) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-12-08 10:15:19 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
yah01 80a2a49681
Improve error message when target id not match (#21043) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-07 17:17:19 +08:00
Jiquan Long dee426a7a5
Fix group checker ut (#21038) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-12-07 16:11:22 +08:00
Ji Bin 6692c9999e
Using first unicastable address as local ip (#20674) 
Signed-off-by: Ji Bin <matrixji@live.com>

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-12-07 15:03:19 +08:00
groot 33a102c6fc
Refine bulkinsert (#20986) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-12-07 14:53:18 +08:00
Xiaofan e131915207
Support 64M Rpc Limit (#20845) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-07 10:07:18 +08:00
SimFG f8cff79804
Support the graceful stop for the query node (#20851) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-06 22:59:19 +08:00
congqixia 5e49b095f5
Fix compaction handler may double release and block forever (#21015) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-06 20:09:18 +08:00
yah01 0297ab1a46
Make progress if any channel/segment was loaded on node (#20775) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-06 18:29:19 +08:00
yah01 11e4445ef7
Check whether segments are fully loaded while fetching shard leaders (#20991) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-06 18:05:18 +08:00
XuanYang-cn f844aa0cab
Remove not inuse stale segments in flushmsg (#20981) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-12-06 17:51:18 +08:00
smellthemoon c49f20ea94
Load should base on the accurate loaded number (#20999) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-06 17:13:18 +08:00
Ten Thousand Leaves 2f5b03fef7
Split small segment proportion and small segment compactable proportion (#20996) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-06 14:37:18 +08:00
jaime 039e9ce4bb
Refine msgstream interface (#20832) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-05 20:55:17 +08:00
jaime 548e90ec68
Support an analogous Java Guava cache implementation (#20831) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-05 20:29:18 +08:00
XuanYang-cn 243d8cff82
Provide different channelNames for channelmanager test (#20969) 
Fixes: #20790

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-12-05 15:39:22 +08:00
yah01 aec347e591
Check last heartbeat response time while fetching shard leaders (#20968) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-05 15:09:20 +08:00
yah01 47b76e13be
Fix shard leader misses to fill target ID while delegates query/search (#20973) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-05 14:09:17 +08:00
congqixia 2791f5526c
Fix BufferData limit calculation for BinaryVector (#20963) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-05 11:15:17 +08:00
XuanYang-cn 3a558aa04f
Add ut for rootcoord/kv/kv_catalog.go (#20909) 
Add ut for CreateCredential, GetCredential, Dropcredential,
ListCredentials, CreateRole, DropRole AlterUserRole, ListRole,
ListUser, ListUserRole, DeleteGrant, AlterGrant, ListGrant, and, ListPolicy

Fixes: #20833

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-12-05 11:05:17 +08:00
yah01 25a3b9ae19
Check target node ID for query/search (#20967) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-04 20:05:17 +08:00
congqixia 4238b5dbab
Make datanode unittest use embed etcd server (#20948) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-02 16:05:16 +08:00
yah01 482986cdf6
Revert "Fix fake segment is never handled (#20935)" (#20945) 
This reverts commit 8f449f2996.

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-02 11:41:16 +08:00
yah01 ec3aef1f3e
Add time records for search path (#20924) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-02 11:39:17 +08:00
liliu-z aecf2e4f4b
Upgrade knowhere to 1.3.4 (#20914) 
Signed-off-by: Li Liu <li.liu@zilliz.com>

Signed-off-by: Li Liu <li.liu@zilliz.com>
 2022-12-02 00:33:21 +08:00
congqixia 40abb13413
Generate unique id with ts for sync task (#20928) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-01 20:35:18 +08:00
congqixia f745d7f489
Fix compaction target segment rowNum is always 0 (#20937) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-01 20:33:17 +08:00
zhenshan.cao 9724ae5e27
Refine log level (#20904) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-12-01 20:07:15 +08:00
yah01 8f449f2996
Fix fake segment is never handled (#20935) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-01 20:03:16 +08:00
Ten Thousand Leaves b238f4ee58
Remove bad & duplicate metric (#20905) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-01 16:39:16 +08:00
cai.zhang 9a42cff9b4
GetFlushedSegments should include flushing segment (#20859) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-12-01 16:35:20 +08:00
Ten Thousand Leaves c373b44c2c
Verify and adjust segment # of rows before saving and passing SegmentInfo (#20837) 
issue: #18120

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-01 16:31:16 +08:00
yah01 ddd29ea6ab
Optimize scheduler, increase merge tasks probability (#20922) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-01 16:25:16 +08:00
smellthemoon f7d603f947
Fix Group Checker test fail (#20911) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-01 13:39:15 +08:00
congqixia 82fe655032
Setup lastSyncTs when add segment to avoid sync next timetick (#20920) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-01 12:29:16 +08:00
xige-16 a20770c172
Delete logs that print sensitive information (#20889) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-12-01 10:35:16 +08:00
Enwei Jiao 2ecdb4ba4a
Etcd config source support TLS (#20874) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-30 18:23:15 +08:00
Cai Yudong 4c5ffc832c
Add param check for BruteForceSearch in segcore (#20838) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-11-30 14:53:15 +08:00
Jiquan Long 5a6a92d603
Refine log level (#20896) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-30 14:11:15 +08:00
cai.zhang 83244afcf3
Reduce some logs level which is frequent (#20894) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-30 14:07:15 +08:00
yah01 060649b8aa
Refactor task scheduler and executor (#20828) 
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-30 13:57:15 +08:00
yah01 18762f82aa
Refine TimeRecorder output (#20897) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-30 13:33:15 +08:00
Jiquan Long 8d78d9943a
Fix term filtering failed on indexed scalar field (#20840) (#20882) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-30 10:05:15 +08:00
smellthemoon a46a5ac900
Fix rocksmq incompatible (#20888) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-30 10:01:15 +08:00
SimFG c6fc1511aa
Update the error message for the rabc method (#20740) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-29 20:05:14 +08:00
Ten Thousand Leaves 0700e56008
Increase MaxWatchDuration and make it configurable (#20884) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-29 17:19:14 +08:00
yah01 5ffbb01a67
Fix IndexNode panic while failed to create index (#20861) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-29 15:31:15 +08:00
Jiquan Long 7f76e798e9
Fix metric ProxyApplyPrimaryKeyLatency (#20486) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-29 15:23:14 +08:00
congqixia a8e6d5d47e
Fix delete operation scope to vchannel (#20866) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-29 15:15:15 +08:00
congqixia 938c09679c
Fix LRU cache may panic on double close (#20854) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-29 09:47:14 +08:00
smellthemoon b5f178f22b
Fix kafka panic (#20860) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-28 18:17:14 +08:00
Ten Thousand Leaves 4f9a294b2e
Refine segment stat metrics on QueryNode (#20763) 
issue: #20760

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-28 14:53:14 +08:00
congqixia dd7e309a29
Add log for transfer segment operation (#20829) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-28 09:53:13 +08:00
Enwei Jiao 9b61170bf0
Fix compile proto failed (#20820) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-28 09:47:14 +08:00
smellthemoon d9185485f1
Remove TraceID in mq (#20697) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-25 23:11:12 +08:00
MrPresent-Han b021d7c59c
[skip e2e] add log for querynode load and remove segment(#20510) (#20810) 
issue: #20510

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2022-11-25 17:33:13 +08:00
congqixia 49c8ce7561
Remove etcd dependency for querynode ut (#20830) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-25 14:45:12 +08:00
XuanYang-cn 7fbe0c4df2
Remove mockTestKv in metatable test (#20689) 
See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-25 11:07:12 +08:00
yah01 882ba6240b
Check whether CodecIndex is nil to avoid panic (#20816) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-25 10:45:12 +08:00
shaoyue 37725d3470
[restapi] Fix insert request lost precision for int64 fields (#20468) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-25 10:43:12 +08:00
Ten Thousand Leaves 3cde5b53ad
Fix a panic issue while reporting metrics (#20776) 
issue: #20771

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-24 17:37:13 +08:00
SimFG 44d45452fa
Make the sync segment request idempotent in the data node (#20707) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-24 10:21:13 +08:00
SimFG 6d9d24b4ca
Make memory and etcd data as consistent as possible (#20683) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-24 10:19:12 +08:00
yah01 1eabfdb199
Avoid allocating and coping when fill column with default values (#20721) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-23 15:15:11 +08:00
Enwei Jiao 958e94f6f0
Use Conan as c++ package manager (#19920) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-23 10:39:11 +08:00
congqixia e6c8f9cdee
Add collection delta channel in mutex protection (#20736) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-23 09:49:11 +08:00
Ten Thousand Leaves a8c7199a52
Fix segment_num, stored_rows_num and stored_rows_count metrics (#20704) 
TODO: better track segment with state other than flushed and dropped.

/kind bug

issue: #20146, #20145
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-22 19:21:13 +08:00
XuanYang-cn d1cb380ee0
Enable to gen compaction plan for stale segments (#20741) 
See also: #20502

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-22 17:33:16 +08:00
yah01 f76ea292d2
Change LRU to a generic type (#20744) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-22 12:11:11 +08:00
congqixia e35b523a99
Use embed etcd for allocator ut (#20749) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-22 09:47:16 +08:00
jaime c761de19bc
Fix broken kafka UT (#20743) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-21 20:43:13 +08:00
xige-16 1ad702cc40
Reduce the memory used by load disk index (#20475) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 19:35:11 +08:00
wei liu 67403fcb3b
fix mannual balance with empty segment list (#20738) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-21 19:29:12 +08:00
wayblink b32f99e2e4
Update datanode.segment.insertBufSize default value to 16777216 (#20713) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-11-21 19:19:11 +08:00
jaime d84cda22ad
Fix start fail with kafka model (#20726) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-21 17:49:12 +08:00
xige-16 34c88cea32
Fix GetDiskUsedSize not work (#20474) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 17:47:17 +08:00
xige-16 15badd0263
Fix cache disk index error (#20419) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 17:45:23 +08:00
xige-16 82570e057c
Limit MinDim to build disk index (#20724) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 17:43:10 +08:00
xige-16 81b6703201
Fix spell PQ error (#20725) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 16:35:11 +08:00
xige-16 ec617d4952
Avoid delete segment which has been loaded (#20730) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 16:33:12 +08:00
smellthemoon 8283d32ac4
Only balance segement in targets (#20709) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-21 16:21:11 +08:00
smellthemoon f5ab719f21
timestamp decided if the pks were the same (#20166) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-21 10:55:10 +08:00
MrPresent-Han d44d50e735
fix getting query segment info error during the period of loading and unloading segments (#20549) 
issue: #20281

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2022-11-21 10:41:10 +08:00
groot bf2107ecf5
Fix a regression of local storage (#20653) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-21 10:19:10 +08:00
groot c8022ea63c
Change import task timeout value (#20717) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-21 10:11:11 +08:00
jaime 0a5270e8c0
Fix alter collection hang (#20694) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-21 10:09:10 +08:00
SimFG dffcd974ff
Make sure that the segment path matches the segment id (#20677) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-18 15:35:09 +08:00
Xiaofan 633a749880
Recude IndexCodec Load Memory (#20621) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-18 10:47:08 +08:00
Bingyi Sun 2390095232
Fix load uses compacted segments' binlogs (#20655) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-11-17 20:37:10 +08:00
bigsheeper fef87f879c
Disable tt delay protection (#20657) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-17 20:21:14 +08:00
Enwei Jiao c05b9ad539
Add event dispatcher for config (#20393) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 18:59:09 +08:00
yah01 cc371d6801
Task/Action won't finish util the RPC returned (#20669) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-17 17:55:09 +08:00
Enwei Jiao 19524a5344
Fix nodeID mismatch at standalone mode (#20648) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 17:15:08 +08:00
wei liu 1177ed1a33
fix duplicate watch channel task (#20667) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-17 15:05:08 +08:00
congqixia e6831d5029
Stop querynode for unit tests (#20665) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-17 14:23:08 +08:00
XuanYang-cn bd20536c6a
Reduce unnecessary logs of get compaction results (#20642) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-17 10:09:12 +08:00
groot 8da44638ac
Refine json parser for import (#20545) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-16 19:05:08 +08:00
yah01 31872f436c
Only balance segments in targets (#20635) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-16 14:33:11 +08:00
cai.zhang 35db3d2352
Reduce log frequency in IndexCoord (#20629) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-16 14:23:09 +08:00
congqixia b0a07983ce
Limit concurrency of decoded index file and do force gc (#20587) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-15 21:15:11 +08:00
cai.zhang 3c09a5f2e8
Use GetSegmentUnsafe when get unhealthy segment (#20617) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-15 19:55:07 +08:00
bigsheeper 2d307c1909
Set delBuf's start-end position of compacted segment (#20614) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-15 19:41:16 +08:00
XuanYang-cn 51093dfccd
Add a string func for allocation (#20611) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-15 19:21:12 +08:00
congqixia ac9a993a39
Fix querynode panics when watch/unsub runs concurrently (#20606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-15 19:03:08 +08:00
aoiasd d8f8296b03
Make internal proxy use internal port and isolate internal interface and external interface in proxy grpc server (#20558) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-11-15 16:53:07 +08:00
Ten Thousand Leaves f3ef430c2e
Keep individual segment's updates in a single Etcd operation (#20525) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-15 16:45:07 +08:00
congqixia 35ebd17fe4
Remove not needed RLock for getVersion (#20596) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-15 15:35:08 +08:00
MrPresent-Han c848896bc8
Fix panic error due to rollDeleteOp ahead by load-and-delete (#20563) 
issue:#20501
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2022-11-15 14:45:07 +08:00
bigsheeper db33ffa518
Pass endPosition with vchannel in datanode flowgraph (#20589) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-15 14:37:07 +08:00
zhenshan.cao c5dc076b04
Fix start of DataCoord (#20581) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-11-15 14:19:06 +08:00
yah01 2d806788fb
Add version for subscribed channel (#20585) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-15 13:21:07 +08:00
Ten Thousand Leaves 480a2b8632
Improve compaction logs (#20490) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-15 11:13:07 +08:00
Xiaofan 58ae746f69
Fix GC Tuner Overflow (#20497) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-15 10:43:06 +08:00
XuanYang-cn 3b45189ce9
Refine query logs (#20559) 
See also: #20406

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-15 10:37:08 +08:00
yah01 d37ebe538b
Record load latency for loaded collection (#20538) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-15 10:11:07 +08:00
yah01 bec1d4c50f
Revert "Make proxy process data definition requests concurrently (#20503)" (#20566) 
This reverts commit 911921d5a7.

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-15 00:41:06 +08:00
liliu-z 0ae4e5265f
Optimize some low efficient code (#20529) 
Signed-off-by: Li Liu <li.liu@zilliz.com>

Signed-off-by: Li Liu <li.liu@zilliz.com>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
 2022-11-14 21:05:07 +08:00
bigsheeper e73bdbe17d
Disk quota only checks healthy segments (#20574) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-14 21:03:09 +08:00
smellthemoon a7ba416b5a
Remove invalid timestamp (#20557) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-14 16:21:07 +08:00
yah01 911921d5a7
Make proxy process data definition requests concurrently (#20503) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-14 16:03:07 +08:00
smellthemoon 7325b3e1c3
Substitute traceid for msgid in rpc (#20450) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-14 15:29:06 +08:00
wei liu 6a2e458f90
fix collection not exist cause query coord panic (#20553) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-14 14:51:06 +08:00
Xiaofan af66a0b621
Fix Time Tick session2ChanTs map use the same nodeID (#20537) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-14 14:41:11 +08:00
cai.zhang 2b02b869ec
Return real-time indexed rows and total rows (#20460) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-14 14:37:07 +08:00
groot e217b951ee
Fix bulkinsert bug for int64 value (#20491) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-14 10:15:10 +08:00
bigsheeper fc15789da9
Ensure compatibility of channel seek position and move syncPeriod to config (#20504) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-12 21:09:04 +08:00
congqixia c21497fbb4
Filter fake segments for recovery info (#20493) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-11 18:03:05 +08:00
yah01 a82c235599
Scheduler should returns err if the queue is full (#20465) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-11 17:55:05 +08:00
yah01 1a7912e300
Fix checkFullLoaded always returns all partitions as unloaded (#20514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-11 16:17:04 +08:00
wei liu eaa5cfdcb5
fix leader observer sync logic (#20478) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-11 11:43:06 +08:00
wei liu 701e7b5a6e
support pulsar tenant and auth (#19727) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-11 10:55:05 +08:00
bigsheeper cd19d99ad7
Add channel level checkpoint (#20350) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-10 22:13:04 +08:00
yah01 663bdf3399
Add RootPath field back (#20488) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-10 19:43:03 +08:00
wei liu 7537dbfa37
skip balance on loading collection (#20483) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-10 17:53:04 +08:00
aoiasd fca2b71e28
add access log for proxy (#19927) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-11-10 17:09:06 +08:00
yah01 c71c6378ff
Clear stale replicas (#20456) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-10 17:01:03 +08:00
liliu-z 174310a14e
Use all() api for bitset check (#20462) 
Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>

Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
 2022-11-10 16:15:05 +08:00
yah01 a68cec85af
Refine QueryCoordV2 metrics (#20461) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-10 15:01:04 +08:00
XuanYang-cn 6bff0b6688
Remove too frequent logs (#20454) 
Half of the log messages come from this log if DataNode in stale.

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-10 14:51:05 +08:00
cai.zhang 03c2a280c1
Drop fail jobs on indexnode (#20425) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-10 12:15:03 +08:00
wgcn 864b9d5953
improve search latency (#20426) 
Signed-off-by: wgcn 1026688210@qq.com

Signed-off-by: wgcn <1026688210@qq.com>

Signed-off-by: wgcn <1026688210@qq.com>
 2022-11-10 11:51:03 +08:00
SimFG c08657e079
Remove the unnecessary lock in the shard_cluster (#20414) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-10 10:59:04 +08:00
groot 53850e340c
Dont expire pending import tasks (#20431) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-10 10:25:08 +08:00
liliu-z 0dc659a04c
Upgrade Knowhere to 1.3.3 to fix potential bugs (#20434) 
Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>

Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
 2022-11-09 20:19:04 +08:00
Jiquan Long 449e4f9153
Fix delete log (#20407) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 19:17:04 +08:00
Jiquan Long 0edb49209d
Fix rootcoord metrics (#20427) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 17:49:04 +08:00
Jiquan Long 854fb6aad2
Fix querycoord metrics (#20429) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 17:45:04 +08:00
Jiquan Long cc9dc0f0fa
Fix log of RootCoord (#20412) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 15:07:03 +08:00
Jiquan Long 2189530f5b
Fix some unreasonable logs in Proxy (#20411) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 15:05:08 +08:00
groot 39a6397043
Fix an error message (#20403) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-09 14:49:03 +08:00
jaime e5419b3bc7
Remove kafka consumer ack (#20424) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-09 14:27:03 +08:00
Jiquan Long a2bf5022cb
Fix log of alter collection (#20413) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-08 22:51:03 +08:00
Jiquan Long 6886cc3fd8
Fix rpc log of Proxy (#20409) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-08 21:59:02 +08:00
SimFG 2aba154ad3
Make the subName unique for the datanode tt channel (#20331) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-08 20:15:04 +08:00
SimFG 9c6436d72d
Add time log for methods of starting the node (#20313) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-08 20:13:03 +08:00
wei liu f925fa7661
fix check segment stale task (#20401) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-08 19:43:05 +08:00
cai.zhang 889eb02172
Add goroutine pool for creating index (#20391) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-08 19:41:02 +08:00
SimFG 6e9820441f
Close the no-use chan in the querycoord node (#20385) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-08 15:55:02 +08:00
Ten Thousand Leaves 3d25a8dc03
Improve panic error readability (#20339) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-08 15:37:03 +08:00
Enwei Jiao 7e56e7a976
Fix datarace at Setlogger (#20376) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-08 14:33:03 +08:00
shaoyue 09ea38615e
Fix gcp oauth token not cached (#20380) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-08 12:43:02 +08:00
congqixia ad0cce8f70
Filter segments when doing FromDmlCPLoadDelete (#20388) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-08 11:47:02 +08:00
wayblink a7bed1c927
Pass backup flag through Import request options (#20334) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-11-08 11:33:03 +08:00
wei liu c5cd92d36e
update target (#19296) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-07 19:37:04 +08:00
MrPresent-Han 21b54709a2
enable delbuf auto flush function to avoid OOM when del msg accumulating (#20213) 
issue: #19713

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: MrPresent-Han <zilliz@zillizdeMacBook-Pro-2.local>
 2022-11-07 18:49:02 +08:00
groot b847c425e1
Fix listImportTasks bug (#20301) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-07 17:11:02 +08:00
congqixia 6e4509d0fc
Fix version not deleted if expired by loadSegment (#20361) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-07 17:07:03 +08:00
wei liu d7ebb25701
skip handoff event on unloaded partition (#20306) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-07 17:05:06 +08:00
congqixia 14c1182418
Add healthy check for InvalidateCollectionMetaCache (#20368) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-07 16:55:01 +08:00
yah01 4112c667d0
Refine task's priority (#20364) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-07 14:53:02 +08:00
cai.zhang 8ea8f91334
Make index log better (#20327) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-07 14:23:02 +08:00
Xiaofan 0d18336b21
Fix GCTuner calculation overflow (#20354) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-07 12:17:05 +08:00
jaime 709a0a94e9
Add some msgstream metrics (#20296) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-07 10:15:02 +08:00
xige-16 4a66965df4
Delete RAW_DATA copy when load IVF_FLAT index data (#20274) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-05 17:33:05 +08:00
Xiaofan ffd52f0d08
Add GC Tuner (#20321) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-04 22:25:01 +08:00
congqixia 31c22a9672
Remove KeyLock from querynode task (#20344) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-04 20:31:01 +08:00
Enwei Jiao 956c5e1b9d
Make Params singleton (#20088) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-04 14:25:38 +08:00
zhagnlu 15b2d1cace
Change default thread core coefficient (#20120) (#20328) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-04 12:41:35 +08:00
congqixia f557af1143
Verify target id for FlushSegments request in datanode (#20308) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-03 22:47:35 +08:00
SimFG deb9963d0e
Delete unused messages for the mq (#20295) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-03 21:41:35 +08:00
yah01 e0506352a4
Fix search failed with nil result (#20304) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-03 21:15:35 +08:00
zhagnlu 39212bc04c
Fix get cpu num for segcore (#20120) (#20307) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-03 19:29:36 +08:00
cai.zhang 6c4658db22
Don't build BIN_FLAT index really (#20309) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-03 18:23:37 +08:00
Jiquan Long e7103310b8
Fix log when building index (#20302) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-03 18:19:35 +08:00
bigsheeper d528aa3d90
Print seek postion time and seek elapse (#20271) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-03 15:03:35 +08:00
yah01 c08f11420f
Fix Release tasks may end with incorrect status (#20263) 
If the resource has been removed, the release tasks should end with
Succeeded status, not stale

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-03 15:01:35 +08:00
XuanYang-cn 22fb1d3b93
Remove dup mock_kv.go and mock_txn_kv.go (#20267) 
Replace them with mockery generated mocks in kv/mocks

See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-03 14:41:35 +08:00
zhagnlu cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-03 14:39:40 +08:00
MrPresent-Han e0baf1536e
[skip e2e] Improve a commment for initial pr (#20287) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2022-11-03 14:05:36 +08:00
yah01 5143df1809
Fix Proxy never updates the shard leaders cache (#20286) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-03 12:27:36 +08:00
Xiaofan 766957e75a
Recuce memory consumption in MinioChunkManager (#20284) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-03 12:19:35 +08:00
Ten Thousand Leaves 5708352a80
Improve DataCoord and DataNode logs (#20265) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-03 12:15:35 +08:00
Ten Thousand Leaves 9751bec04c
Split large Etcd operations while altering segment info (#20278) 
/kind bug

issue: #20275
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-03 11:01:35 +08:00
congqixia e1e3ed486e
Fix rmq consumer cancel may block forever (#20269) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-02 19:01:35 +08:00
jaime b7ee057d6b
Fix querynode panic during stop (#20242) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-02 14:59:34 +08:00
bigsheeper 51acba7d99
Add role name to ttDelay metric (#20239) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-02 10:55:34 +08:00
groot cd3b3d3da6
Only allow one file for row-based import (#20234) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-02 10:23:35 +08:00
SimFG 56605980df
Make sure the query node isn't created repeatedly in concurrent situation (#20197) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-02 09:37:34 +08:00
bigsheeper 29f2ed67a9
Release delta flowgraph if loadSegment failed (#20238) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-01 21:31:35 +08:00
Bingyi Sun 8d47356aa9
Use compaction generated segment's checkpoint as compacted segment's seekPosition (#20240) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-11-01 21:01:34 +08:00
Xiaofan d17f9de343
revert concurrent segment load (#20244) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 20:33:35 +08:00
Xiaofan 33349677fc
Fix unsafe lock (#20219) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 19:37:35 +08:00
Xiaofan ded919a7a3
Refine Concurrent Load (#20220) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 19:25:37 +08:00
Xiaofan 4f6b87311e
Refine request merger (#20189) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 14:27:34 +08:00
yah01 0b939c5735
Check whether ubsubscribe/release task is stale (#20165) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-01 12:33:35 +08:00
shaoyue 9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164) 
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-01 11:07:35 +08:00
bigsheeper a2d5947fe4
Change limiter burst type to float64 (#20215) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-31 20:59:34 +08:00
Xiaofan 99b958e360
Fix BF Concurrency issue (#20211) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 19:27:35 +08:00
congqixia 5f5fdb0789
Compaction bypass stale segments (#20202) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-31 19:13:34 +08:00
wei liu 9293524de4
remove outdated handoff event (#20181) 
Signed-off-by: wei.liu <wei.liu@zilliz.com>

Signed-off-by: wei.liu <wei.liu@zilliz.com>
 2022-10-31 19:11:40 +08:00
congqixia 4c4db1f556
Display all shard error if search/query fails (#20169) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-31 19:09:39 +08:00
Xiaofan 2bfecf5b4e
Refine bloomfilter and memory usage (#20168) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 17:41:34 +08:00
groot c6151ad351
Parse utf32 string of numpy file (#20176) 
Signed-off-by: groot <yihua.mo@zilliz.com>

Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-10-31 17:39:34 +08:00
xige-16 05a966afbe
Delete row num check before load (#20205) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-31 17:17:34 +08:00
bigsheeper 37dce8a660
Make s3 client creation thread safe, prevent crashing (#20200) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-31 17:09:33 +08:00
jaime 48f5c60070
Refine msgstream close (#20194) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-31 15:13:33 +08:00
Xiaofan 4136009a9a
Support load segments/channels in parallel (#20036) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 13:55:33 +08:00
wgcn bf5fb3cd99
improve show collection (#20124) 
Signed-off-by: wgcn <1026688210@qq.com>
 2022-10-31 13:25:35 +08:00
cai.zhang 61775122ea
Don't expose index params when error occurred (#20187) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-31 12:11:33 +08:00
Bingyi Sun 2ff21f0e10
Fix querycoord stop panic (#20155) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2022-10-31 12:05:34 +08:00
cai.zhang bcffb3c389
Check indexes in indexcoord instead of proxy (#20193) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-31 11:39:33 +08:00
SimFG c604d2f933
Make param names clearer in the `policy.go` (#20136) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-31 11:35:34 +08:00
Jiquan Long 8d0cc4226c
Fix IVF_SQ nbits check (#20183) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-31 10:13:34 +08:00
congqixia a8e2a69aa2
Fix mergeSyncTask logic (#20185) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-29 18:47:33 +08:00
congqixia 19300bd36f
Remove delta vchannel when clean up empty flowgraph (#20182) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-29 18:31:33 +08:00
jaime 19baecdc4d
[skip e2e]Refine error msg for authentication checking (#20117) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-28 19:25:34 +08:00
bigsheeper bddd47c60f
Prevent limiter burst overflow (#20153) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-28 19:05:32 +08:00
wei liu 4412cfcaaf
reduce querycoord unnecessary panic (#19925) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-10-28 17:15:32 +08:00
cai.zhang 53289a63e6
Specification log (#20119) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-28 17:01:32 +08:00
cai.zhang 26d26aa8ab
Set index name in proxy when drop index (#20160) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-28 16:41:32 +08:00
yah01 8bfa55e560
Fix memory & goroutine leak (#20152) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-28 14:55:32 +08:00
Jiquan Long 89541a6f68
Revert parallel ddl (#20118) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-28 13:25:34 +08:00
congqixia 6224b3c44c
Fix lastVersion not expire when there is only loadSegmentsTask (#20156) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-28 13:05:31 +08:00
XuanYang-cn d5bc4e2585
Make DN able to process syncing request in batch (#20126) 
See also: #19834

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-27 21:25:33 +08:00
yah01 404fc68afa
Fix dead locks (#20121) 
the collection's lock and meta replica's lock

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-27 20:53:32 +08:00
cai.zhang 1be4d1e267
Fill the WriteHandoff field in the upgrade script (#20139) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-27 20:25:35 +08:00
cai.zhang 4848edda8d
Append field indexID for index name (#20046) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-27 20:23:38 +08:00
SimFG a90e28cf45
Fix `sessions` concurrency problem (#20140) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-27 19:39:31 +08:00
Xiaofan d00028edba
Mark segment as fake in flush path (#20141) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-27 19:29:33 +08:00
congqixia 1e590d787f
Release last version after expiration (#20137) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-27 19:15:32 +08:00
SimFG 729e9d48fe
Improve the `mockProxyComponent` methods' receiver (#20115) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-27 18:47:32 +08:00
bigsheeper b949fa0ea9
Increase default ttProtection max delay (#20125) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-27 17:37:32 +08:00
Xiaofan 11efa0bb5f
Fix compacted segment not dropped (#20128) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-27 17:15:32 +08:00
groot bee66631e3
Refine bulkload (#19671) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-10-27 16:21:34 +08:00
Xiaofan def5972e01
fix duplicated handoff request (#20108) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-27 15:15:32 +08:00
aoiasd e470cd3dfa
Move Rocksmq page info log to Rokcsmq monitor (#20022) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-10-27 15:13:31 +08:00
bigsheeper b074f530e6
Forbid createIndex if collection loaded before (#20100) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-27 13:05:31 +08:00
Jiquan Long 3ff0112e49
Fix querynode panic occasionally (#20106) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-27 10:05:31 +08:00
xige-16 14968685a2
Limit disk index search list range (#20104) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-26 21:03:31 +08:00
jaime 976e9940bf
Add some logs for handoff observer (#20097) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-26 19:37:31 +08:00
bigsheeper 51e50bba73
Use Wlock when release segment (#20092) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-26 18:59:32 +08:00
XuanYang-cn c24244240e
Fix insert search concurrency problem (#20065) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-26 18:57:36 +08:00
congqixia d525cacb8d
Remove empty delta flow graph (#20076) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-26 17:17:31 +08:00
jaime d8ca56e084
Fix empty segment handoff fails (#20073) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-26 16:49:31 +08:00
xige-16 5bb2626466
Set diskUsageRatio to 4.0 when build disk index (#20066) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-26 16:11:31 +08:00
xige-16 89a9b6fda0
Init storage config's SSL and IAM config (#20082) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-26 16:07:30 +08:00
congqixia 27ebb766e7
Clean BF for compacted segment in datanode (#20085) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-26 14:35:31 +08:00
yah01 5429a973b4
Fix forget to fill the channel name (#20070) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-25 21:49:32 +08:00
bigsheeper f0c616050a
Reduce and improve quota center log (#20058) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-25 19:39:30 +08:00
Xiaofan 192151bc11
Refine Data Log (#19996) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 19:31:30 +08:00
Xiaofan 3d316fc052
Fix Query Log Level (#19995) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 19:29:36 +08:00
XuanYang-cn 0a2436fbe7
Remove meaningless logs (#20051) 
Fixes: #20050

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-25 17:53:32 +08:00
cai.zhang a60db370e6
Compatibility with handling index params (#19997) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 16:59:30 +08:00
Jiquan Long 290b902ab0
Fix rootcoord too frequent useless log (#20044) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 14:19:30 +08:00
Jiquan Long 377cd210c2
Make knowhere use segcore's cmake build type (#20041) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 14:17:37 +08:00
xige-16 32ba4fafb8
Delete Creating bucket when create core/minioChunkManager (#20049) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-25 14:15:30 +08:00
Xiaofan 1f170ce1ba
Refine kafka consumer (#19846) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 13:23:30 +08:00
cai.zhang 85ef7331ee
Set timeout for rpc (#19535) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 11:55:30 +08:00
cai.zhang c551de8f72
Catch errors on gloabl meta cache (#20023) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 11:29:30 +08:00
cai.zhang e7d58b4fb7
Set index state as retry when error occurred (#19978) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 11:09:30 +08:00
Jiquan Long db4ede217f
Fix index not exist (#20027) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 10:01:30 +08:00
Jiquan Long 7a13b1c329
Fix panic when stop rootcoord (#20019) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 09:51:30 +08:00
smellthemoon 2f19e6595a
Fill SourceID (#20000) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-10-24 23:43:30 +08:00
yah01 aa6b8af80d
Migrate metadata of QueryCoord from 2.1 to 2.2 (#19986) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-24 23:41:35 +08:00
xige-16 c8ecb81c04
Return IndexInfo if index build done (#19976) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-24 17:17:30 +08:00
Jiquan Long 14ddfe679f
Fix failed to show partitions (#20007) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-24 15:57:29 +08:00
Xiaofan 1b5e765307
Block Flowgraph before handle compacted segments (#19988) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-24 15:09:29 +08:00
congqixia 454ff2f1df
Fix ctx confusion caused goroutine leakage (#20017) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-24 14:31:29 +08:00
Xiaofan 868bdd504d
Fix flaky unit test (#20006) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-24 11:51:29 +08:00
XuanYang-cn eaae3aa49e
Fix metric recording auto flush segments twice (#19967) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-24 10:51:30 +08:00
Enwei Jiao 4e5d1c2e5f
Add log to debug bitset (#19982) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-10-24 10:09:28 +08:00
Jiquan Long 308a30fc46
Make dropping collection atomic (#19969) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-22 15:19:28 +08:00
congqixia 2e6ddd7f2a
Add datanode&channel match check for Flush (#19985) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-22 12:09:28 +08:00
Xiaofan 7d2ba60412
If autoID enabled, check to make sure user didn't specify autoID field (#19943) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-21 19:03:27 +08:00
zhagnlu 05bbb12d30
Skip build aws sdk when not need diskann (#19965) (#19966) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-10-21 18:35:29 +08:00
Jiquan Long a1a74e3891
Optimize ddl case when collection not exist (#19950) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-21 16:37:29 +08:00
liliu-z ce8e4d17b3
Fix mis-copy of chunk in make_with_timestamps (#19970) 
Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>

Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
 2022-10-21 16:35:29 +08:00
smellthemoon 581e1d8c35
Fill MsgBase ALL (#19912) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-10-21 15:57:28 +08:00
cai.zhang 94c15a49e9
Can't drop loaded partition (#19938) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-21 14:41:28 +08:00
yah01 a4fb3cc3e2
Fix panic while applying deletions (#19956) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-21 14:25:29 +08:00
Xiaofan 3d3289ce3e
Fix collection metrics leakage (#19944) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-21 10:13:28 +08:00
yah01 e8ca754dca
Fix compaction produces empty segment (#19936) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 22:49:28 +08:00
xige-16 36accde034
Fix wrong msg when load failed (#19884) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-20 19:55:27 +08:00
jaime cf1491e41a
Add status for GetVersionResponse and CheckHealthResponse (#19895) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-20 19:47:33 +08:00
congqixia e8cf049392
Fix proxy trace operation name (#19932) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 18:35:28 +08:00
XuanYang-cn e081eb287d
Refine flowgraph nodes for readability (#19864) 
- delete compaction tasks from executor while dropping

See also: #19780

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-20 16:39:29 +08:00
congqixia 3de74165ac
Fix SyncSegment generate source (#19929) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 16:35:28 +08:00
congqixia b15e97a61a
Improve proxy leader cache lock logic (#19917) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 16:11:27 +08:00
aoiasd 7fe8e50689
print some group info in rocksmq monitor (#19863) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-10-20 16:05:27 +08:00
congqixia 0095869d5b
Keep segment statslog if no insert applied (#19910) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 14:33:27 +08:00
yah01 96cf74d53a
Add more panic info (#19922) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 13:33:27 +08:00
wayblink dbd60757c9
Support pass start_ts and end_ts through Import API (#19810) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-20 12:15:27 +08:00
yah01 470e8ef419
Store index ID to filter out invalid handoff event (#19703) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 11:29:27 +08:00
zhagnlu 02a8a07322
Fix BucketExists bug in minio chunk manager (#19548) (#19854) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-10-20 10:23:27 +08:00
yah01 5dcff9ab72
Fix QueryNode panic when meta changed (#19902) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 10:07:30 +08:00
cai.zhang 836081c8c7
Dropping index must happen without collection being loaded (#19886) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-19 23:17:44 +08:00
bigsheeper db0486c870
Fix missing import limit and rename functions in quotaCenter (#19907) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-19 21:43:30 +08:00
wei liu 87248dc1a9
fix clean handoff event (#19903) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-10-19 20:41:26 +08:00
yah01 52cd40fbc4
Remove codes related to deprecated field (#19897) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 18:29:26 +08:00
Jiquan Long 579c50fa1a
Fix io pool goroutine leakage (#19892) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-19 18:23:27 +08:00
jaime b79687687d
Replace segment index file path with index file id within segment index meta (#19866) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-19 16:55:27 +08:00
bigsheeper 99df2d9b54
Add binlogSize and ttDelay metrics monitor (#19872) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-19 15:13:26 +08:00
Ten Thousand Leaves 762045adff
Fix an issue where working task might get cleaned up (#19888) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-19 13:23:28 +08:00
congqixia b361842e9a
Add log for querynode checkpoint timestamp lag (#19883) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-19 13:21:31 +08:00
yah01 7cf26011e6
Add more logs for checkpoint of channel (#19885) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 13:19:27 +08:00
yah01 c61a4d1243
Fix panic due to double stop (#19849) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 12:13:28 +08:00
smellthemoon 108e51b2f0
[test]Create index before load and fix error message (#19874) (#19857) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Co-authored-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2022-10-19 10:01:26 +08:00
congqixia 48463a12ef
Fix querynode concurrent map rw panic (#19867) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-18 19:47:36 +08:00
Xiaofan 5b9988b627
Support collection level monitoring metrics (#19686) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-18 19:17:27 +08:00
wayblink 4cc57adc5d
Initiatively release session in etcd when service close (#19825) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-18 19:13:26 +08:00
cai.zhang b3f6b67977
Alloc timestamp instead of using start position timestamp for segment index (#19820) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-18 17:05:26 +08:00
bigsheeper 15f7592739
Intercept import request in Proxy interceptor (#19856) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-18 16:53:26 +08:00
XuanYang-cn 2785f98de6
Refactor segment replica in DN (#19815) 
- Replace isNew and isFlushed with sType
- Add compacted into proto
- Rename replica to channel
- Rename segmentReplica to channelMeta
- Replace 4 maps with 1 map
- Add segment.go and segment_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-18 15:33:26 +08:00
congqixia a1cdc55bcb
Make newSegment transfer state after SaveBinlogPath succuess (#19858) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-18 14:07:25 +08:00
jaime 59bcbf0cf6
Support check health for the service level (#19779) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-18 13:39:26 +08:00
Ten Thousand Leaves db33f401fd
Fix proxy crash issue when meta cache miss (#19844) 
issue: #19557

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-18 13:11:26 +08:00
Ten Thousand Leaves a4a6f60279
Refine NQ check logic (#19819) 
issue: #19682

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-18 13:07:34 +08:00
Enwei Jiao 0a83cdfe1b
Refactor grpclient, make it generic (#19791) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-10-17 20:05:26 +08:00
yah01 b0404cbbab
Remove dropped segments' info (#19839) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-17 19:41:26 +08:00
yah01 17ce0c3979
Refine the error of QueryNode (#19836) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-17 19:39:26 +08:00
congqixia a3f84f2ea3
Remove dependency for dropped segment checkpoint in querynode (#19831) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-17 18:47:25 +08:00
cai.zhang 9d43947f1c
Must create index before load (#19516) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-17 18:01:25 +08:00
bigsheeper 760e9c5df2
Enable rate limit and update quota unit (#19816) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-17 16:23:25 +08:00
Jiquan Long b0a12116c8
Fix meta migration tool (#19814) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-17 15:07:25 +08:00
xige-16 158787811e
Move assemble/disassemble func to core (#19420) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-16 21:17:25 +08:00
cai.zhang 9e8a59ac4c
Add nameing rules for index name (#19803) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-16 21:05:25 +08:00
SimFG a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
bigsheeper a92b242ecd
Fix ttFactor in QuoterCenter (#19802) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-16 09:17:24 +08:00
yah01 89c9cb3680
Fix scan may break GC limitation (#19670) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-15 15:13:24 +08:00
SimFG 3d12325796
Gracefully handle the hook plugin loading failure error (#19789) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-14 19:59:24 +08:00
bigsheeper 3a5aaeb7ad
Fix read queue metrics and memory protection (#19787) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-14 18:05:24 +08:00
xige-16 a1db9038fb
Move disk index params to config file (#19714) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-14 17:51:24 +08:00
bigsheeper bdaddd7b09
Revert DML quota check (#19790) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-14 17:17:23 +08:00
Ten Thousand Leaves 8b8df0a5e9
Do not use DataCoord context when DataNode is handling import task (#19732) 
So that when DataCoord is done, DataNode can still proceed.

/kind bug

issue: #19730
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-14 15:15:24 +08:00
Jiquan Long cb51d410a2
Limit max shard num to 64 (#19776) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-14 14:51:23 +08:00
xige-16 8c9c1672ae
Assign different storage config for indexes (#19517) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-14 14:45:23 +08:00
congqixia 089912baa7
Fix segcore unit test name (#19783) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-14 14:29:24 +08:00
bigsheeper 22b32465e7
Make bulkLoad failed if quota exceeds and remove autoindex config (#19770) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-14 14:27:25 +08:00
Enwei Jiao 82dd1bba1c
Refactor healthz API (#19747) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-10-14 14:19:24 +08:00
congqixia a20d83bccd
Assemble RAW_DATA files before storing into index.raw_data (#19771) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-14 12:03:24 +08:00
jaime 8e7a4733ec
Add GetVersion API for the client (#19694) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-14 09:59:24 +08:00
Ten Thousand Leaves be4b0189b1
Fix potential long log (#19757) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-13 19:21:23 +08:00
SimFG 5091a7f362
Add the `TotalRows` value for the DescribeIndex (#19759) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-13 17:43:24 +08:00
wayblink fb14f4cc7e
Add active-standby support in querycoord V2 (#19748) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-13 16:51:24 +08:00
bigsheeper efdce464f5
Fix ttProtection.enabled and memProtection (#19756) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-13 16:15:25 +08:00
yah01 146a3342c3
Use task's ID as the MsgID instead of source ID (#19751) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-13 15:21:23 +08:00
bigsheeper f1e330a997
Add read result rate protection (#19728) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-13 14:57:24 +08:00
jaime ab88dd77e7
Remove unnecessary log for datacoord gc (#19698) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-13 14:43:23 +08:00
Enwei Jiao 4df3552f74
Prepare http admin API (#19676) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-10-13 10:37:23 +08:00
Ten Thousand Leaves ae373d450f
Make NQ <= 16384 limit work (#19726) 
https://milvus.io/docs/limitations.md

issue: #19682

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-12 18:37:23 +08:00
congqixia 04e1333552
Fix insert buffer node time range uint64 conversion error (#19725) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-12 17:19:23 +08:00
mumon b48c37cb44
Fix typo in core (#19661) 
Signed-off-by: Ziyu Wang <15871035978@163.com>

Signed-off-by: Ziyu Wang <15871035978@163.com>
 2022-10-12 17:05:23 +08:00
XuanYang-cn d73186b2eb
Fix timetravel error msg (#19722) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-12 15:17:23 +08:00
Jiquan Long a8a074162f
Add meta migration tool (#19709) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-12 11:37:23 +08:00
groot a561614d48
Fix bulkload failed bug (#19702) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-10-12 10:27:23 +08:00
bigsheeper e6c76aba0e
Force deny writing if disk quota exceeded (#19680) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-12 10:03:22 +08:00
aoiasd 0a77561e78
Fix some message in rocksmq could not get acked timestamp (#19592) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-10-11 23:17:23 +08:00
congqixia 2a2d32c4fc
Fix datacoord panics when collection info not found (#19706) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-11 21:39:24 +08:00
jaime 10c03de628
Refine method name from BroadCastXx to BroadcastXx (#19700) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-11 21:07:23 +08:00
Eng Zer Jun 438790744d
test: use `T.Setenv` to set env vars in tests (#19649) 
This commit replaces `os.Setenv` with `t.Setenv` in tests. The
environment variable is automatically restored to its original value
when the test and all its subtests complete.

Reference: https://pkg.go.dev/testing#T.Setenv
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2022-10-11 18:43:23 +08:00
Enwei Jiao 667982a239
Fix static-check (#19707) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-10-11 17:59:22 +08:00
XuanYang-cn 28e949cf31
Add GetDataVChanPositions (#19657) 
The original GetVChanPositions are specialized for QueryCoord,
the recovery info needed by DataNode shouldn't take `indexed` state
into account.

This PR splits GetVChanPositions into GetDataVChanPositions and
GetQueryVChanPositions.

See also: #19653

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-11 16:51:22 +08:00
SimFG 125f827ca4
Use underscores to separate parameter names in the `GetLoadingProgressRequest` (#19692) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-11 11:55:22 +08:00
yah01 1c71844b8d
Add license header (#19678) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-11 11:39:22 +08:00
yah01 2925d8a1ac
Remove QueryCoordV1 (#19681) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-11 11:11:23 +08:00
congqixia 68b9a2be23
Perform health check after release cluster segment (#19684) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-11 09:17:22 +08:00
Xiaofan db3f4be49d
Simplify flow graph node to pipeline mode (#19667) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-10 22:15:22 +08:00
jaime 31db01b6ae
Support TTL for collection level (#19383) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-10 20:31:22 +08:00
wayblink 85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface (#19594) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-10 15:55:22 +08:00
XuanYang-cn 2c92ce0c0e
Prevent datacoord from syncing unflushed segments (#19659) 
See also: #19653

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-10 11:33:21 +08:00
wayblink 3049dbb5a5
Fix exclusive consumer conflict in import (#19632) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-10 10:49:21 +08:00
Ten Thousand Leaves b9feeab15f
Do not try to report task state when DataNode itself is down (#19647) 
/kind bug

issue: #19502
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-10 09:45:21 +08:00
congqixia d64f318a80
Add collection schema in watch channel info (#19663) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-10 01:37:21 +08:00
XuanYang-cn b6ba226059
Fix pagination check in QueryNode (#19497) 
See also: #19482

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-09 16:26:57 +08:00
Ten Thousand Leaves b24e97e6a8
Improve some DataNode service logs (#19648) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-09 15:54:58 +08:00
xige-16 49daee2e17
Fix lost disk index data (#19636) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-09 14:36:58 +08:00
bigsheeper f98dbcf5be
Update quota params and disable limit for showCollection (#19641) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-09 14:30:58 +08:00
groot 746e9ece5a
Fix a random failure of unittest (#19643) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-10-09 14:22:57 +08:00
Jiquan Long bdd6171231
Make creating collection/partition follow tt mechanism (#19628) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-09 12:32:57 +08:00
Cai Yudong 246693acb7
Fix panic when expr invalid (#19642) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-10-09 11:48:57 +08:00
Xiaofan 6826964e60
Rename some interface and files in id allocator (#19605) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-09 10:06:58 +08:00