milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,093 Commits
137 Branches
115 Tags
572 MiB
Commit Graph

6127 Commits (8283d32ac4f9a164b8d3c7a723528781f79a1420)

Author SHA1 Message Date
yah01 e0506352a4
Fix search failed with nil result (#20304) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-03 21:15:35 +08:00
zhagnlu 39212bc04c
Fix get cpu num for segcore (#20120) (#20307) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-03 19:29:36 +08:00
cai.zhang 6c4658db22
Don't build BIN_FLAT index really (#20309) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-11-03 18:23:37 +08:00
Jiquan Long e7103310b8
Fix log when building index (#20302) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-03 18:19:35 +08:00
bigsheeper d528aa3d90
Print seek postion time and seek elapse (#20271) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-03 15:03:35 +08:00
yah01 c08f11420f
Fix Release tasks may end with incorrect status (#20263) 
If the resource has been removed, the release tasks should end with
Succeeded status, not stale

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-03 15:01:35 +08:00
XuanYang-cn 22fb1d3b93
Remove dup mock_kv.go and mock_txn_kv.go (#20267) 
Replace them with mockery generated mocks in kv/mocks

See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-03 14:41:35 +08:00
zhagnlu cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-03 14:39:40 +08:00
MrPresent-Han e0baf1536e
[skip e2e] Improve a commment for initial pr (#20287) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2022-11-03 14:05:36 +08:00
yah01 5143df1809
Fix Proxy never updates the shard leaders cache (#20286) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-03 12:27:36 +08:00
Xiaofan 766957e75a
Recuce memory consumption in MinioChunkManager (#20284) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-03 12:19:35 +08:00
Ten Thousand Leaves 5708352a80
Improve DataCoord and DataNode logs (#20265) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-03 12:15:35 +08:00
Ten Thousand Leaves 9751bec04c
Split large Etcd operations while altering segment info (#20278) 
/kind bug

issue: #20275
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-03 11:01:35 +08:00
congqixia e1e3ed486e
Fix rmq consumer cancel may block forever (#20269) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-02 19:01:35 +08:00
jaime b7ee057d6b
Fix querynode panic during stop (#20242) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-02 14:59:34 +08:00
bigsheeper 51acba7d99
Add role name to ttDelay metric (#20239) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-02 10:55:34 +08:00
groot cd3b3d3da6
Only allow one file for row-based import (#20234) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-02 10:23:35 +08:00
SimFG 56605980df
Make sure the query node isn't created repeatedly in concurrent situation (#20197) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-02 09:37:34 +08:00
bigsheeper 29f2ed67a9
Release delta flowgraph if loadSegment failed (#20238) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-01 21:31:35 +08:00
Bingyi Sun 8d47356aa9
Use compaction generated segment's checkpoint as compacted segment's seekPosition (#20240) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-11-01 21:01:34 +08:00
Xiaofan d17f9de343
revert concurrent segment load (#20244) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 20:33:35 +08:00
Xiaofan 33349677fc
Fix unsafe lock (#20219) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 19:37:35 +08:00
Xiaofan ded919a7a3
Refine Concurrent Load (#20220) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 19:25:37 +08:00
Xiaofan 4f6b87311e
Refine request merger (#20189) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-01 14:27:34 +08:00
yah01 0b939c5735
Check whether ubsubscribe/release task is stale (#20165) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-01 12:33:35 +08:00
shaoyue 9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164) 
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-01 11:07:35 +08:00
bigsheeper a2d5947fe4
Change limiter burst type to float64 (#20215) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-31 20:59:34 +08:00
Xiaofan 99b958e360
Fix BF Concurrency issue (#20211) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 19:27:35 +08:00
congqixia 5f5fdb0789
Compaction bypass stale segments (#20202) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-31 19:13:34 +08:00
wei liu 9293524de4
remove outdated handoff event (#20181) 
Signed-off-by: wei.liu <wei.liu@zilliz.com>

Signed-off-by: wei.liu <wei.liu@zilliz.com>
 2022-10-31 19:11:40 +08:00
congqixia 4c4db1f556
Display all shard error if search/query fails (#20169) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-31 19:09:39 +08:00
Xiaofan 2bfecf5b4e
Refine bloomfilter and memory usage (#20168) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 17:41:34 +08:00
groot c6151ad351
Parse utf32 string of numpy file (#20176) 
Signed-off-by: groot <yihua.mo@zilliz.com>

Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-10-31 17:39:34 +08:00
xige-16 05a966afbe
Delete row num check before load (#20205) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-31 17:17:34 +08:00
bigsheeper 37dce8a660
Make s3 client creation thread safe, prevent crashing (#20200) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-31 17:09:33 +08:00
jaime 48f5c60070
Refine msgstream close (#20194) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-31 15:13:33 +08:00
Xiaofan 4136009a9a
Support load segments/channels in parallel (#20036) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 13:55:33 +08:00
wgcn bf5fb3cd99
improve show collection (#20124) 
Signed-off-by: wgcn <1026688210@qq.com>
 2022-10-31 13:25:35 +08:00
cai.zhang 61775122ea
Don't expose index params when error occurred (#20187) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-31 12:11:33 +08:00
Bingyi Sun 2ff21f0e10
Fix querycoord stop panic (#20155) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2022-10-31 12:05:34 +08:00
cai.zhang bcffb3c389
Check indexes in indexcoord instead of proxy (#20193) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-31 11:39:33 +08:00
SimFG c604d2f933
Make param names clearer in the `policy.go` (#20136) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-31 11:35:34 +08:00
Jiquan Long 8d0cc4226c
Fix IVF_SQ nbits check (#20183) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-31 10:13:34 +08:00
congqixia a8e2a69aa2
Fix mergeSyncTask logic (#20185) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-29 18:47:33 +08:00
congqixia 19300bd36f
Remove delta vchannel when clean up empty flowgraph (#20182) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-29 18:31:33 +08:00
jaime 19baecdc4d
[skip e2e]Refine error msg for authentication checking (#20117) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-28 19:25:34 +08:00
bigsheeper bddd47c60f
Prevent limiter burst overflow (#20153) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-28 19:05:32 +08:00
wei liu 4412cfcaaf
reduce querycoord unnecessary panic (#19925) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-10-28 17:15:32 +08:00
cai.zhang 53289a63e6
Specification log (#20119) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-28 17:01:32 +08:00
cai.zhang 26d26aa8ab
Set index name in proxy when drop index (#20160) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-28 16:41:32 +08:00
yah01 8bfa55e560
Fix memory & goroutine leak (#20152) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-28 14:55:32 +08:00
Jiquan Long 89541a6f68
Revert parallel ddl (#20118) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-28 13:25:34 +08:00
congqixia 6224b3c44c
Fix lastVersion not expire when there is only loadSegmentsTask (#20156) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-28 13:05:31 +08:00
XuanYang-cn d5bc4e2585
Make DN able to process syncing request in batch (#20126) 
See also: #19834

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-27 21:25:33 +08:00
yah01 404fc68afa
Fix dead locks (#20121) 
the collection's lock and meta replica's lock

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-27 20:53:32 +08:00
cai.zhang 1be4d1e267
Fill the WriteHandoff field in the upgrade script (#20139) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-27 20:25:35 +08:00
cai.zhang 4848edda8d
Append field indexID for index name (#20046) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-27 20:23:38 +08:00
SimFG a90e28cf45
Fix `sessions` concurrency problem (#20140) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-27 19:39:31 +08:00
Xiaofan d00028edba
Mark segment as fake in flush path (#20141) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-27 19:29:33 +08:00
congqixia 1e590d787f
Release last version after expiration (#20137) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-27 19:15:32 +08:00
SimFG 729e9d48fe
Improve the `mockProxyComponent` methods' receiver (#20115) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-27 18:47:32 +08:00
bigsheeper b949fa0ea9
Increase default ttProtection max delay (#20125) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-27 17:37:32 +08:00
Xiaofan 11efa0bb5f
Fix compacted segment not dropped (#20128) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-27 17:15:32 +08:00
groot bee66631e3
Refine bulkload (#19671) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-10-27 16:21:34 +08:00
Xiaofan def5972e01
fix duplicated handoff request (#20108) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-27 15:15:32 +08:00
aoiasd e470cd3dfa
Move Rocksmq page info log to Rokcsmq monitor (#20022) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-10-27 15:13:31 +08:00
bigsheeper b074f530e6
Forbid createIndex if collection loaded before (#20100) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-27 13:05:31 +08:00
Jiquan Long 3ff0112e49
Fix querynode panic occasionally (#20106) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-27 10:05:31 +08:00
xige-16 14968685a2
Limit disk index search list range (#20104) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-26 21:03:31 +08:00
jaime 976e9940bf
Add some logs for handoff observer (#20097) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-26 19:37:31 +08:00
bigsheeper 51e50bba73
Use Wlock when release segment (#20092) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-26 18:59:32 +08:00
XuanYang-cn c24244240e
Fix insert search concurrency problem (#20065) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-26 18:57:36 +08:00
congqixia d525cacb8d
Remove empty delta flow graph (#20076) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-26 17:17:31 +08:00
jaime d8ca56e084
Fix empty segment handoff fails (#20073) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-26 16:49:31 +08:00
xige-16 5bb2626466
Set diskUsageRatio to 4.0 when build disk index (#20066) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-26 16:11:31 +08:00
xige-16 89a9b6fda0
Init storage config's SSL and IAM config (#20082) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-26 16:07:30 +08:00
congqixia 27ebb766e7
Clean BF for compacted segment in datanode (#20085) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-26 14:35:31 +08:00
yah01 5429a973b4
Fix forget to fill the channel name (#20070) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-25 21:49:32 +08:00
bigsheeper f0c616050a
Reduce and improve quota center log (#20058) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-25 19:39:30 +08:00
Xiaofan 192151bc11
Refine Data Log (#19996) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 19:31:30 +08:00
Xiaofan 3d316fc052
Fix Query Log Level (#19995) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 19:29:36 +08:00
XuanYang-cn 0a2436fbe7
Remove meaningless logs (#20051) 
Fixes: #20050

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-25 17:53:32 +08:00
cai.zhang a60db370e6
Compatibility with handling index params (#19997) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 16:59:30 +08:00
Jiquan Long 290b902ab0
Fix rootcoord too frequent useless log (#20044) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 14:19:30 +08:00
Jiquan Long 377cd210c2
Make knowhere use segcore's cmake build type (#20041) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 14:17:37 +08:00
xige-16 32ba4fafb8
Delete Creating bucket when create core/minioChunkManager (#20049) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-25 14:15:30 +08:00
Xiaofan 1f170ce1ba
Refine kafka consumer (#19846) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 13:23:30 +08:00
cai.zhang 85ef7331ee
Set timeout for rpc (#19535) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 11:55:30 +08:00
cai.zhang c551de8f72
Catch errors on gloabl meta cache (#20023) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 11:29:30 +08:00
cai.zhang e7d58b4fb7
Set index state as retry when error occurred (#19978) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-25 11:09:30 +08:00
Jiquan Long db4ede217f
Fix index not exist (#20027) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 10:01:30 +08:00
Jiquan Long 7a13b1c329
Fix panic when stop rootcoord (#20019) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 09:51:30 +08:00
smellthemoon 2f19e6595a
Fill SourceID (#20000) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-10-24 23:43:30 +08:00
yah01 aa6b8af80d
Migrate metadata of QueryCoord from 2.1 to 2.2 (#19986) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-24 23:41:35 +08:00
xige-16 c8ecb81c04
Return IndexInfo if index build done (#19976) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-24 17:17:30 +08:00
Jiquan Long 14ddfe679f
Fix failed to show partitions (#20007) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-24 15:57:29 +08:00
Xiaofan 1b5e765307
Block Flowgraph before handle compacted segments (#19988) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-24 15:09:29 +08:00
congqixia 454ff2f1df
Fix ctx confusion caused goroutine leakage (#20017) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-24 14:31:29 +08:00
Xiaofan 868bdd504d
Fix flaky unit test (#20006) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-24 11:51:29 +08:00
XuanYang-cn eaae3aa49e
Fix metric recording auto flush segments twice (#19967) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-24 10:51:30 +08:00
Enwei Jiao 4e5d1c2e5f
Add log to debug bitset (#19982) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-10-24 10:09:28 +08:00
Jiquan Long 308a30fc46
Make dropping collection atomic (#19969) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-22 15:19:28 +08:00
congqixia 2e6ddd7f2a
Add datanode&channel match check for Flush (#19985) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-22 12:09:28 +08:00
Xiaofan 7d2ba60412
If autoID enabled, check to make sure user didn't specify autoID field (#19943) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-21 19:03:27 +08:00
zhagnlu 05bbb12d30
Skip build aws sdk when not need diskann (#19965) (#19966) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-10-21 18:35:29 +08:00
Jiquan Long a1a74e3891
Optimize ddl case when collection not exist (#19950) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-21 16:37:29 +08:00
liliu-z ce8e4d17b3
Fix mis-copy of chunk in make_with_timestamps (#19970) 
Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>

Signed-off-by: Li Liu <liliu@LideMacBook-Pro.local>
Co-authored-by: Li Liu <liliu@LideMacBook-Pro.local>
 2022-10-21 16:35:29 +08:00
smellthemoon 581e1d8c35
Fill MsgBase ALL (#19912) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-10-21 15:57:28 +08:00
cai.zhang 94c15a49e9
Can't drop loaded partition (#19938) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-21 14:41:28 +08:00
yah01 a4fb3cc3e2
Fix panic while applying deletions (#19956) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-21 14:25:29 +08:00
Xiaofan 3d3289ce3e
Fix collection metrics leakage (#19944) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-21 10:13:28 +08:00
yah01 e8ca754dca
Fix compaction produces empty segment (#19936) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 22:49:28 +08:00
xige-16 36accde034
Fix wrong msg when load failed (#19884) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-20 19:55:27 +08:00
jaime cf1491e41a
Add status for GetVersionResponse and CheckHealthResponse (#19895) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-20 19:47:33 +08:00
congqixia e8cf049392
Fix proxy trace operation name (#19932) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 18:35:28 +08:00
XuanYang-cn e081eb287d
Refine flowgraph nodes for readability (#19864) 
- delete compaction tasks from executor while dropping

See also: #19780

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-20 16:39:29 +08:00
congqixia 3de74165ac
Fix SyncSegment generate source (#19929) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 16:35:28 +08:00
congqixia b15e97a61a
Improve proxy leader cache lock logic (#19917) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 16:11:27 +08:00
aoiasd 7fe8e50689
print some group info in rocksmq monitor (#19863) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-10-20 16:05:27 +08:00
congqixia 0095869d5b
Keep segment statslog if no insert applied (#19910) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 14:33:27 +08:00
yah01 96cf74d53a
Add more panic info (#19922) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 13:33:27 +08:00
wayblink dbd60757c9
Support pass start_ts and end_ts through Import API (#19810) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-20 12:15:27 +08:00
yah01 470e8ef419
Store index ID to filter out invalid handoff event (#19703) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 11:29:27 +08:00
zhagnlu 02a8a07322
Fix BucketExists bug in minio chunk manager (#19548) (#19854) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-10-20 10:23:27 +08:00
yah01 5dcff9ab72
Fix QueryNode panic when meta changed (#19902) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-20 10:07:30 +08:00
cai.zhang 836081c8c7
Dropping index must happen without collection being loaded (#19886) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-19 23:17:44 +08:00
bigsheeper db0486c870
Fix missing import limit and rename functions in quotaCenter (#19907) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-19 21:43:30 +08:00
wei liu 87248dc1a9
fix clean handoff event (#19903) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-10-19 20:41:26 +08:00
yah01 52cd40fbc4
Remove codes related to deprecated field (#19897) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 18:29:26 +08:00
Jiquan Long 579c50fa1a
Fix io pool goroutine leakage (#19892) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-19 18:23:27 +08:00
jaime b79687687d
Replace segment index file path with index file id within segment index meta (#19866) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-19 16:55:27 +08:00
bigsheeper 99df2d9b54
Add binlogSize and ttDelay metrics monitor (#19872) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-19 15:13:26 +08:00
Ten Thousand Leaves 762045adff
Fix an issue where working task might get cleaned up (#19888) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-19 13:23:28 +08:00
congqixia b361842e9a
Add log for querynode checkpoint timestamp lag (#19883) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-19 13:21:31 +08:00
yah01 7cf26011e6
Add more logs for checkpoint of channel (#19885) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 13:19:27 +08:00
yah01 c61a4d1243
Fix panic due to double stop (#19849) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-19 12:13:28 +08:00
smellthemoon 108e51b2f0
[test]Create index before load and fix error message (#19874) (#19857) 
Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>

Signed-off-by: zhuwenxing <wenxing.zhu@zilliz.com>
Co-authored-by: zhuwenxing <wenxing.zhu@zilliz.com>
 2022-10-19 10:01:26 +08:00
congqixia 48463a12ef
Fix querynode concurrent map rw panic (#19867) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-18 19:47:36 +08:00
Xiaofan 5b9988b627
Support collection level monitoring metrics (#19686) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-18 19:17:27 +08:00
wayblink 4cc57adc5d
Initiatively release session in etcd when service close (#19825) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-18 19:13:26 +08:00
cai.zhang b3f6b67977
Alloc timestamp instead of using start position timestamp for segment index (#19820) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-18 17:05:26 +08:00
bigsheeper 15f7592739
Intercept import request in Proxy interceptor (#19856) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-18 16:53:26 +08:00
XuanYang-cn 2785f98de6
Refactor segment replica in DN (#19815) 
- Replace isNew and isFlushed with sType
- Add compacted into proto
- Rename replica to channel
- Rename segmentReplica to channelMeta
- Replace 4 maps with 1 map
- Add segment.go and segment_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-18 15:33:26 +08:00
congqixia a1cdc55bcb
Make newSegment transfer state after SaveBinlogPath succuess (#19858) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-18 14:07:25 +08:00
jaime 59bcbf0cf6
Support check health for the service level (#19779) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-18 13:39:26 +08:00
Ten Thousand Leaves db33f401fd
Fix proxy crash issue when meta cache miss (#19844) 
issue: #19557

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-18 13:11:26 +08:00
Ten Thousand Leaves a4a6f60279
Refine NQ check logic (#19819) 
issue: #19682

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-18 13:07:34 +08:00
Enwei Jiao 0a83cdfe1b
Refactor grpclient, make it generic (#19791) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-10-17 20:05:26 +08:00
yah01 b0404cbbab
Remove dropped segments' info (#19839) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-17 19:41:26 +08:00
yah01 17ce0c3979
Refine the error of QueryNode (#19836) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-10-17 19:39:26 +08:00