4781db8a2a
Add datanode import ( #16414 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-12 22:19:34 +08:00
442c73a544
Support passing channel names to DataNode ( #16424 )
...
/kind feature
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-12 13:25:34 +08:00
1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. ( #16374 )
...
/kind feature
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-06 15:33:32 +08:00
45be3deb3f
Implement RC, DC, DN calling path for import. ( #16321 )
...
/kind feature
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-01 11:33:28 +08:00
0d54697d42
Implement ReportImport related logic. ( #16312 )
...
/kind feature
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-31 13:51:28 +08:00
a77dd1034c
Adopt the extended state in DataCoord ( #16200 )
...
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.
- WatchSuccess > log this info
- WatchFailure/WatchTimeout > ToRelease
- ReleaseSuccess > Delete, reassign if not from DropCollection
- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
reassgin if not from DropCollection.
Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.
See also: #15846
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-28 22:33:27 +08:00
1c4b949a1d
modify import rpc interface ( #16129 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
2078b24dca
Prepare for import ( #16083 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
89356ebcb9
Modify import rpc interfaces ( #16006 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
5fdef607d5
Add rpc interfaces for import ( #15930 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig ( #15843 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
bd7d294c6d
Refine log level in data ( #15821 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
3eb074a9b0
Update some comments for data coord with some minor code refactoring. ( #15686 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-22 13:15:51 +08:00
7daed4ee47
Return flush state true when segment in Flushing state ( #15530 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-10 16:55:46 +08:00
be42627ab2
Move Logutil to Util dir ( #15447 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
4bccc85534
Use MsgChannelConfig in GlobalParams for all components ( #15163 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
cc5bee4684
Stop handoff if segment has been compacted ( #15181 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-13 21:49:35 +08:00
76a9687310
Adjust usage of lock in datanode ( #14631 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2022-01-04 19:35:32 +08:00
3e51997699
Add log info ( #14430 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-01-04 19:29:45 +08:00
891c202b73
Use GlobalParamTable for all components ( #13885 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
b3a2e816c7
Improve import statement ( #13732 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-21 19:13:13 +08:00
73773a84ba
[skip e2e] Refine datacoord services comment ( #13882 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-21 19:05:19 +08:00
0b2c0fb261
[skip e2e] Remove commented unused code ( #13843 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-21 15:38:20 +08:00
7a8e49451f
Add log in datacoord ( #13600 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-21 09:34:53 +08:00
3e7f76e265
[skip e2e]Improve log ( #13642 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-20 10:42:43 +08:00
1dcd06cc78
Add log file info for all kind of segment binlogs ( #13606 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
63ebf76318
[skip e2e]Improve log print ( #13640 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-17 19:48:42 +08:00
c7513bb1ad
[skip e2e]Improve log print ( #13641 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-17 19:18:52 +08:00
371017ad39
Improve log print in datacoord ( #13621 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-17 19:11:19 +08:00
5c2302d711
Improve compaction log ( #13394 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 16:07:10 +08:00
bd03236b79
Improve log ( #13396 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 16:03:12 +08:00
663a3aa333
Improve compaction log ( #13395 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 16:01:11 +08:00
fcd1ee9c8e
Improve log ( #13397 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 13:15:10 +08:00
0c39e0370c
Move segment statistics update to datanode tt loop ( #13233 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
f480b314d9
Improve compaction log ( #13393 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:43:15 +08:00
4a5aa1cfea
Improve datacoord log ( #13392 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:41:27 +08:00
a9a332dbcf
Fix empty segment compact and load ( #12710 )
...
1. Fix compaction save empty segment bug
2. Fix load empty segment bug
3. Add UT for dmlchannels to 100%
Resolves : #12450
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-09 11:03:08 +08:00
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode ( #12563 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
5e3e3b3b41
Remove grpc returned error ( #12537 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 22:17:46 +08:00
27898928b0
[skip ci]Modify WatchChannels comment in services.go ( #12417 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@gmail.com>
2021-11-29 23:09:41 +08:00
06ced4d1d1
Add DropVirtualChannel for DataCoord ( #12361 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-29 22:35:41 +08:00
01be91116e
Add GetFlushState interface ( #12170 )
...
issue: #12075
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-23 10:55:14 +08:00
6777d24c65
[skip ci] Add comment for exported items ( #12126 )
...
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2021-11-19 18:05:25 +08:00
ee0f753f7a
Fix datarace between GetComponentStates and Register ( #11935 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-19 13:57:12 +08:00
dfc70108c4
Add load collection for AssignSegmentID ( #12077 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-18 22:31:34 +08:00
175a656ff1
Unwatch dropped channel when init channel manager ( #11986 )
...
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-17 23:25:12 +08:00
7dbb4d10e5
Fix DeltaLogs empty yet appending a deltaInfo bug ( #11984 )
...
- Add more logs for startpositions
Fixes : #11970
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 17:57:11 +08:00
e3e4dc7153
Improve GetVChanPosistion and fix unit tests ( #11888 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-16 21:41:50 +08:00
d38043f235
Filter with partition id in GetVChanPosition ( #11746 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-13 08:45:09 +08:00
93149c5ad9
Load growing segment in query node ( #11664 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-12 18:27:10 +08:00
5c8978e5d6
Fix drop segment bug ( #11709 )
...
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 17:31:11 +08:00
54b40da4c5
Add segment drop logic ( #11621 )
...
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 00:22:42 +08:00
7a3873960b
Fix watch channel with nil seek position ( #11650 )
...
issue: #11649
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-11 19:32:43 +08:00
5574a104c9
Add triggerInfo in compactionTask ( #11517 )
...
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-11 15:54:42 +08:00
5357e301f4
Fix retrieve search error ( #11501 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-11 00:54:45 +08:00
f01063a020
[skip ci]Add comment for GetCompactionState ( #11599 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-11 00:00:45 +08:00
1104f752f5
[skip ci]Add comment for ManualCompaction ( #11598 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-10 23:58:57 +08:00
adea19352c
[skip ci]Add comment for CompleteCompaction ( #11597 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-10 23:57:38 +08:00
c3f0c5a3ff
Add compaction interfaces in proxy ( #11431 )
...
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-09 14:47:02 +08:00
05681364c9
Fix some review comments in compaction. related pr #9906 ( #11399 )
...
issue: #9904
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2021-11-08 21:45:00 +08:00
c26cd79532
Add compaction ( #9906 )
...
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
c019e80dbb
Change SeekPosition to earliest of all segments ( #10771 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-28 14:30:26 +08:00
e5a88374bd
[skip ci]Update license for dc server ( #10589 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-25 19:46:28 +08:00
d6bf0c928f
Get stats delta path from datacoord ( #10362 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 14:31:13 +08:00
6a886d63f2
Support stats and delta log SaveBinlog logic ( #10156 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 14:32:41 +08:00
044b06421a
Use etcd to watch channel on datanode ( #7115 )
...
issue: #6578
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-14 15:44:34 +08:00
26a365c06d
Print more information when handling SaveBinlogPathsRequest ( #9803 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-13 20:54:33 +08:00
a11f9b7eb5
Add log for Flush to print return segments with CollectionID ( #8928 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-30 18:12:25 +08:00
5390136cd6
[skip ci] Fix golint error of services.go in datacoord ( #8524 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-24 21:33:55 +08:00
79b7dcc75a
Add trace log for insert ( #8301 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-09-22 19:33:54 +08:00
c76f91233f
Put errors together ( #7958 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-15 21:05:52 +08:00
b7103357fd
Load failed if collection beyonds the memory limit ( #7434 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-09-07 11:35:18 +08:00
1f590d6c26
Add datacoord services related unit tests ( #7467 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-06 11:12:42 +08:00
godchen
Ten Thousand Leaves
XuanYang-cn
groot
Cai Yudong
Xiaofan
congqixia
xige-16
zhenshan.cao
Bingyi Sun
Xiangyu Wang
edward.zeng
shaoyue
dragondriver
bigsheeper
sunby
cai.zhang