milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,526 Commits
141 Branches
117 Tags
562 MiB
Commit Graph

665 Commits (8ec5ba13dc326140d2d887ba5564c34aa2570e74)

Author SHA1 Message Date
XuanYang-cn c70af73005
Fix datacoord set wrong state for node registering (#17376) 
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-06 15:10:05 +08:00
shaoyue 76eaa3fc50
Support access s3 through AWS IAM role (#17292) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-02 19:42:03 +08:00
Ten Thousand Leaves f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324) 
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-02 18:54:04 +08:00
XuanYang-cn 42f643e727
Make DataNode release rather than delete when reassign (#17293) 
1. Reassgin now will assign to the original Node if no other nodes
   avaliable
2. Make AddNode balance async: ToRealse + Reassign

See also: #16114, #17270

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-02 13:56:04 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 16:36:03 +08:00
groot affcc2acad
Fix bulkload entity number bug (#17278) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-31 15:40:04 +08:00
congqixia 55f1e03ca7
Allow DataNode clean vchannel with meta error without panicking (#17247) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-27 16:20:00 +08:00
Ten Thousand Leaves 5e1e7a6896
Fix an issue that causes flush hanging (#17169) 
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-25 14:34:00 +08:00
bigsheeper 9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-19 10:13:56 +08:00
SimFG cf08b5aa11
Fix 'TestChannelManager_StateTransfer' ut when etcd is slow (#17082) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-05-18 17:19:57 +08:00
XuanYang-cn 127dd34b37
Make proxy use roundrobin to choose replica (#17063) 
Fixes: #17055

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-17 22:35:57 +08:00
Xiaofan 62658dcda6
Fix pulsar unscubsribe fail because of consumer not found (#16839) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-09 12:07:52 +08:00
Ten Thousand Leaves 6b2d3c3ec1
Make each dataNode only accept one bulk load task simultaneously (#16820) 
issue: #16674

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-07 14:05:52 +08:00
Enwei Jiao 67ce206cf0
make SegmentMaxLifetime configurable (#16783) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-05-06 23:41:51 +08:00
Ten Thousand Leaves a8e1c8fa9d
Make duplicate flush recalls success and other logic update (#16755) 
Latest logic:
1) Duplicate flush calls on same segments will not result in errors (same as the original design)
2) `FlushSegments` now still flushes stale segments even if non-stale segments failed to get flushed

issue: #16749

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-06 17:49:51 +08:00
cai.zhang 2be46a01c0
Reduce the number of retries and add error log (#16754) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-05-05 09:31:51 +08:00
zhenshan.cao 97757405de
Fix bug: wrong metrics name and forget to register metrics (#16714) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-29 15:35:47 +08:00
zhenshan.cao 0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
Xiaofan be5d1f1243
Fix Datacoord ut when etcd is slow (#16689) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-27 21:49:47 +08:00
groot bc5e9ec929
Add list import tasks api (#16605) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-25 17:37:46 +08:00
Ten Thousand Leaves e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:07:47 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
Ten Thousand Leaves 289e468a7a
Make bulk load fully work (#16512) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-20 14:03:40 +08:00
godchen 4781db8a2a
Add datanode import (#16414) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-12 22:19:34 +08:00
jaime ed3c29eae7
Support Kafka (#16149) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-12 19:47:33 +08:00
Ten Thousand Leaves 442c73a544
Support passing channel names to DataNode (#16424) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-12 13:25:34 +08:00
codeman 4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
Ten Thousand Leaves 1e6f6bd072
Implement set segment state call from RootCoord to DataCoord. (#16374) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-06 15:33:32 +08:00
XuanYang-cn 5494f3c318
Remove offLineNodes from watch states check (#16347) 
This PR also
- adds more information in log
- makes Session able to logged by zap.Any/zap.String

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-01 22:33:29 +08:00
Ten Thousand Leaves 45be3deb3f
Implement RC, DC, DN calling path for import. (#16321) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-01 11:33:28 +08:00
congqixia 376a7f6567
Fix Datacoord unsubAttempt using pchannel (#16337) 
Fix the channel name used when Datacoord unsub for DataNode
This channel shall always be pChannel instead of vChannel

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-01 11:25:28 +08:00
Ten Thousand Leaves 0d54697d42
Implement ReportImport related logic. (#16312) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-31 13:51:28 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
XuanYang-cn a77dd1034c
Adopt the extended state in DataCoord (#16200) 
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.

- WatchSuccess > log this info

- WatchFailure/WatchTimeout > ToRelease

- ReleaseSuccess > Delete, reassign if not from DropCollection

- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
  reassgin if not from DropCollection.

Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.

See also: #15846

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-28 22:33:27 +08:00
XuanYang-cn a00968fdfe
Fix unregister in datacoord cluster (#16182) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-25 12:09:25 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
groot 1c4b949a1d
modify import rpc interface (#16129) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-22 15:11:24 +08:00
groot 2078b24dca
Prepare for import (#16083) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-03-21 15:47:23 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
xige-16 09b2c46b61
Support create collection with VarChar field (#15904) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-14 23:20:02 +08:00
jaime 29975a7a26
Remove Consume() API from mqstream interface (#15886) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-11 20:09:59 +08:00
groot 89356ebcb9
Modify import rpc interfaces (#16006) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-03-11 17:13:59 +08:00
groot 5fdef607d5
Add rpc interfaces for import (#15930) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-03-09 18:33:59 +08:00
congqixia 6513be5c44
Extend watch/release channel logic in DataNode (#15925) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-09 15:39:58 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
Xiaofan bd7d294c6d
Refine log level in data (#15821) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-02 15:35:55 +08:00
yah01 0cfb6a85be
Make compaction not generate empty segment (#15707) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2022-02-28 10:17:54 +08:00
congqixia b0923f1299
Fix DataCoord panics if message queue service quits before it (#15702) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-23 11:09:52 +08:00
Nemo 3eb074a9b0
Update some comments for data coord with some minor code refactoring. (#15686) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-02-22 13:15:51 +08:00
Jenny Li 83f1358408
Update codecov.yml (#15639) 
Signed-off-by: Jenny Li <jing.li@zilliz.com>
 2022-02-18 18:29:50 +08:00
congqixia 5751759c85
Implement mix compaction logic (#15542) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-18 14:51:49 +08:00
Bingyi Sun daf25bcb81
Add log when remove keys in garbage collector. (#15453) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-02-18 11:59:49 +08:00
congqixia c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-15 15:07:48 +08:00
congqixia 7daed4ee47
Return flush state true when segment in Flushing state (#15530) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-10 16:55:46 +08:00
Xiaofan be42627ab2
Move Logutil to Util dir (#15447) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-02-09 18:55:46 +08:00
Cai Yudong 8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-09 14:41:45 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-08 20:57:47 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong 4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-02 00:35:43 +08:00
Xiaofan 885ecaf222
Refine time sync logic log (#15251) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-01-27 11:25:41 +08:00
Bingyi Sun 515fe962d5
Fix subscription path (#15373) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-25 13:27:44 +08:00
Bingyi Sun 3987574448
Unsubscribe topic in DataCoord (#15353) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-24 19:47:38 +08:00
xige-16 cc5bee4684
Stop handoff if segment has been compacted (#15181) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-13 21:49:35 +08:00
Cai Yudong 6323907566
Use CommonConfig in GlobalParams for all components (#15106) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 19:03:35 +08:00
shaoyue 8893772b03
[skip e2e] Fix comment lint for datacoord handler (#15087) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-10 18:29:40 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
godchen f2baa987fb
Handle trigger compaction error (#13636) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-07 17:41:40 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
godchen 69ec3832a4
[skip e2e]Sort imports (#14811) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 17:17:26 +08:00
godchen 1e0a0dee24
[skip e2e]Sort imports (#14817) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:20:04 +08:00
godchen 88f4c1ce38
[skip e2e]Sort imports (#14816) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:18:08 +08:00
godchen 3874edf163
[skip e2e]Sort imports (#14815) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:16:17 +08:00
godchen a0534d3139
[skip e2e]Sort imports (#14814) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:14:21 +08:00
godchen 0bba119500
[skip e2e]Sort imports (#14813) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:12:27 +08:00
godchen 43e8c4f6e5
[skip e2e]Sort imports (#14812) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:10:35 +08:00
godchen 3cc45eca9a
[skip e2e]Sort imports (#14810) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:08:42 +08:00
godchen 8fe4af3584
[skip e2e]Sort imports (#14809) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:06:48 +08:00
godchen b0357360d2
[skip e2e]Sort imports (#14807) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:04:46 +08:00
godchen 77ae88fc37
[skip e2e]Fix function typo (#14321) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 10:15:24 +08:00
zhenshan.cao 76a9687310
Adjust usage of lock in datanode (#14631) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
 2022-01-04 19:35:32 +08:00
Bingyi Sun 3e51997699
Add log info (#14430) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-04 19:29:45 +08:00
congqixia c3423a09c8
Add comment for chanPartSegments (#14748) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-04 17:33:22 +08:00
Bingyi Sun 37c9419707
Use logutil in DataCoord (#14737) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-04 17:31:23 +08:00
zhenshan.cao 7abebe576c
Add more logs on write path (#14600) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-04 15:47:18 +08:00
Bingyi Sun 75b0e9935b
[skip e2e]Use logutil in DataCoord (#14739) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-01-04 13:53:39 +08:00
godchen d7da870cc4
[skip e2e]Fix variable typo (#14325) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-31 15:49:19 +08:00
godchen f0e6e190df
[skip e2e]Fix variable typo (#14327) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-31 15:47:19 +08:00
congqixia 9e036af714
Set collectionID in FlushSegmentsRequest (#14563) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-30 19:11:30 +08:00
godchen ededf126c3
[skip e2e]Fix log typo (#14322) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-30 16:54:12 +08:00
godchen 11b1a13835
[skip e2e]Sort imports (#14319) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-30 16:51:54 +08:00
godchen d29243cbc5
[skip e2e]Sort imports (#14317) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-30 16:49:37 +08:00
Xieql 1c19aec7a8
[skip e2e] Improve annotation (#14564) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2021-12-30 16:33:41 +08:00
zhenshan.cao 32a5126705
Add the limitation of the traceable interval to timeTravel (#14498) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-29 21:29:45 +08:00
godchen 3e02ea1822
[skip e2e]Sort imports (#14316) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-29 20:37:46 +08:00
godchen 3d593a5c30
[skip e2e]Sort imports (#14315) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-29 20:36:03 +08:00
godchen ef677e2d0f
[skip e2e]Sort imports (#14314) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-29 20:34:18 +08:00
Jiquan Long 00b3fcb98b
Add log for flowgraph (#14441) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-29 14:55:21 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
Xiangyu Wang 5d89e69a93
Improve import statement (#14424) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-29 12:39:20 +08:00
Xiangyu Wang 6b432d2143
[skip e2e]Fix license missing issue in datacoord (#14491) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-29 11:40:48 +08:00
Xiangyu Wang 80ffa9c4d0
[skip e2e]Fix license missing issue in datacoord (#14489) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-29 11:38:57 +08:00
Xiangyu Wang 092c18b2ac
[skip e2e]Fix license missing issue in datacoord (#14490) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-29 11:36:47 +08:00
Xiaofan 1408926cd9
Remove logs without enough info (#14440) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 10:06:47 +08:00
Bingyi Sun 02e568603b
Add log with context in DataCoord (#14431) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-28 19:42:19 +08:00
Xiangyu Wang 4233e4e5be
[skip e2e]Fix license missing issue in datacoord (#14426) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-28 16:16:04 +08:00
Xiangyu Wang dba6a3f4b0
[skip e2e]Fix license missing issue in datacoord (#14427) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-28 15:43:51 +08:00
Xiangyu Wang 2dc0e702ee
[skip e2e]Fix license missing issue in datacoord (#14425) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-28 15:41:50 +08:00
godchen ef2a62387b
[skip e2e]Sort imports (#14329) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-27 18:54:37 +08:00
godchen a039178339
[skip e2e]Sort imports (#14328) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-27 18:52:57 +08:00
godchen 3cfa45ef29
[skip e2e]Sort imports (#14323) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-27 18:51:21 +08:00
godchen ee6765e1db
[skip e2e]Fix function typo (#14320) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-27 18:49:11 +08:00
godchen b8dcbf6604
[skip e2e]Sort imports (#14318) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-27 18:46:42 +08:00
Bingyi Sun 098516f3d7
Improve log (#13696) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-27 16:52:06 +08:00
congqixia 7c83b52de0
Add retry logic for DataCoord GC minio bucket (#14095) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-23 21:48:25 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
cxytz01 454c7126bd
[skip e2e] Grammar correction (#14053) 
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
 2021-12-23 15:12:21 +08:00
congqixia 336d5f917b
[skip e2e] Add comment for channel drop related handler functions (#14032) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-23 15:10:31 +08:00
Xiangyu Wang 011f54e4ac
Improve import statement (#13729) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-23 10:29:08 +08:00
congqixia 74ae8d3923
Fix nil segment caused panic in handleSignal (#13990) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-22 21:23:10 +08:00
guoxiangzhou 032c1c5f72
[skip e2e] Modify code comments according to naming conventions (#13942) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-22 15:21:22 +08:00
guoxiangzhou 1bd6b8b567
[skip e2e] Modify error text according to naming conventions (#13941) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-22 15:19:31 +08:00
Bingyi Sun ca71e134d1
[skip e2e]Improve log (#13700) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-22 15:15:05 +08:00
JackLCL 16769f8bfa
[skip e2e]Fix error log output format in meta.go (#13675) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-22 09:57:06 +08:00
Xiangyu Wang b3a2e816c7
Improve import statement (#13732) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-21 19:13:13 +08:00
Xiangyu Wang 3f79e1bfda
Improve import statement (#13723) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-21 19:11:11 +08:00
edward.zeng 8ede9c70b2
[skip e2e] Refine datacoord util comment (#13883) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-21 19:07:15 +08:00
edward.zeng 73773a84ba
[skip e2e] Refine datacoord services comment (#13882) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-21 19:05:19 +08:00
congqixia e1bcc0e037
Normalize DataNode term in DataCoord metrics_info.go (#13837) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-21 19:03:20 +08:00
congqixia 0b2c0fb261
[skip e2e] Remove commented unused code (#13843) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-21 15:38:20 +08:00
congqixia 6f63635686
[skip e2e] Add comment for Compaction method in session_manager.go (#13858) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-21 15:36:05 +08:00
Xiangyu Wang 1341dcd3f8
Improve import statement (#13730) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-21 15:21:25 +08:00
guoxiangzhou fc6c39cc88
[skip e2e] Modify log text according to naming conventions (#13842) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-21 12:03:22 +08:00
guoxiangzhou 094b5f48dc
[skip e2e] Modify log text according to naming conventions (#13841) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-21 12:01:31 +08:00
Bingyi Sun 7a8e49451f
Add log in datacoord (#13600) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-21 09:34:53 +08:00
godchen c1cb69ed7e
[skip ci]Fix segment info typo (#13643) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-21 09:19:53 +08:00
godchen d59647bb3f
[skip ci]Fix datanode compaction comment typo (#13631) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-21 09:19:10 +08:00
Bingyi Sun 9d44ea034c
[skip e2e]Improve log (#13704) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-21 09:17:00 +08:00
edward.zeng 38add97685
[skip e2e] Refine datacoord server comment (#13797) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-20 19:54:58 +08:00
edward.zeng 66b28259d9
[skip e2e] Refine datacoord session comment (#13796) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-20 19:48:53 +08:00
edward.zeng 1848dae589
[skip e2e] Refine datacoord policy comment (#13794) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-20 19:46:59 +08:00
Xiangyu Wang 8e78aeccaf
Improve import statement (#13726) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 18:13:05 +08:00
Xiangyu Wang 1fdd8fa823
Improve import statement (#13727) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 18:07:08 +08:00
guoxiangzhou 1dc4418059
[skip e2e] Modify log text according to naming conventions (#13710) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-20 17:42:07 +08:00
Xiangyu Wang 8960f54454
Improve import statement (#13731) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 15:47:38 +08:00
Xiangyu Wang e704af8d1f
Improve import statement (#13725) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 15:46:03 +08:00
Xiangyu Wang 7806115752
Improve import statement (#13728) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 14:36:38 +08:00
Xiangyu Wang 562d53fb15
Improve import statement (#13724) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 14:30:55 +08:00
guoxiangzhou 71c595e696
[skip e2e] Modify log text according to naming conventions (#13711) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-20 11:50:43 +08:00
guoxiangzhou 8248bd7d01
[skip e2e] Modify code comments according to naming conventions (#13709) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2021-12-20 11:20:43 +08:00
Bingyi Sun 7ee0a2bf9f
[skip e2e]Improve log (#13707) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-20 11:18:49 +08:00
Bingyi Sun 229a727b3a
[skip e2e]Improve log (#13706) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-20 11:16:52 +08:00
Bingyi Sun 9e0219ec3f
[skip e2e]Improve log (#13702) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-20 11:14:51 +08:00
Bingyi Sun 06c4662e97
[skip e2e]Improve log (#13705) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-20 11:06:43 +08:00
Bingyi Sun 279316e09b
[skip e2e]Improve log (#13701) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-20 11:02:45 +08:00
Bingyi Sun 3e7f76e265
[skip e2e]Improve log (#13642) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-20 10:42:43 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-19 20:00:42 +08:00
godchen f4caa94ea8
[skip e2e]Fix segment manager typo (#13644) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 22:34:42 +08:00
Bingyi Sun 63ebf76318
[skip e2e]Improve log print (#13640) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-17 19:48:42 +08:00
godchen c983af85c5
[skip e2e]Fix policy typo (#13645) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 19:46:42 +08:00
Bingyi Sun c7513bb1ad
[skip e2e]Improve log print (#13641) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-17 19:18:52 +08:00
godchen ec33f3b494
[skip e2e]Fix datanode compaction policy typo (#13635) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 19:16:55 +08:00
godchen e36f7287ae
[skip e2e]Fix datanode compaction variable typo error (#13630) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 19:14:59 +08:00
Bingyi Sun 371017ad39
Improve log print in datacoord (#13621) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-17 19:11:19 +08:00
edward.zeng 6b74aed32a
[skip e2e] Refine datacoord segment info comment (#13602) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-17 18:54:26 +08:00
edward.zeng d5791289fc
[skip e2e] Refine datacoord segment manager comment (#13605) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-17 18:49:09 +08:00
edward.zeng 5431e475ae
[skip e2e] Add comment to datacoord handler (#13555) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-17 18:05:24 +08:00
congqixia 016514bee6
Normalize DataCoord in error string (#13557) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-17 11:05:15 +08:00
edward.zeng 098e1375fb
[skip ci] Refine datacoord compaction trigger comment (#13491) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-16 09:59:12 +08:00
edward.zeng d6b0a47248
[skip ci] Refine datacoord compaction trigger comment (#13483) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-16 09:57:25 +08:00
edward.zeng d9bd36b58a
[skip ci] Refine datacoord compaction policy comment (#13489) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-16 09:41:10 +08:00
Xiangyu Wang 25022fa800
Improve import statements (#13435) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-15 20:49:10 +08:00
Xiangyu Wang c6908679b4
Improve import statements (#13436) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-15 20:47:10 +08:00
Xiangyu Wang 006a095370
Improve import statements (#13437) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-15 19:57:13 +08:00
edward.zeng dbae087d21
[skip ci] Refine datacoord garbage collector comment (#13430) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-15 18:25:43 +08:00
edward.zeng f44ede7b9d
[skip ci] Refine datacoord meta comment (#13432) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-15 16:57:23 +08:00
xige-16 97d5231a0a
Reorganize querycoord proto (#13181) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-15 16:53:12 +08:00
xiangzhou.guo 5e644a780c
correct spelling error (#13409) 
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
 2021-12-15 16:47:13 +08:00
Bingyi Sun 5c2302d711
Improve compaction log (#13394) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 16:07:10 +08:00
Bingyi Sun bd03236b79
Improve log (#13396) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 16:03:12 +08:00
Bingyi Sun 663a3aa333
Improve compaction log (#13395) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 16:01:11 +08:00
xiangzhou.guo a87e690d53
Correct spelling error in code comment (#13413) 
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
 2021-12-15 15:59:17 +08:00
xiangzhou.guo 78d583a3bb
Correct spelling error (#13412) 
Signed-off-by: xiangzhou.guo <xiangzhou.guo@zilliz.com>
 2021-12-15 15:57:09 +08:00
Bingyi Sun fcd1ee9c8e
Improve log (#13397) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 13:15:10 +08:00
edward.zeng f8b85c4646
[skip ci] Refine datacoord cluster comment (#13408) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-15 11:58:31 +08:00
cai.zhang 9f23fc7f2a
Register the service when the component state is healthy (#13248) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-15 11:47:10 +08:00
Bingyi Sun 0c39e0370c
Move segment statistics update to datanode tt loop (#13233) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 10:53:16 +08:00
Bingyi Sun f480b314d9
Improve compaction log (#13393) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 10:43:15 +08:00
Bingyi Sun 4a5aa1cfea
Improve datacoord log (#13392) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 10:41:27 +08:00
congqixia 3b4d114b71
Add comment for DataCoord startFlushLoop (#13379) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-15 09:45:09 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
congqixia 8476791492
Process only flushed/flushing segment in global compaction (#13339) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-14 15:15:10 +08:00
Bingyi Sun 3a2aace6f0
Fix ci issue caused by only compact flushed segments (#13317) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-14 13:55:07 +08:00
edward.zeng 6593e277f7
[skip ci] Refine datacoord metrics info comment (#13114) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-09 22:14:14 +08:00
edward.zeng d323b3bd84
[skip ci] Refine datacoord handler comment (#13113) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-09 22:10:23 +08:00
edward.zeng 4459611126
[skip ci] Refine datacoord meta comment (#13112) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-09 22:07:10 +08:00
XuanYang-cn a9a332dbcf
Fix empty segment compact and load (#12710) 
1. Fix compaction save empty segment bug
2. Fix load empty segment bug
3. Add UT for dmlchannels to 100%

Resolves: #12450

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-09 11:03:08 +08:00
Bingyi Sun a78b2010e9
[skip ci]Improve log style (#13008) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-08 20:27:06 +08:00
Bingyi Sun 48c5632277
[skip ci]Improve log style (#13007) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-08 20:25:15 +08:00
Bingyi Sun 320cfbb606
[skip ci]Improve log style (#13006) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-08 20:19:26 +08:00
congqixia aea5a61e9a
Add switch for auto compaction (#12974) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-08 19:47:05 +08:00
edward.zeng 260d3e124d
[skip ci] Refine datacoord channel store comment (#12991) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-08 17:57:05 +08:00
edward.zeng 2e84eef229
[skip ci] Refine datacoord channel manager comment (#12989) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-08 17:53:22 +08:00
edward.zeng 8ca248dce6
[skip ci] Refine datacoord allocator comment (#12985) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-08 17:35:06 +08:00
congqixia de454956fe
Refactor session WatchSessions to allow rewatch when Rev Compacted (#12880) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-08 10:11:04 +08:00
JackLCL f90410fe8e
[skip ci]Fix error log output format in segment_manager.go (#12856) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-07 14:36:27 +08:00
JackLCL de00aa038a
[skip ci]Fix error log output format in meta.go (#12854) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-07 14:32:51 +08:00
congqixia 22fddc537c
Fix not all segment state is drop after DropVirtualChannel (#12702) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-03 18:55:34 +08:00
congqixia 58664bb6b9
Fix zero length slice declaration in datacoord/channel_manager.go (#12582) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-02 19:17:38 +08:00
edward.zeng f8c826c597
Fix typo in datacoord compaction trigger (#12636) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-02 19:09:37 +08:00
congqixia 8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-02 16:39:33 +08:00
Bingyi Sun 0778e6f715
Fix alloc segment with dropped segment (#12572) 
issue: #12465
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-01 22:39:37 +08:00
godchen 5e3e3b3b41
Remove grpc returned error (#12537) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-12-01 22:17:46 +08:00
edward.zeng 5db458a44e
[skip ci] Add license for data coord compaction (#12561) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-01 19:33:32 +08:00
edward.zeng 446b5a3a48
[skip ci] Refine datacoord cluster comment (#12562) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-01 19:29:31 +08:00
Bingyi Sun 7ff15490c9
[skip ci]Add comment for GetVChanPosition (#12495) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-30 23:15:33 +08:00
Bingyi Sun 27898928b0
[skip ci]Modify WatchChannels comment in services.go (#12417) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@gmail.com>
 2021-11-29 23:09:41 +08:00
congqixia 06ced4d1d1
Add DropVirtualChannel for DataCoord (#12361) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-29 22:35:41 +08:00
congqixia eb48013042
Set start position for compact result segment (#12280) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-25 17:19:17 +08:00
congqixia 2ee0c7c074
Add compacted segment into check list when drop collection (#12272) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-25 15:13:15 +08:00
bigsheeper e245ba87a9
Add excluded segments for dropped segemnts (#12247) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-25 14:25:15 +08:00
dragondriver 6c4c0ef6b5
Add more deploy metrics (#12248) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-24 15:43:15 +08:00
congqixia 7b870e3392
Limit datacoord gc scan prefixes (#12231) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-24 09:55:15 +08:00
congqixia 8e0c8febe9
Add dropped segment ids in vchaninfo (#12230) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-23 18:59:15 +08:00
Bingyi Sun 3be9442c0f
Clear dropped segment meta on etcd (#12191) 
issue: #9613
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-23 11:23:15 +08:00
Bingyi Sun 01be91116e
Add GetFlushState interface (#12170) 
issue: #12075
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-23 10:55:14 +08:00
congqixia 8c107ab15f
Set segment DroppedAt when compacted (#12182) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-22 20:11:14 +08:00
congqixia 4121e31df1
Send SIGINT to runner goroutine after etcd disconnects (#12163) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-22 16:23:17 +08:00
shaoyue 6777d24c65
[skip ci] Add comment for exported items (#12126) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-19 18:05:25 +08:00
dragondriver ee0f753f7a
Fix datarace between GetComponentStates and Register (#11935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-19 13:57:12 +08:00
congqixia dfc70108c4
Add load collection for AssignSegmentID (#12077) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-18 22:31:34 +08:00
congqixia b99164f43d
Add datacoord GC configuration (#12061) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-18 22:29:40 +08:00
Bingyi Sun 175a656ff1
Unwatch dropped channel when init channel manager (#11986) 
issue: #11558
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-17 23:25:12 +08:00
XuanYang-cn 7dbb4d10e5
Fix DeltaLogs empty yet appending a deltaInfo bug (#11984) 
- Add more logs for startpositions

Fixes: #11970

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-17 17:57:11 +08:00
congqixia 5edbb82610
Add session revoke (#11908) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-16 22:31:14 +08:00
Bingyi Sun e3e4dc7153
Improve GetVChanPosistion and fix unit tests (#11888) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-16 21:41:50 +08:00
Xiangyu Wang 12f50cb22c
[skip ci]Update OWNERS files (#11898) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-16 15:41:11 +08:00
congqixia 7e43b568ee
Add dropped ts and apply when gc scan dropped segments (#11838) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-16 14:23:21 +08:00
shaoyue 6322eb31e7
[skip ci] Add comments for exported things in policy.go (#11876) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-16 13:21:16 +08:00
Bingyi Sun b8c5239784
Add compaction retention config (#11793) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-16 11:47:10 +08:00
Bingyi Sun 8b60aa8c4e
Skip single compaction if there's no delta log in a segment (#11806) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-16 11:23:11 +08:00
Bingyi Sun 902e856a06
[skip ci]Remove redundancy log (#11732) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-15 10:25:09 +08:00
congqixia d38043f235
Filter with partition id in GetVChanPosition (#11746) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-13 08:45:09 +08:00
Xiaofan 71e814f796
Remove common.yaml and component.yaml (#11661) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-12 21:25:08 +08:00
bigsheeper 93149c5ad9
Load growing segment in query node (#11664) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-12 18:27:10 +08:00
Bingyi Sun 5c8978e5d6
Fix drop segment bug (#11709) 
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-12 17:31:11 +08:00