23dfbf1f34
Fix eventmanager retry interval ( #15631 )
...
See also: #15608
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-18 17:31:49 +08:00
5751759c85
Implement mix compaction logic ( #15542 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-18 14:51:49 +08:00
be42627ab2
Move Logutil to Util dir ( #15447 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig ( #15501 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-09 14:41:45 +08:00
2263425cde
Fix EntriesNum is not passed in flush task ( #15482 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-09 10:05:46 +08:00
1f337891e1
Rename base_param to service_param, rename global_param to component_param ( #15463 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
43b6c295a4
Put logs of compaction elapse in defer ( #15485 )
...
Before this PR, time elapse are logged after `compact()` finished
normally. No elapse will be logged with any errors.
This PR put elapse logs in defer once time-consuming steps are finished.
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-08 19:15:45 +08:00
46dea0f006
Add elapse logs in compaction ( #15414 )
...
See also: #15413
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-08 14:11:45 +08:00
f6873d3dc1
Implement automatic item expiration on compaction ( #14418 )
...
* Implement automatic item expiration on compaction
Signed-off-by: JunSung Park <writtic@gmail.com>
* Change entityExpiration value comparison logic
Signed-off-by: JunSung Park <writtic@gmail.com>
* Add logging for the number of expired entity
Signed-off-by: JunSung Park <writtic@gmail.com>
* Set default value of CompactionEntityExpiration to off
Signed-off-by: JunSung Park <writtic@gmail.com>
* Fix errors for code checker
Signed-off-by: JunSung Park <writtic@gmail.com>
* Set default entityExpiration to max value
Signed-off-by: JunSung Park <writtic@gmail.com>
2022-02-07 22:45:46 +08:00
5eef9886b3
Add more specific log when exiting BackGroundGC goroutine ( #15273 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-07 16:55:44 +08:00
f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable ( #15434 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-07 10:09:45 +08:00
4bccc85534
Use MsgChannelConfig in GlobalParams for all components ( #15163 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-02 00:35:43 +08:00
273890a91e
Fix datanode flush manager injection unit test ( #15358 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-27 10:23:40 +08:00
8ab9d769de
Fix logs of compaction_executor ( #15365 )
...
- Remove not useful log in `stopExecutingtaskByVChannelName`
- Add planID info in `mergeFlushedSegments`
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-25 13:29:40 +08:00
1e61112b5a
Add retry logic in pulsar consumer unsubscribe ( #15284 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-20 01:23:37 +08:00
87394e29b6
Fix compaction ut datarace ( #15267 )
...
Fixes : #15241
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-18 17:49:39 +08:00
7f775096c5
Unsubscribe when close pulsar consumer ( #15221 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-01-17 18:53:36 +08:00
fd8d452cfd
remove unused fieldIDs and fieldTypes ( #15229 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-01-16 17:37:35 +08:00
2528b6850d
Refactor flush manager injection to reduce goroutine number ( #15180 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-13 18:49:34 +08:00
008e08a996
Add flowgraph manager and event manager ( #15102 )
...
1. Add flowgraph manager to manager start and release of dataSyncService
2. Add event manager to manager etcd watchInfo event
- Make put event able to retry
- Make delete event able to terminate the retry loop
See also: #14604 , #14300
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-13 14:21:34 +08:00
6e847ff194
Make dn handleChannelInfo log more consist ( #14936 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-11 09:37:35 +08:00
b6a48817b0
Add MinioConfig in GlobalParams for all components ( #15099 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-10 17:29:34 +08:00
45f1f97ac4
Fix deltalog savebinlogpath save log ( #15092 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-01-10 15:45:34 +08:00
7efb02a4c5
Use PulsarConfig in GlobalParams for all components ( #15046 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-09 23:45:33 +08:00
f8f61fb116
Use BaseParams etcd configuration for all components ( #15012 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-01-07 16:15:26 +08:00
8b06aac7a9
[skip e2e] Improve annotation ( #14976 )
...
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-07 13:57:49 +08:00
a5f8bdeb42
Fix bug: data race in MsgFactory ( #14839 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-06 14:57:20 +08:00
25b00230c8
[skip e2e] Improve annotation ( #14908 )
...
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-06 14:15:20 +08:00
c488c41dfb
[skip e2e] Improve annotation ( #14912 )
...
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
2022-01-06 13:38:08 +08:00
ecdc3f7bf7
[skip e2e] Improve comment for DataNode delete node ( #14825 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-01-05 14:29:35 +08:00
98232b4942
[skip e2e]Sort imports ( #14820 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:25:48 +08:00
e1eb721532
[skip e2e]Sort imports ( #14819 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:23:52 +08:00
7031a511ce
[skip e2e]Sort imports ( #14806 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-01-05 14:00:38 +08:00
b10c361822
Fix bug: should use Lock instead of RLock ( #14797 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-04 23:33:24 +08:00
76a9687310
Adjust usage of lock in datanode ( #14631 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
2022-01-04 19:35:32 +08:00
7abebe576c
Add more logs on write path ( #14600 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-01-04 15:47:18 +08:00
f0a40f1569
Make FieldBinlog in a segment unique for a field ( #14632 )
...
See also: #14620
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-30 20:29:22 +08:00
ebb9b24b47
Improve name of flowgraph node ( #14538 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-30 10:33:46 +08:00
9d0bc136e0
Add vChannelName in IBN timetick log ( #14527 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-30 10:09:48 +08:00
69087ff8dd
Support embedded etcd ( #14333 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-12-29 14:35:21 +08:00
a8c7e0c827
[skip e2e] Add comment for Compaction of DataNode ( #14350 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-28 09:38:07 +08:00
3cfa88d6f2
Fix delete node not aware of merged segments error ( #14190 )
...
This PR made merge gracefully remove merged segment by moving
them into `compactedSegments`
And everytime when delete_node operates, if compactedSegments
have contents, delete_node will replace the latest segment in delBuf
merged from the merged segments, and truely remove the compactedSegments
Fixes : #14085
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-27 11:02:18 +08:00
9a2c91d326
Fix bug: refresh config after session init ( #14194 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-12-24 22:22:17 +08:00
3d24007efa
Add more logs for load sealed segment ( #14079 )
...
Add logs before&after flowgraph_ddnode forwarding delete msg
See also: #13879
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-24 10:50:07 +08:00
891c202b73
Use GlobalParamTable for all components ( #13885 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-12-23 18:39:11 +08:00
0871bd3d63
Add log when update pk range ( #14006 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-23 11:57:08 +08:00
6b91e5e9d1
Add warning log to trace old messages ( #13939 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-22 16:59:05 +08:00
2b352bcde5
Fix compact result segment log size always zero ( #13953 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-22 14:27:06 +08:00
2a6168e937
Improve import statement ( #13734 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-21 19:15:32 +08:00
87071f3f41
[skip e2e]Fix error log output format in segment_replica.go ( #13806 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-20 22:51:25 +08:00
5d4c4f38c8
Improve import statement ( #13735 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 15:49:06 +08:00
cabc40cb09
Improve import statement ( #13737 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 14:39:12 +08:00
a7dfd53f53
Improve import statement ( #13736 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 14:38:25 +08:00
08be6888d5
Improve import statement ( #13733 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-20 14:33:47 +08:00
2de5cf7762
[skip e2e] Improve log in flush manager ( #13689 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-20 12:22:43 +08:00
d4afc812d1
Normalize DataNode term in errors.go ( #13688 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-20 12:20:50 +08:00
1dcd06cc78
Add log file info for all kind of segment binlogs ( #13606 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-19 20:00:42 +08:00
20c956b675
[skip e2e]Fix error log output format in segment_replica.go ( #13676 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-19 12:32:42 +08:00
154f11e3a1
[skip e2e]Fix redundant conversion ( #13648 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-17 22:28:50 +08:00
e191b23249
[skip e2e]Fix error string ( #13647 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-17 19:22:42 +08:00
fefeaa2e1d
[skip e2e]Fix compaction executor typo ( #13646 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2021-12-17 19:20:46 +08:00
9f23fc7f2a
Register the service when the component state is healthy ( #13248 )
...
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
2021-12-15 11:47:10 +08:00
0c39e0370c
Move segment statistics update to datanode tt loop ( #13233 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-15 10:53:16 +08:00
30afcf2311
[skip ci]Fix error log output format in segment_replica.go ( #13350 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-14 16:51:09 +08:00
5f02a3738a
[skip ci]Fix error log output format in meta_service.go ( #13349 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-14 16:43:25 +08:00
405b3cd932
Use the same lint rules with golint in revive ( #13324 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
8f87cc7ef1
Fix JoinIDPath logic problem ( #13039 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-09 12:05:04 +08:00
dbf757e08b
Change genKey into JoinIDPath ( #12959 )
...
1. Simplify genKey(), make genKey a rpc call
2. Unify the usage of JoinIDPath
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-09 11:09:06 +08:00
a9a332dbcf
Fix empty segment compact and load ( #12710 )
...
1. Fix compaction save empty segment bug
2. Fix load empty segment bug
3. Add UT for dmlchannels to 100%
Resolves : #12450
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-12-09 11:03:08 +08:00
1195f441b2
Update DataNode state before stop ( #12946 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-08 14:19:04 +08:00
73b63b9a02
[skip ci]Fix error log output format in segment_replica.go ( #12929 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-08 13:55:04 +08:00
a6ce7d7f53
[skip ci]Fix error log output format in meta_service.go ( #12928 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-08 13:51:09 +08:00
e51ef53632
[skip ci]Fix error log output format in flush_manager.go ( #12857 )
...
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-07 14:38:15 +08:00
b20a238c38
Fix newDataSyncService panicking with normal segment has no dml pos ( #12771 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 16:21:35 +08:00
58c5b6093d
Fix close10000 ut has chances to fail ( #12740 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-06 09:59:35 +08:00
8bbaae04ff
Decrease compaction parallel ( #12707 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-12-03 18:57:33 +08:00
ffd05eb140
Fix deadlock in mergedTimeTickSender ( #12697 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-03 16:39:33 +08:00
45cbec1f7b
[skip ci] Refine datanode binlog io comment ( #12638 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-02 19:11:32 +08:00
c2b73e1b55
[skip ci] Refine data node flow graph comment ( #12637 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-02 18:59:33 +08:00
8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode ( #12563 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-02 16:39:33 +08:00
5e3e3b3b41
Remove grpc returned error ( #12537 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-12-01 22:17:46 +08:00
271aaa7537
[skip ci] Refine data node param table comment ( #12560 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2021-12-01 19:31:36 +08:00
e813867652
Implement Drop mode for flush manager ( #12469 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-01 10:11:39 +08:00
f31ed089b5
Add timeout for reader next ( #12308 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-26 22:45:24 +08:00
48b45d82e5
Add ut for binlog_io to 100 coverage ( #12283 )
...
Make DN ut coverage upto 90%
Resolves : #8058
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-26 17:43:17 +08:00
b6d0150dc5
Fix health check when disconnect to etcd ( #12292 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-26 11:39:16 +08:00
e245ba87a9
Add excluded segments for dropped segemnts ( #12247 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-11-25 14:25:15 +08:00
8a8ebed23d
Fix DN GC bug ( #12258 )
...
Fixes : #12249
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-25 09:43:15 +08:00
59d19384dc
Fix flush manager injection logic with multiple segments ( #12260 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-24 18:41:16 +08:00
e3f49858c5
Add subscription prefix ( #12251 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-24 17:47:15 +08:00
6c4c0ef6b5
Add more deploy metrics ( #12248 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-24 15:43:15 +08:00
e72c5b9351
Filter dropped segments in DN ( #12242 )
...
See also: #12131 , #12230
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-24 14:11:15 +08:00
6bf0c4326d
Fix compaction generates wrong insert binlogs ( #12204 )
...
This PR
- changes Get() to GetRow() in BinlogIterator
- removes Get() and Length() of FieldData
- adds ut in BinlogIterator for floatvector and binaryvector
- fix a log bug
Fixes : #12146
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-23 17:09:15 +08:00
4121e31df1
Send SIGINT to runner goroutine after etcd disconnects ( #12163 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-22 16:23:17 +08:00
51b353b52c
Add msgstream reader ( #12130 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-19 15:57:12 +08:00
7e91bcf115
Normalize datanode dminput node log ( #12128 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-19 14:21:13 +08:00
ee0f753f7a
Fix datarace between GetComponentStates and Register ( #11935 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-11-19 13:57:12 +08:00
038f42851e
Add more log for debug ( #11893 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-18 10:37:41 +08:00
7dbb4d10e5
Fix DeltaLogs empty yet appending a deltaInfo bug ( #11984 )
...
- Add more logs for startpositions
Fixes : #11970
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 17:57:11 +08:00
5e550a8793
Add close for flushManager ( #11874 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-17 10:39:15 +08:00
9bd3664672
Fix compaction bug ( #11816 )
...
This PR fixes:
- timeout in compaction
- BinlogIterator using rawID as PKField bug
See also: #11757
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-17 10:07:13 +08:00
5edbb82610
Add session revoke ( #11908 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-16 22:31:14 +08:00
12f50cb22c
[skip ci]Update OWNERS files ( #11898 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-11-16 15:41:11 +08:00
fa6314393a
Fix compactor loop closure captured ( #11814 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-15 19:44:10 +08:00
579416e3d3
Apply retry for flush manager execution ( #11808 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-15 17:19:10 +08:00
f3852c1db0
Add more specific logs ( #11724 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-15 15:25:09 +08:00
5c8978e5d6
Fix drop segment bug ( #11709 )
...
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-12 17:31:11 +08:00
c4bb73dc74
Complete drop collection procedure in DN ( #11552 )
...
drop, cancel compaction, flush all buffers and release fg
See also: #11426
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-11 20:56:49 +08:00
5357e301f4
Fix retrieve search error ( #11501 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-11 00:54:45 +08:00
14d2891a69
[skip ci]Fix download typo ( #11577 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-10 20:11:35 +08:00
0fc1e12faf
[skip ci]Fix download typo ( #11575 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-10 20:09:33 +08:00
038d0f1adf
[skip ci]Fix download typo ( #11573 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-10 20:07:39 +08:00
4c37e94a39
Add type assertion for compactTask ( #11468 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-10 19:16:40 +08:00
5f08b67156
Move node.yaml to component.yaml ( #11363 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-10 14:27:37 +08:00
863f1bb34e
Fix multi delete data not effect ( #11422 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 15:01:17 +08:00
0eb71f7ee5
Logging traceID in process of processing delete request ( #11374 )
...
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-11-09 11:01:02 +08:00
639800241d
Make flow graph for delta channel ( #11379 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-09 09:27:04 +08:00
3cd0584a04
Add compaction in datanode ( #11353 )
...
See also: #9530
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-08 19:49:07 +08:00
3ac7d7a9de
Add detailed comments for dataSyncService in DataNode ( #11285 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-05 22:59:04 +08:00
c26cd79532
Add compaction ( #9906 )
...
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-11-05 22:25:00 +08:00
0e5931a3da
Add mergedTimetickSender for datanode ( #11297 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-05 14:59:32 +08:00
9186d5527e
Forward delete msg ( #11210 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-05 11:59:02 +08:00
8863c01ff7
Make DN fail faster when watching an invalid chan ( #11102 )
...
See also: #11098
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-04 15:56:28 +08:00
0ef95c5df1
Add timeticker logger for insert buffer node ( #11216 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-04 15:40:14 +08:00
b63a9521c2
Make gen msg private ( #10115 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-04 15:36:19 +08:00
a0a3a889e7
Add common endian for global use ( #11092 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-02 18:16:32 +08:00
225b2c66a0
Fix datanode ut using register bug ( #10885 )
...
Resolves : #10881
See also: #8058
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-01 11:01:49 +08:00
1e3fc5076f
Change etcdkv clientv3 into MetaKv interface ( #10903 )
...
This pr:
- changed etcdkv clientv3 into MetaKv interface
- replaced fmt.Sprintf with path.Join for kv key
See also: #8058
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-01 10:19:55 +08:00
2df2d488e1
Fix Unit test fatal fail ( #10933 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-30 10:24:38 +08:00
756be81bce
Add injectFlush method for flushManager interface ( #10659 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-28 19:16:31 +08:00
6fddcb3925
Fix Session checker for proxy ( #10737 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-27 21:58:33 +08:00
42ce9b18d1
Add methods in segment replica ( #10582 )
...
see also: #8058
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-27 17:14:28 +08:00
169af67eb1
Fix segments not removed from flushingCache ( #10738 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-27 16:06:24 +08:00
aff5cd9598
Supprot to read the configuration of minio from environment variables ( #10534 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-10-26 15:10:03 +08:00
095a2378ca
Add datanode ut ( #10433 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-25 22:52:27 +08:00
ae2a301662
Add injection logic for FlushManager ( #10580 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 20:17:34 +08:00
21d98c2825
Add binlog io for compaction ( #10536 )
...
See also: #9530
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-25 20:13:51 +08:00
d2e0943f21
Fix delete node flush error ( #10549 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-25 18:49:20 +08:00
2713988ff2
Merge auto manual flush with same segment id ( #10550 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 18:03:42 +08:00
bd90a3831c
Allow flushed segments trigger flushe process ( #10539 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-25 18:01:50 +08:00
cd06f50645
Remove schema in delete codec ( #10517 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-10-24 09:59:10 +08:00
31393d8801
[skip ci]Fix datanode comment typo error ( #10502 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-23 16:57:35 +08:00
ffc0c07610
Change delete data primary key to int64 ( #10438 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 15:37:12 +08:00
d6bf0c928f
Get stats delta path from datacoord ( #10362 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-22 14:31:13 +08:00
7ce7cb7a5e
Add a timed flush trigger mechanism ( #10197 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-20 15:02:36 +08:00
6279fb817a
Fix stats log load with prefix error ( #10216 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:50:35 +08:00
8daeb0d519
Change bloom filter use pk ( #10193 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-10-19 20:18:47 +08:00
4031b8746f
Fix panic when flushing segment ( #10182 )
...
issue: #10180
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-10-19 19:06:37 +08:00
ed4e516ce7
Save stats and delta logs from DataNode ( #10159 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 14:34:35 +08:00
5737e0075f
Apply flush manager logic in datanode ( #10142 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-10-19 11:04:34 +08:00
XuanYang-cn
congqixia
Xiaofan
Cai Yudong
JunSung Park
bigsheeper
yah01
Xieql
zhenshan.cao
godchen
Jiquan Long
Bingyi Sun
Xiangyu Wang
JackLCL
cai.zhang
edward.zeng
sunby