milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
14,353 Commits
159 Branches
130 Tags
610 MiB
Commit Graph

463 Commits (c16195c74726caeeb3436ffb3e25817a6d3eb660)

Author SHA1 Message Date
Xieql c488c41dfb
[skip e2e] Improve annotation (#14912) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2022-01-06 13:38:08 +08:00
congqixia ecdc3f7bf7
[skip e2e] Improve comment for DataNode delete node (#14825) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-05 14:29:35 +08:00
godchen 98232b4942
[skip e2e]Sort imports (#14820) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:25:48 +08:00
godchen e1eb721532
[skip e2e]Sort imports (#14819) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:23:52 +08:00
godchen 7031a511ce
[skip e2e]Sort imports (#14806) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-05 14:00:38 +08:00
zhenshan.cao b10c361822
Fix bug: should use Lock instead of RLock (#14797) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-04 23:33:24 +08:00
zhenshan.cao 76a9687310
Adjust usage of lock in datanode (#14631) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: yangxuan <xuan.yang@zilliz.com>
 2022-01-04 19:35:32 +08:00
zhenshan.cao 7abebe576c
Add more logs on write path (#14600) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-04 15:47:18 +08:00
XuanYang-cn f0a40f1569
Make FieldBinlog in a segment unique for a field (#14632) 
See also: #14620

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-30 20:29:22 +08:00
Jiquan Long ebb9b24b47
Improve name of flowgraph node (#14538) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-30 10:33:46 +08:00
XuanYang-cn 9d0bc136e0
Add vChannelName in IBN timetick log (#14527) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-30 10:09:48 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
congqixia a8c7e0c827
[skip e2e] Add comment for Compaction of DataNode (#14350) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-28 09:38:07 +08:00
XuanYang-cn 3cfa88d6f2
Fix delete node not aware of merged segments error (#14190) 
This PR made merge gracefully remove merged segment by moving
them into `compactedSegments`

And everytime when delete_node operates, if compactedSegments
have contents, delete_node will replace the latest segment in delBuf
merged from the merged segments, and truely remove the compactedSegments

Fixes: #14085

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-27 11:02:18 +08:00
zhenshan.cao 9a2c91d326
Fix bug: refresh config after session init (#14194) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-24 22:22:17 +08:00
XuanYang-cn 3d24007efa
Add more logs for load sealed segment (#14079) 
Add logs before&after flowgraph_ddnode forwarding delete msg

See also: #13879

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-24 10:50:07 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
Jiquan Long 0871bd3d63
Add log when update pk range (#14006) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-23 11:57:08 +08:00
Bingyi Sun 6b91e5e9d1
Add warning log to trace old messages (#13939) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-22 16:59:05 +08:00
congqixia 2b352bcde5
Fix compact result segment log size always zero (#13953) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-22 14:27:06 +08:00
Xiangyu Wang 2a6168e937
Improve import statement (#13734) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-21 19:15:32 +08:00
JackLCL 87071f3f41
[skip e2e]Fix error log output format in segment_replica.go (#13806) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-20 22:51:25 +08:00
Xiangyu Wang 5d4c4f38c8
Improve import statement (#13735) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 15:49:06 +08:00
Xiangyu Wang cabc40cb09
Improve import statement (#13737) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 14:39:12 +08:00
Xiangyu Wang a7dfd53f53
Improve import statement (#13736) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 14:38:25 +08:00
Xiangyu Wang 08be6888d5
Improve import statement (#13733) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-20 14:33:47 +08:00
congqixia 2de5cf7762
[skip e2e] Improve log in flush manager (#13689) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-20 12:22:43 +08:00
congqixia d4afc812d1
Normalize DataNode term in errors.go (#13688) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-20 12:20:50 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-19 20:00:42 +08:00
JackLCL 20c956b675
[skip e2e]Fix error log output format in segment_replica.go (#13676) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-19 12:32:42 +08:00
godchen 154f11e3a1
[skip e2e]Fix redundant conversion (#13648) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 22:28:50 +08:00
godchen e191b23249
[skip e2e]Fix error string (#13647) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 19:22:42 +08:00
godchen fefeaa2e1d
[skip e2e]Fix compaction executor typo (#13646) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-17 19:20:46 +08:00
cai.zhang 9f23fc7f2a
Register the service when the component state is healthy (#13248) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-15 11:47:10 +08:00
Bingyi Sun 0c39e0370c
Move segment statistics update to datanode tt loop (#13233) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-15 10:53:16 +08:00
JackLCL 30afcf2311
[skip ci]Fix error log output format in segment_replica.go (#13350) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-14 16:51:09 +08:00
JackLCL 5f02a3738a
[skip ci]Fix error log output format in meta_service.go (#13349) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-14 16:43:25 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
congqixia 8f87cc7ef1
Fix JoinIDPath logic problem (#13039) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-09 12:05:04 +08:00
XuanYang-cn dbf757e08b
Change genKey into JoinIDPath (#12959) 
1. Simplify genKey(), make genKey a rpc call
2. Unify the usage of JoinIDPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-09 11:09:06 +08:00
XuanYang-cn a9a332dbcf
Fix empty segment compact and load (#12710) 
1. Fix compaction save empty segment bug
2. Fix load empty segment bug
3. Add UT for dmlchannels to 100%

Resolves: #12450

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-09 11:03:08 +08:00
congqixia 1195f441b2
Update DataNode state before stop (#12946) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-08 14:19:04 +08:00
JackLCL 73b63b9a02
[skip ci]Fix error log output format in segment_replica.go (#12929) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-08 13:55:04 +08:00
JackLCL a6ce7d7f53
[skip ci]Fix error log output format in meta_service.go (#12928) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-08 13:51:09 +08:00
JackLCL e51ef53632
[skip ci]Fix error log output format in flush_manager.go (#12857) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-07 14:38:15 +08:00
congqixia b20a238c38
Fix newDataSyncService panicking with normal segment has no dml pos (#12771) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-06 16:21:35 +08:00
congqixia 58c5b6093d
Fix close10000 ut has chances to fail (#12740) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-06 09:59:35 +08:00
Bingyi Sun 8bbaae04ff
Decrease compaction parallel (#12707) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-12-03 18:57:33 +08:00
congqixia ffd05eb140
Fix deadlock in mergedTimeTickSender (#12697) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-03 16:39:33 +08:00
edward.zeng 45cbec1f7b
[skip ci] Refine datanode binlog io comment (#12638) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-02 19:11:32 +08:00
edward.zeng c2b73e1b55
[skip ci] Refine data node flow graph comment (#12637) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-02 18:59:33 +08:00
congqixia 8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-02 16:39:33 +08:00
godchen 5e3e3b3b41
Remove grpc returned error (#12537) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-12-01 22:17:46 +08:00
edward.zeng 271aaa7537
[skip ci] Refine data node param table comment (#12560) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-01 19:31:36 +08:00
congqixia e813867652
Implement Drop mode for flush manager (#12469) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-01 10:11:39 +08:00
godchen f31ed089b5
Add timeout for reader next (#12308) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-26 22:45:24 +08:00
XuanYang-cn 48b45d82e5
Add ut for binlog_io to 100 coverage (#12283) 
Make DN ut coverage upto 90%
Resolves: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-26 17:43:17 +08:00
dragondriver b6d0150dc5
Fix health check when disconnect to etcd (#12292) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-26 11:39:16 +08:00
bigsheeper e245ba87a9
Add excluded segments for dropped segemnts (#12247) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-25 14:25:15 +08:00
XuanYang-cn 8a8ebed23d
Fix DN GC bug (#12258) 
Fixes: #12249

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-25 09:43:15 +08:00
congqixia 59d19384dc
Fix flush manager injection logic with multiple segments (#12260) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-24 18:41:16 +08:00
godchen e3f49858c5
Add subscription prefix (#12251) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-24 17:47:15 +08:00
dragondriver 6c4c0ef6b5
Add more deploy metrics (#12248) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-24 15:43:15 +08:00
XuanYang-cn e72c5b9351
Filter dropped segments in DN (#12242) 
See also: #12131, #12230

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-24 14:11:15 +08:00
XuanYang-cn 6bf0c4326d
Fix compaction generates wrong insert binlogs (#12204) 
This PR
- changes Get() to GetRow() in BinlogIterator
- removes Get() and Length() of FieldData
- adds ut in BinlogIterator for floatvector and binaryvector
- fix a log bug

Fixes: #12146

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-23 17:09:15 +08:00
congqixia 4121e31df1
Send SIGINT to runner goroutine after etcd disconnects (#12163) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-22 16:23:17 +08:00
godchen 51b353b52c
Add msgstream reader (#12130) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-19 15:57:12 +08:00
congqixia 7e91bcf115
Normalize datanode dminput node log (#12128) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-19 14:21:13 +08:00
dragondriver ee0f753f7a
Fix datarace between GetComponentStates and Register (#11935) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-19 13:57:12 +08:00
Bingyi Sun 038f42851e
Add more log for debug (#11893) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-18 10:37:41 +08:00
XuanYang-cn 7dbb4d10e5
Fix DeltaLogs empty yet appending a deltaInfo bug (#11984) 
- Add more logs for startpositions

Fixes: #11970

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-17 17:57:11 +08:00
congqixia 5e550a8793
Add close for flushManager (#11874) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-17 10:39:15 +08:00
XuanYang-cn 9bd3664672
Fix compaction bug (#11816) 
This PR fixes:
- timeout in compaction
- BinlogIterator using rawID as PKField bug

See also: #11757

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-17 10:07:13 +08:00
congqixia 5edbb82610
Add session revoke (#11908) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-16 22:31:14 +08:00
Xiangyu Wang 12f50cb22c
[skip ci]Update OWNERS files (#11898) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-16 15:41:11 +08:00
congqixia fa6314393a
Fix compactor loop closure captured (#11814) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-15 19:44:10 +08:00
congqixia 579416e3d3
Apply retry for flush manager execution (#11808) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-15 17:19:10 +08:00
XuanYang-cn f3852c1db0
Add more specific logs (#11724) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-15 15:25:09 +08:00
Bingyi Sun 5c8978e5d6
Fix drop segment bug (#11709) 
issue: #11708
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2021-11-12 17:31:11 +08:00
XuanYang-cn c4bb73dc74
Complete drop collection procedure in DN (#11552) 
drop, cancel compaction, flush all buffers and release fg

See also: #11426

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-11 20:56:49 +08:00
godchen 5357e301f4
Fix retrieve search error (#11501) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-11 00:54:45 +08:00
godchen 14d2891a69
[skip ci]Fix download typo (#11577) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-10 20:11:35 +08:00
godchen 0fc1e12faf
[skip ci]Fix download typo (#11575) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-10 20:09:33 +08:00
godchen 038d0f1adf
[skip ci]Fix download typo (#11573) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-10 20:07:39 +08:00
congqixia 4c37e94a39
Add type assertion for compactTask (#11468) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-10 19:16:40 +08:00
Xiaofan 5f08b67156
Move node.yaml to component.yaml (#11363) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-10 14:27:37 +08:00
godchen 863f1bb34e
Fix multi delete data not effect (#11422) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-09 15:01:17 +08:00
Xiangyu Wang 0eb71f7ee5
Logging traceID in process of processing delete request (#11374) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-09 11:01:02 +08:00
godchen 639800241d
Make flow graph for delta channel (#11379) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-09 09:27:04 +08:00
XuanYang-cn 3cd0584a04
Add compaction in datanode (#11353) 
See also: #9530

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-08 19:49:07 +08:00
congqixia 3ac7d7a9de
Add detailed comments for dataSyncService in DataNode (#11285) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-05 22:59:04 +08:00
sunby c26cd79532
Add compaction (#9906) 
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-11-05 22:25:00 +08:00
congqixia 0e5931a3da
Add mergedTimetickSender for datanode (#11297) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-05 14:59:32 +08:00
godchen 9186d5527e
Forward delete msg (#11210) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-05 11:59:02 +08:00
XuanYang-cn 8863c01ff7
Make DN fail faster when watching an invalid chan (#11102) 
See also: #11098

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-04 15:56:28 +08:00
congqixia 0ef95c5df1
Add timeticker logger for insert buffer node (#11216) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-04 15:40:14 +08:00
godchen b63a9521c2
Make gen msg private (#10115) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-04 15:36:19 +08:00
godchen a0a3a889e7
Add common endian for global use (#11092) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-02 18:16:32 +08:00
XuanYang-cn 225b2c66a0
Fix datanode ut using register bug (#10885) 
Resolves: #10881
See also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-01 11:01:49 +08:00
XuanYang-cn 1e3fc5076f
Change etcdkv clientv3 into MetaKv interface (#10903) 
This pr:
- changed etcdkv clientv3 into MetaKv interface
- replaced fmt.Sprintf with path.Join for kv key

See also: #8058

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-01 10:19:55 +08:00