milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,666 Commits
127 Branches
113 Tags
553 MiB
Commit Graph

62 Commits (556c1cfc8a4d41d7b3914179d49903283a0fd199)

Author SHA1 Message Date
jaime d126f06946
Decouple mq module from internal proto definition (#22536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-03-04 23:21:50 +08:00
Xiaofan 3d6bded115
Fix Row Number Mismatch (#22307) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-27 10:41:46 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
cai.zhang 511265c68c
Modify lastSyncTime in advance to prevent multiple flush binlogs (#22048) (#22088) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-12 18:56:33 +08:00
wayblink 6a722396bd
Integration test framework (#21283) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-12 19:49:40 +08:00
Xiaofan 4b4944ecee
Support sync all segments while close (#21421) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-06 14:49:36 +08:00
congqixia 40abb13413
Generate unique id with ts for sync task (#20928) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-01 20:35:18 +08:00
Bingyi Sun 2390095232
Fix load uses compacted segments' binlogs (#20655) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-11-17 20:37:10 +08:00
smellthemoon a7ba416b5a
Remove invalid timestamp (#20557) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-14 16:21:07 +08:00
bigsheeper cd19d99ad7
Add channel level checkpoint (#20350) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-10 22:13:04 +08:00
Enwei Jiao 956c5e1b9d
Make Params singleton (#20088) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-04 14:25:38 +08:00
Xiaofan 2bfecf5b4e
Refine bloomfilter and memory usage (#20168) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 17:41:34 +08:00
Xiaofan 192151bc11
Refine Data Log (#19996) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 19:31:30 +08:00
smellthemoon 581e1d8c35
Fill MsgBase ALL (#19912) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-10-21 15:57:28 +08:00
congqixia 0095869d5b
Keep segment statslog if no insert applied (#19910) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-20 14:33:27 +08:00
XuanYang-cn 2785f98de6
Refactor segment replica in DN (#19815) 
- Replace isNew and isFlushed with sType
- Add compacted into proto
- Rename replica to channel
- Rename segmentReplica to channelMeta
- Replace 4 maps with 1 map
- Add segment.go and segment_test.go

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-10-18 15:33:26 +08:00
congqixia a1cdc55bcb
Make newSegment transfer state after SaveBinlogPath succuess (#19858) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-18 14:07:25 +08:00
SimFG a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
congqixia 838a633584
Add ctx parameter for ChunkManager methods (#19546) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-29 16:18:56 +08:00
jaime 91df8f2d6a
Use log id instead of log path of the binlog in metastore (#19123) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-09-25 15:56:51 +08:00
SimFG d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
bigsheeper 5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector (#18636) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-16 09:56:47 +08:00
congqixia 68a6587374
Set insert&stats binlog timestamp range (#19005) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-04 09:05:09 +08:00
congqixia 29c72cad15
Use segment statslog instead of separate statslog (#18775) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-09-02 10:34:59 +08:00
congqixia c91bb0b015
Unify rootPath in configs and ChunkManager (#18808) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-25 19:32:53 +08:00
Ten Thousand Leaves 0ebe407221
Make DataNode not panic if stale segment is not found during flush (#18322) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-20 15:06:30 +08:00
congqixia be6ca6358d
Use serialized size before compression in log meta (#18336) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-19 21:30:30 +08:00
Ten Thousand Leaves ecc250c20e
Improve logs to better cover segment flush lifecycle (#18157) 
/kind improvements

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-07-08 14:50:21 +08:00
congqixia 12b3a29db8
Fix segments info in drop virtual channel request is not set (#17641) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-20 13:40:12 +08:00
congqixia 55f1e03ca7
Allow DataNode clean vchannel with meta error without panicking (#17247) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-27 16:20:00 +08:00
zhenshan.cao 0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
godchen 78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
cai.zhang 88527fc0a9
Fix metrics label conflict (#15839) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-03 16:05:57 +08:00
Xiaofan bd7d294c6d
Refine log level in data (#15821) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-03-02 15:35:55 +08:00
bigsheeper 13177a90af
Add prometheus metrics for DataNode (#15650) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
 2022-02-28 19:11:55 +08:00
congqixia 273890a91e
Fix datanode flush manager injection unit test (#15358) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-27 10:23:40 +08:00
congqixia 2528b6850d
Refactor flush manager injection to reduce goroutine number (#15180) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-13 18:49:34 +08:00
XuanYang-cn 45f1f97ac4
Fix deltalog savebinlogpath save log (#15092) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-01-10 15:45:34 +08:00
Xieql 8b06aac7a9
[skip e2e] Improve annotation (#14976) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2022-01-07 13:57:49 +08:00
Xieql c488c41dfb
[skip e2e] Improve annotation (#14912) 
Signed-off-by: Xieql <qianglong.xie@zilliz.com>
 2022-01-06 13:38:08 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
congqixia 2de5cf7762
[skip e2e] Improve log in flush manager (#13689) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-20 12:22:43 +08:00
congqixia 1dcd06cc78
Add log file info for all kind of segment binlogs (#13606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-19 20:00:42 +08:00
XuanYang-cn dbf757e08b
Change genKey into JoinIDPath (#12959) 
1. Simplify genKey(), make genKey a rpc call
2. Unify the usage of JoinIDPath

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-09 11:09:06 +08:00
JackLCL e51ef53632
[skip ci]Fix error log output format in flush_manager.go (#12857) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-07 14:38:15 +08:00
congqixia 8f9e62fa18
Apply DropVirtualChannel and FlushManager drop mode (#12563) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-02 16:39:33 +08:00
congqixia e813867652
Implement Drop mode for flush manager (#12469) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-01 10:11:39 +08:00
XuanYang-cn 8a8ebed23d
Fix DN GC bug (#12258) 
Fixes: #12249

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-25 09:43:15 +08:00