Commit Graph

721 Commits (e660cc3fe3fc46a7b71b45e60495332db7d5bee2)

Author SHA1 Message Date
Xiaofan f57fe6d70b
Add compaction log (#24976)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-06-19 14:18:41 +08:00
cui fliter 35cb0b5ba6
fix function name in comment (#24879)
Signed-off-by: cui fliter <imcusg@gmail.com>
2023-06-14 16:20:38 +08:00
wayblink bfae6b49af
Remove datanode timetick mq, use rpc to report instead (#23156)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-06-14 14:16:38 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-08 15:36:36 +08:00
yiwangdr 37c02c9927
clean up meta interface (#24729)
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
2023-06-08 14:18:35 +08:00
cai.zhang 93ea9c4925
Support return pending index rows when describe index (#24588)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-06-01 18:14:32 +08:00
yihao.dai 19164ea753
Make binlog size metric to collect healthy binlog only (#24557)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-31 16:45:29 +08:00
congqixia 6375236533
Make SegmentInfo.size atomic (#24457)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-29 11:31:28 +08:00
Ikko Eltociear Ashimine eb9ef23cca
Fix typo in handler.go (#24125)
postitions -> positions

Signed-off-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2023-05-17 14:25:23 +08:00
congqixia 73a181d226
Fix get vector it timeout and improve some string const usage (#24141)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-16 17:41:22 +08:00
yihao.dai fe0a1bc2d9
Fix panic caused by wrong logic of getting unindexed segments (#24044)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-15 20:59:28 +08:00
congqixia f6dec281c4
Add Statistic entity id log for datacoord (#24073)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-12 18:11:21 +08:00
Enwei Jiao 7e645dabf1
Add trace log for datacoord (#24046)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-05-12 09:53:21 +08:00
congqixia 5aa9db0d38
Add collection level auto compaction enabled config (#24013)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-05-10 17:45:20 +08:00
smellthemoon 146050db82
Fix some wrong ut (#23990)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-05-10 09:31:19 +08:00
yihao.dai fc010ee86b
Improve the metric of stored binlog size to the level of collection and segment (#23859)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-07 20:52:38 +08:00
wayblink 899702f13c
Implement GetIndexStatistics interface (#23603)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-05-06 10:34:39 +08:00
MrPresent-Han b517bc9e6a
refine balance mechanism including:(#23454) (#23763) (#23791)
1. balance granuity to replica to avoid influence unrelated replicas
2. avoid balance back and forth

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-05-04 12:22:40 +08:00
Xiaofan 016311ad48
Fix Has Collection Failed on datacoord (#23710)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-04-27 09:50:34 +08:00
yihao.dai ed8836cd15
Add disk quota at the collection level (#23704)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-26 21:52:36 +08:00
XuanYang-cn 72485c9eeb
Remove index check to trigger compaction (#23657)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-04-24 20:26:34 +08:00
congqixia 4a22af6e1a
Unwatch channel in watch buffer (#23548)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-20 10:34:31 +08:00
wei liu cbfe7a45ef
fix pull target (#23491)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-04-18 18:30:32 +08:00
congqixia d83654c33f
Add Close method for ChannelManager in datacoord (#23493)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-18 17:54:31 +08:00
zhenshan.cao 4a32b842e8
Improve the check logic of channel remove (#23473)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-18 02:58:30 +08:00
congqixia ba84f52119
Fix watcher loop quit and channel shouldDrop logic (#23402)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-14 09:54:28 +08:00
cai.zhang 43a9e175a3
Exit component process when session key is deleted (#21658) (#22164)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-12 20:12:28 +08:00
zhenshan.cao dc5abe086c
Revert "Restrain gc files by collection (#22504)" (#23380)
This reverts commit fa86de530d.

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-04-12 19:38:28 +08:00
cai.zhang 9288020da3
Fix bug for batch delete files on gcp of minio (#23052) (#23083) (#23090)
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2023-04-09 16:22:30 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
yihao.dai 17d3190fcc
Fix DescribeCollection failed with undefined msg type (#23192)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-03 19:52:24 +08:00
smellthemoon ce91f61e5a
Pre-create topics (#21907)
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-04-03 16:44:23 +08:00
Ten Thousand Leaves 1596abf6e8
Add lock for segment manager during bulk insert (#23111)
/kind bug

issue: #23106

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2023-03-31 20:16:24 +08:00
congqixia 127867b873
Add ratedgroup for some info/warning log (#23095)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-31 15:22:23 +08:00
zhenshan.cao 1287ca699a
Refine usage of TimeRecorder.Record (#23142)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2023-03-30 18:56:22 +08:00
yihao.dai a182594bf5
Add FlushAll support (#22942)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-03-30 11:28:25 +08:00
yah01 73f1f15c1c
Fix DataCoord's meta may be broken with DataNode of prior version (#23030)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-29 16:58:02 +08:00
Bingyi Sun d6bfe8fbb9
Change frequent logs to debug level (#23012)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-03-28 07:12:01 +08:00
aoiasd b8b7d1f47e
CreateIndex should wait all segments which has data insert before CreateIndex finish (#22561)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-20 10:21:56 +08:00
MrPresent-Han 77c9e33e70
support dml channel balancer on datacoord (#22324) (#22377) (#22692)
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2023-03-20 10:01:56 +08:00
SimFG 4a90490a67
Fix the `segment not found` error (#22772)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-17 17:27:56 +08:00
SimFG b57e476089
Fix the nil point about the session (#22748)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-03-14 20:07:54 +08:00
congqixia 732986aa04
Remove fmt.Print from internal package (#22722)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-14 17:36:05 +08:00
wayblink 28390dfe09
Add bulkinsert integration test (#22621)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-03-10 11:03:52 +08:00
wayblink aa6212d5b0
[Cherry-pick] Fix channel checkpoint issues and fix GC (#22559)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: Ten Thousand Leaves <69466447+soothing-rain@users.noreply.github.com>
2023-03-09 14:13:52 +08:00
yah01 efdc77bce6
Fix not fill timestamp when use collection created ts (#22629)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-08 18:53:51 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
yah01 319f1773af
Fix segment not found if forward delete to empty segment (#22528)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-03 14:13:49 +08:00
congqixia fa86de530d
Restrain gc files by collection (#22504)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-03-02 11:49:47 +08:00