Commit Graph

18 Commits (fe22720ff380e6342d5e7bdbd5bae4e0921f560e)

Author SHA1 Message Date
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
XuanYang-cn b90fa5f459
Refine codes of datanode buffer (#23168)
This PR refines deltabuffer of datanode:
- Add last sync time for compacted segment, see also: #23210
- Ensure all deltabuffermanager handles all delete related operations
- Change usedMemory to atomic.Int64
- Remove allocator in delete buffer

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-04-10 18:42:30 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
XuanYang-cn be0827417a
Fix buffer calculate memorysize negative (#23152)
Along with the following small patches:
- And fix logging dup segmentIDs for delete buffer status
- Add String method for PriorityQueue to debug

See also: #23105

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-03-31 18:30:23 +08:00
XuanYang-cn 9691315c21
Fix flush deltabuf with segmentID=0 (#23059)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-03-28 19:14:01 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536)
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-03-04 23:21:50 +08:00
wayblink 416866e42f
Add memory usage too large sync policy (#22244)
Signed-off-by: wayblink <anyang.wang@zilliz.com>
2023-02-27 17:47:51 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-02-26 11:31:49 +08:00
Xiaofan 4b4944ecee
Support sync all segments while close (#21421)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-01-06 14:49:36 +08:00
bigsheeper f595500383
Move sync delete policy from deleteNode to insertBufferNode (#21152)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-12-13 16:15:21 +08:00
Xiaofan e977e014a9
Fix flush didn't respect binaryvector and other schemas (#21120)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-12-12 10:33:26 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987)
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-12-07 18:01:19 +08:00
congqixia 2791f5526c
Fix BufferData limit calculation for BinaryVector (#20963)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-12-05 11:15:17 +08:00
bigsheeper 2d307c1909
Set delBuf's start-end position of compacted segment (#20614)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-15 19:41:16 +08:00
MrPresent-Han c848896bc8
Fix panic error due to rollDeleteOp ahead by load-and-delete (#20563)
issue:#20501
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
2022-11-15 14:45:07 +08:00
bigsheeper cd19d99ad7
Add channel level checkpoint (#20350)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-11-10 22:13:04 +08:00
MrPresent-Han 21b54709a2
enable delbuf auto flush function to avoid OOM when del msg accumulating (#20213)
issue: #19713

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: MrPresent-Han <zilliz@zillizdeMacBook-Pro-2.local>
2022-11-07 18:49:02 +08:00
XuanYang-cn e081eb287d
Refine flowgraph nodes for readability (#19864)
- delete compaction tasks from executor while dropping

See also: #19780

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-10-20 16:39:29 +08:00