Commit Graph

7 Commits (cb18f18c1d9100883eeec361b7f097cad879279b)

Author SHA1 Message Date
congqixia 39be35804c
enhance: Add back clean compacted segment info logic (#28646)
See also #27675
Compacted segment info shall be removed after all buffer belongs to it
is sync-ed.
This PR add the cleanup function after triggerSyncTask logic:
- The buffer is stable and protected by mutex
- Cleanup fetches compacted & non-sync segment
- Remove segment info only there is no buffered maintained in manager

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-24 15:38:25 +08:00
congqixia 18dc6b61ce
enhance: fix LevelZero segment sync logic (#28482)
See also #27675
- Fix LevelZero segment cannot be flushed
- Add level option for syncTask
- Invoke `AddSegment` when new LevelZero segment is allocated

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-17 21:46:20 +08:00
congqixia 0b905078e7
Use writebuffer, sync manager refactory in datanode (#28320)
See also #27675
This PR make previously merged refactory of datanode go online
- Use write node to replace insert/delete node
- Use write buffer manager to control all buffers
- Use sync manager to control sync tasks instead of flush manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-15 15:24:18 +08:00
congqixia bf2f62c1e7
Add `WriteBuffer` to provide abstraction for delta policy (#27874)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-04 12:10:17 +08:00
congqixia 1e51255c15
Implement `Injection` for SyncManager with block and meta transition (#28093)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-03 04:48:15 +08:00
congqixia 233bf90c55
Add SyncManager to replace flush manager (#27873)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-31 02:30:16 +08:00
congqixia 98e2aad752
Refine datanode metacache and implement CoW (#27985)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-28 11:12:11 +08:00