milvus/internal/flushcommon/pipeline
yihao.dai 6130a85444
enhance: Remove bf from streaming node (#35902)
Remove bf from streaming node:
1. When watching vchannels, skip loading bloom filters for segments.
2. Bypass bloom filter checks for delete messages, directly writing to
L0 segments.
3. Remove flushed segments proactively after flush.

issue: https://github.com/milvus-io/milvus/issues/33285,
https://github.com/milvus-io/milvus/issues/34585

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2024-09-03 14:17:02 +08:00
..
data_sync_service.go enhance: Remove bf from streaming node (#35902) 2024-09-03 14:17:02 +08:00
data_sync_service_test.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_dd_node.go enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406) 2024-08-29 10:03:08 +08:00
flow_graph_dd_node_test.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_dmstream_input_node.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_dmstream_input_node_test.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_manager.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_manager_test.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_message.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_message_test.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_node.go enhance: Organize the common modules of streamingNode and dataNode (#34773) 2024-07-22 11:33:51 +08:00
flow_graph_time_tick_node.go enhance: Implement flusher in streamingNode (#34942) 2024-08-02 18:30:23 +08:00
flow_graph_write_node.go enhance: using streaming service in insert/upsert/flush/delete/querynode (#35406) 2024-08-29 10:03:08 +08:00
mock_fgmanager.go enhance: Organize the common modules of streamingNode and dataNode (#34773) 2024-07-22 11:33:51 +08:00
testutils_test.go enhance: Add back BF lazy load logic for datanode watch channel (#35646) 2024-08-22 19:42:57 +08:00