milvus/internal/querynodev2/segments
Ted Xu ebb648f647
feat: integrate manifest-based statistics for V3 storage segments (#48005)
See: #48006

design doc:
https://github.com/milvus-io/milvus-design-docs/blob/main/design_docs/20260226-manifest-format.md

Integrate manifest-based statistics storage (bloom filter, BM25,
text index, JSON key index) for V3 LOON segments across all write
and read paths.

Key changes:
- Add StatsResolver to centralize V2/V3 stats branching with lazy
  manifest caching, shared across segment loading, L0 compaction,
  and flush recovery
- Write-side: pack_writer_v3, sort_compaction, task_stats register
  stats in manifest with memory_size metadata
- Read-side: segment_loader, l0_compactor, data_sync_service use
  StatsResolver for unified path resolution
- New FFI layer for manifest stats reading/writing
- PackSegmentLoadInfo clears legacy fields when manifest_path set
- Fix manifest version chaining across sequential stats registrations

---------

Signed-off-by: Ted Xu <ted.xu@zilliz.com>
2026-03-23 12:01:28 +08:00
..
metricsutil
state
OWNERS
agg_reducer.go feat: support query aggregtion(#36380) (#44394) 2026-01-06 16:29:25 +08:00
agg_reducer_test.go feat: support query aggregtion(#36380) (#44394) 2026-01-06 16:29:25 +08:00
cgo_util.go
collection.go enhance: optimize qn load speed (#47423) 2026-03-05 16:55:21 +08:00
collection_test.go enhance: optimize qn load speed (#47423) 2026-03-05 16:55:21 +08:00
count_reducer.go
default_limit_reducer.go
disk_usage_fetcher.go enhance: replace fmt.Sprint(paramtable.GetNodeID()) with paramtable.GetStringNodeID() (#47789) 2026-02-27 23:43:47 +08:00
index_attr_cache.go
index_attr_cache_test.go
load_index_info.go enhance: replace fmt.Sprint(paramtable.GetNodeID()) with paramtable.GetStringNodeID() (#47789) 2026-02-27 23:43:47 +08:00
manager.go enhance: replace fmt.Sprint(paramtable.GetNodeID()) with paramtable.GetStringNodeID() (#47789) 2026-02-27 23:43:47 +08:00
manager_test.go
mock_collection_manager.go
mock_loader.go enhance: [loon] support loading deltalogs from StorageV3 manifest in QueryNode (#48095) 2026-03-10 09:59:24 +08:00
mock_segment.go enhance: remove deprecated lazy load code (#47590) 2026-02-10 14:14:44 +08:00
mock_segment_manager.go
pool.go enhance: save BM25 stats to the disk while loading the segment (#46468) 2026-01-09 19:29:26 +08:00
pool_test.go
reducer.go feat: support query aggregtion(#36380) (#44394) 2026-01-06 16:29:25 +08:00
result.go feat: impl StructArray -- support element-level query (#47906) 2026-03-13 17:55:29 +08:00
result_sorter.go
result_test.go feat: impl StructArray -- support element-level query (#47906) 2026-03-13 17:55:29 +08:00
retrieve.go enhance: replace fmt.Sprint(paramtable.GetNodeID()) with paramtable.GetStringNodeID() (#47789) 2026-02-27 23:43:47 +08:00
retrieve_test.go enhance: Add sparse filter in search (#47444) 2026-02-02 10:09:49 +08:00
search.go enhance: Go search path optimizations for improved QPS (#47734) 2026-02-24 22:12:47 +08:00
search_reduce.go enhance: Go search path optimizations for improved QPS (#47734) 2026-02-24 22:12:47 +08:00
search_reduce_test.go
search_test.go enhance: remove stream reduce logic(#47516) (#47544) 2026-02-09 18:25:52 +08:00
segcore.go
segment.go fix: prevent heap-use-after-free in Reopen by pinning segment (#48265) 2026-03-16 14:29:24 +08:00
segment_do.go enhance: remove deprecated lazy load code (#47590) 2026-02-10 14:14:44 +08:00
segment_filter.go enhance: Add sparse filter in search (#47444) 2026-02-02 10:09:49 +08:00
segment_interface.go enhance: remove deprecated lazy load code (#47590) 2026-02-10 14:14:44 +08:00
segment_l0.go enhance: upgrade Go dependencies (casbin, gin, lo, cockroachdb/errors) (#47943) 2026-03-16 21:11:26 +08:00
segment_loader.go feat: integrate manifest-based statistics for V3 storage segments (#48005) 2026-03-23 12:01:28 +08:00
segment_loader_test.go enhance: [loon] support loading deltalogs from StorageV3 manifest in QueryNode (#48095) 2026-03-10 09:59:24 +08:00
segment_test.go fix:remove snapshot when DumpSnapshot Assert failure (#48250) 2026-03-17 12:07:26 +08:00
statistics.go enhance: Add sparse filter in search (#47444) 2026-02-02 10:09:49 +08:00
trace.go
utils.go enhance: auto warmup for the big tenant collection (#47630) 2026-02-26 22:49:19 +08:00
utils_test.go enhance: add async warmup policy support for caching layer (#47627) 2026-03-10 14:45:23 +08:00
validate.go enhance: Add sparse filter in search (#47444) 2026-02-02 10:09:49 +08:00