From 4b62f177ee36904d9a5482bb6dfd142e6e3bfb3a Mon Sep 17 00:00:00 2001 From: Xiaofan <83447078+xiaofan-luan@users.noreply.github.com> Date: Mon, 4 Oct 2021 22:02:02 +0800 Subject: [PATCH] [skip ci] Add document for growing segment internal parameters (#9225) Signed-off-by: xiaofan-luan --- docs/design_docs/segcore/segment_growing.md | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/docs/design_docs/segcore/segment_growing.md b/docs/design_docs/segcore/segment_growing.md index 1d609bdf1e..4f60009947 100644 --- a/docs/design_docs/segcore/segment_growing.md +++ b/docs/design_docs/segcore/segment_growing.md @@ -16,6 +16,25 @@ When insert, first allocate enough space to ensure `total_size <= num_chunk * s During search, each 'chunk' will be searched, and the search results will be saved as 'subquery result', then merged. -GrGowing Segment also implements small batch index for vectors. The parameters of small batch index are preset in 'segcoreconfig' +Growing Segment also implements small batch index for vectors. The parameters of small batch index are preset in 'segcoreconfig' When `metric type ` is specified in the schema, the default parameters will build index for each chunk to accelerate query + +## SegmentGrowingImpl internal paramters +1. SegcoreConfig contains parameters for Segcore，it has to be speficied before create segment +2. InsertRecord inserted data here +3. DeleteRecord wait for implementation +4. IndexingRecord data with small index +5. SealedIndexingRecord not used any more + +### SegcoreConfig +1. Manage chunk_sizeand small index parateters +2. `parse_from` can parse from yaml files（this function is not enabled by default） + 1. refer to `${milvus}/internal/core/unittest/test_utils/test_segcore.yaml` +3. `default_config` offers default parameters + +### InsertRecord +Used to manage concurrent inserted data, incluing +1. `atomic reserved` reserved space calculation +2. `AckResponder` calculate which segment to insert，returns current inserted data offset +3. `ConcurrentVector` store data columns, each column has one concurrent vector