milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[skip ci] Add document for growing segment internal parameters (#9225) 

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
pull/9238/head
Xiaofan 2021-10-04 22:02:02 +08:00 committed by GitHub
parent 57d5e72407
commit 4b62f177ee
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 20 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
docs/design_docs/segcore/segment_growing.md
View File

@ -16,6 +16,25 @@ When insert, first allocate enough space to ensure `total_size <= num_chunk * s
During search, each 'chunk' will be searched, and the search results will be saved as 'subquery result', then merged.
GrGowing Segment also implements small batch index for vectors. The parameters of small batch index are preset in 'segcoreconfig'
Growing Segment also implements small batch index for vectors. The parameters of small batch index are preset in 'segcoreconfig'
When `metric type ` is specified in the schema, the default parameters will build index for each chunk to accelerate query
## SegmentGrowingImpl internal paramters
1. SegcoreConfig contains parameters for Segcore，it has to be speficied before create segment
2. InsertRecord inserted data here
3. DeleteRecord wait for implementation
4. IndexingRecord data with small index
5. SealedIndexingRecord not used any more
### SegcoreConfig
1. Manage chunk_sizeand small index parateters
2. `parse_from` can parse from yaml files（this function is not enabled by default）
1. refer to `${milvus}/internal/core/unittest/test_utils/test_segcore.yaml`
3. `default_config` offers default parameters
### InsertRecord
Used to manage concurrent inserted data, incluing
1. `atomic<int64_t> reserved` reserved space calculation
2. `AckResponder` calculate which segment to insert，returns current inserted data offset
3. `ConcurrentVector` store data columns, each column has one concurrent vector