milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

[skip e2e] Fix typo for design doc (#14920) 

Signed-off-by: yhmo <yihua.mo@zilliz.com>
pull/14921/head
groot 2022-01-06 10:57:20 +08:00 committed by GitHub
parent 3db94a7d0c
commit 7670bcec36
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 11 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

22
docs/design_docs/segcore/timetravel.md
View File

@ -1,23 +1,23 @@
# Time Travel Implementation (Segment Level)
Currently，there are two paths to implement time travel:
Currently， there are two paths to implement time travel:
1. Restrict with vec_count, used in growing segment
2. Generate bitmask and combine it with DSL calculation results. It is mainly used in sealed segment
1. Restrict with vec_count, used in growing segment.
2. Generate bitmask and combine it with DSL calculation results. It is mainly used in sealed segment.
## Growing Segment Time Travel
1. When inserting, ensure that the inserted data is in ascending time order
2. Find the location of timestamp with binary search and record it as vec_count
3. Call vector_search interface and rest is handled inside segment, no need to handle bitmask generated by DSL
1. When inserting, ensure that the inserted data is in ascending time order.
2. Find the location of timestamp with binary search and record it as vec_count.
3. Call vector_search interface and rest is handled inside segment, no need to handle bitmask generated by DSL.
## SealedSegment Time Travel
1. During load, data is placed in a continuous memory area named chunk, with the following properties:
1. Data is divided into multiple segments
2. Data in one segment is ordered by primary key
3. Data between Segments is in timestamp order. That is, the timestamp of every entity in the previous segment must be less than the timestamp of the first entity in the next segment
1. Data is divided into multiple segments.
2. Data in one segment is ordered by primary key.
3. Data between Segments is in timestamp order. That is, the timestamp of every entity in the previous segment must be less than the timestamp of the first entity in the next segment.
2. The Algorithm for time travel is:
1. Use get_active_count interface, find the last segment containing a legal ts, and return the last element position of this segment as vec_count
1. Use get_active_count interface, find the last segment containing a legal ts, and return the last element position of this segment as vec_count.
2. Calculate the bitset mask with a timestamp. Due to the above properties, all the entities of the previous segment meet the conditions, and all the subsequent segments do not meet the conditions. Only the "last segment" needs to be calculated.
3. the calculated results of Bitset and DSL are combined and sent to vector search interface
3. the calculated results of Bitset and DSL are combined and sent to vector search interface.