milvus

Commit Graph

Author	SHA1	Message	Date
smellthemoon	6ef014d931	fix: get correct size when sealed segment chunked (#37062 ) #37019 Signed-off-by: lixinguo <xinguo.li@zilliz.com> Co-authored-by: lixinguo <xinguo.li@zilliz.com>	2024-10-25 12:01:31 +08:00
Bingyi Sun	a75bb85f3a	feat: support chunked column for sealed segment (#35764 ) This PR splits sealed segment to chunked data to avoid unnecessary memory copy and save memory usage when loading segments so that loading can be accelerated. To support rollback to previous version, we add an option `multipleChunkedEnable` which is false by default. Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-10-12 15:04:52 +08:00
Bingyi Sun	f229f244d2	enhance: add chunk basic impl (#34634 ) https://github.com/milvus-io/milvus/issues/35112 This pr would not affect milvus functionality by now. It implments a Chunk memory layout that looks like ``` VariableColumn \|offset\|offset\|offset\| \|data\|data\|data\| ``` We maybe move offsets to the beginning and add null bitmaps later but not in this PR. And mmap test will also be added in another PR. --------- Signed-off-by: sunby <sunbingyi1992@gmail.com>	2024-08-01 10:29:51 +08:00

Author

SHA1

Message

Date

smellthemoon

6ef014d931

fix: get correct size when sealed segment chunked (#37062 )

#37019

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>

2024-10-25 12:01:31 +08:00

Bingyi Sun

a75bb85f3a

feat: support chunked column for sealed segment (#35764 )

This PR splits sealed segment to chunked data to avoid unnecessary
memory copy and save memory usage when loading segments so that loading
can be accelerated.

To support rollback to previous version, we add an option
`multipleChunkedEnable` which is false by default.

Signed-off-by: sunby <sunbingyi1992@gmail.com>

2024-10-12 15:04:52 +08:00

Bingyi Sun

f229f244d2

enhance: add chunk basic impl (#34634 )

https://github.com/milvus-io/milvus/issues/35112
This pr would not affect milvus functionality by now.
It implments a Chunk memory layout that looks like 
```
VariableColumn
|offset|offset|offset|
|data|data|data|

```
We maybe move offsets to the beginning and add null bitmaps later but
not in this PR.
And mmap test will also be added in another PR.

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>

2024-08-01 10:29:51 +08:00

3 Commits (b3791a6f903c0fdefaef4bc26b55f091de8334a0)