milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/core/unittest
History
Buqian Zheng 070dfc77bf
feat: [Sparse Float Vector] segcore basics and index building (#30357) 
This commit adds sparse float vector support to segcore with the
following:

1. data type enum declarations
2. Adds corresponding data structures for handling sparse float vectors
in various scenarios, including:
* FieldData as a bridge between the binlog and the in memory data
structures
* mmap::Column as the in memory representation of a sparse float vector
column of a sealed segment;
* ConcurrentVector as the in memory representation of a sparse float
vector of a growing segment which supports inserts.
3. Adds logic in payload reader/writer to serialize/deserialize from/to
binlog
4. Adds the ability to allow the index node to build sparse float vector
index
5. Adds the ability to allow the query node to build growing index for
growing segment and temp index for sealed segment without index built

This commit also includes some code cleanness, comment improvement, and
some unit tests for sparse vector.

https://github.com/milvus-io/milvus/issues/29419

Signed-off-by: Buqian Zheng <zhengbuqian@gmail.com>
 		2024-03-11 14:45:02 +08:00
..
bench fix: Restore the MVCC functionality. (#29749) 2024-01-09 11:38:48 +08:00
test_utils feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
CMakeLists.txt enhance: [skip-e2e] disable asan (#30498) 2024-02-04 21:25:05 +08:00
init_gtest.cpp Add ut for growing segment load binlog (#26268) 2023-08-13 20:41:31 +08:00
test_always_true_expr.cpp fix: pass active count to query context instead of timestamp (#29541) 2023-12-31 16:08:48 +08:00
test_array.cpp feat: support inverted index (#28783) 2023-12-31 19:50:47 +08:00
test_array_expr.cpp enhance: Support more relational operators for binary expressions (#30902) 2024-03-01 16:57:00 +08:00
test_azure_chunk_manager.cpp enhance: find collection schema from cache (#28782) 2023-12-03 19:22:33 +08:00
test_bf.cpp feat: Support knowhere trace using OpenTelemetry (#30750) 2024-02-28 12:29:00 +08:00
test_binary.cpp add growing segment index (#23615) 2023-04-26 10:14:41 +08:00
test_binlog_index.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_bitmap.cpp Support SIMD of several Expr (#23715) (#23717) 2023-05-12 14:11:20 +08:00
test_bool_index.cpp Support SIMD of several Expr (#23715) (#23717) 2023-05-12 14:11:20 +08:00
test_c_api.cpp enhance: fix old pr cpp format issue (#30894) 2024-02-28 16:28:20 +08:00
test_chunk_cache.cpp Get vector concurrently (#27838) 2023-10-30 15:44:12 +08:00
test_common.cpp Support using mmap to load data (#22052) 2023-03-01 18:07:49 +08:00
test_concurrent_vector.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_data_codec.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_disk_file_manager_test.cpp enhance: optimize the memory usage and speed up loading variable length data (#30787) 2024-02-28 16:45:00 +08:00
test_exec.cpp fix: pass active count to query context instead of timestamp (#29541) 2023-12-31 16:08:48 +08:00
test_expr.cpp enhance: Support more relational operators for binary expressions (#30902) 2024-03-01 16:57:00 +08:00
test_float16.cpp feat: Support knowhere trace using OpenTelemetry (#30750) 2024-02-28 12:29:00 +08:00
test_group_by.cpp feat: Support knowhere trace using OpenTelemetry (#30750) 2024-02-28 12:29:00 +08:00
test_growing.cpp enhance: reduce 1x copy for variable length field while retrieving (#28345) 2023-11-15 18:08:20 +08:00
test_growing_index.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_index_c_api.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_index_wrapper.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_indexing.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_init.cpp enhance: update cagra index params in config and add params check (#29045) 2023-12-26 11:04:47 +08:00
test_integer_overflow.cpp feat: Support knowhere trace using OpenTelemetry (#30750) 2024-02-28 12:29:00 +08:00
test_inverted_index.cpp enhance: full-support for wildcard pattern matching (#30288) 2024-02-01 12:37:04 +08:00
test_local_chunk_manager.cpp Migrate the ability to upload and download binlog to cpp (#22984) 2023-06-25 14:38:44 +08:00
test_minio_chunk_manager.cpp enhance: Add precheck when chunk manager init (#28330) 2023-11-23 19:56:32 +08:00
test_offset_ordered_array.cpp fix: panic in concurrent insert/query scenario (#29408) 2023-12-26 15:10:48 +08:00
test_offset_ordered_map.cpp fix: panic in concurrent insert/query scenario (#29408) 2023-12-26 15:10:48 +08:00
test_parquet_c.cpp Remove other Exceptions, keeps SegcoreError only (#27017) 2023-09-14 14:05:20 +08:00
test_plan_proto.cpp Fix error message of proto parser not clear (#26365) 2023-08-16 20:20:17 +08:00
test_query.cpp feat: Support knowhere trace using OpenTelemetry (#30750) 2024-02-28 12:29:00 +08:00
test_range_search_sort.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_reduce.cpp Fix cpplint (#22657) 2023-03-10 09:47:54 +08:00
test_reduce_c.cpp Fix search binary pagination failure (#22477) 2023-03-08 11:03:51 +08:00
test_regex_query.cpp enhance: support pattern matching on json field (#30779) 2024-02-28 18:31:00 +08:00
test_regex_query_util.cpp enhance: support pattern matching on json field (#30779) 2024-02-28 18:31:00 +08:00
test_relational.cpp Fix cpplint (#22657) 2023-03-10 09:47:54 +08:00
test_remote_chunk_manager.cpp enhance: find collection schema from cache (#28782) 2023-12-03 19:22:33 +08:00
test_retrieve.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00
test_scalar_index.cpp enhance: change opendal as compile configurable (#30384) 2024-02-20 19:16:52 +08:00
test_scalar_index_creator.cpp feat: support inverted index (#28783) 2023-12-31 19:50:47 +08:00
test_sealed.cpp feat: Support knowhere trace using OpenTelemetry (#30750) 2024-02-28 12:29:00 +08:00
test_segcore.cpp Fix deleted data is still visible (#24849) 2023-06-16 17:16:41 +08:00
test_simd.cpp fix: move test from NEON to X86 (#30324) 2024-02-21 11:58:53 +08:00
test_similarity_corelation.cpp Add binary metric types SUBSTRUCTURE/SUPERSTRUCTURE back (#26766) 2023-08-31 20:07:00 +08:00
test_span.cpp enhance: reduce 1x copy for variable length field while retrieving (#28345) 2023-11-15 18:08:20 +08:00
test_storage.cpp fix: Prevent ChunkCache use absolute path in All-in-one mode (#30666) 2024-02-19 20:58:51 +08:00
test_string_expr.cpp enhance: fix old pr cpp format issue (#30894) 2024-02-28 16:28:20 +08:00
test_string_index.cpp enhance: change opendal as compile configurable (#30384) 2024-02-20 19:16:52 +08:00
test_timestamp_index.cpp Fix cpplint (#22657) 2023-03-10 09:47:54 +08:00
test_tracer.cpp fix: Save traceID and spanID as hex string into search config (#31071) 2024-03-11 14:21:01 +08:00
test_utils.cpp feat: [Sparse Float Vector] segcore basics and index building (#30357) 2024-03-11 14:45:02 +08:00