milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
milvus/internal/storage
History
congqixia 7774b7275e
enhance: Replace PrimaryKey slice with PrimaryKeys saving memory (#37127) 
Related to #35303

Slice of `storage.PrimaryKey` will have extra interface cost for each
element, which may cause notable memory usage when delta row count
number is large.

This PR replaces PrimaryKey slice with PrimaryKeys interface saving the
extra interface cost.

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 		2024-10-28 10:29:30 +08:00
..
aliyun Identify service providers based on addresses (#27907) 2023-10-25 17:28:10 +08:00
gcp Format the code (#27275) 2023-09-21 09:45:27 +08:00
tencent feat: Support tencent cloud object storage for milvus (#30163) 2024-01-23 11:28:56 +08:00
OWNERS [skip ci]Update OWNERS files (#11898) 2021-11-16 15:41:11 +08:00
azure_object_storage.go enhance: Add nilness linter and fix some small issues (#34049) 2024-06-24 14:52:03 +08:00
azure_object_storage_test.go enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 2024-04-25 20:41:27 +08:00
binlog_iterator.go enhance: legacy code clean up (#33838) 2024-06-14 14:25:56 +08:00
binlog_iterator_test.go enhance: Add Segment Level in milvus segment info APIs (#34763) 2024-07-26 10:01:46 +08:00
binlog_reader.go fix: descriptor event in previous version not has nullable to parse error (#34235) 2024-07-01 16:38:06 +08:00
binlog_test.go enhance: binlog primary key turn off dict encoding (#34358) 2024-07-17 17:47:44 +08:00
binlog_util.go Move some modules from internal to public package (#22572) 2023-04-06 19:14:32 +08:00
binlog_util_test.go Format the code (#27275) 2023-09-21 09:45:27 +08:00
binlog_writer.go feat: support embedding bm25 sparse vector and flush bm25 stats log (#36036) 2024-09-19 10:57:12 +08:00
binlog_writer_test.go enhance: binlog primary key turn off dict encoding (#34358) 2024-07-17 17:47:44 +08:00
data_codec.go fix: binlog reader not released in time (#36078) 2024-09-07 08:15:06 +08:00
data_codec_test.go enhance: import supports null in parquet and json formats (#35558) 2024-08-20 16:50:55 +08:00
data_sorter.go feat: [Sparse Float Vector] add sparse vector support to milvus components (#30630) 2024-03-13 14:32:54 -07:00
data_sorter_test.go enhance: Enable linux code checker (#35084) 2024-07-30 15:53:51 +08:00
delta_data.go enhance: Replace PrimaryKey slice with PrimaryKeys saving memory (#37127) 2024-10-28 10:29:30 +08:00
delta_data_test.go enhance: Add unittest for `storage.DeleteLog` (#34190) 2024-06-26 17:14:04 +08:00
event_data.go enhance: Fix lint issues from recent PRs (#34482) 2024-07-09 10:06:24 +08:00
event_header.go Move some modules from internal to public package (#22572) 2023-04-06 19:14:32 +08:00
event_reader.go enhance: support null in go payload (#32296) 2024-06-19 17:08:00 +08:00
event_test.go enhance: binlog primary key turn off dict encoding (#34358) 2024-07-17 17:47:44 +08:00
event_writer.go enhance: binlog primary key turn off dict encoding (#34358) 2024-07-17 17:47:44 +08:00
event_writer_test.go enhance: binlog primary key turn off dict encoding (#34358) 2024-07-17 17:47:44 +08:00
factory.go enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 2024-09-30 13:23:32 +08:00
field_stats.go feat: Major compaction (#33620) 2024-06-10 21:34:08 +08:00
field_stats_test.go feat: Major compaction (#33620) 2024-06-10 21:34:08 +08:00
field_value.go fix: refine handling type for segment pruner(#34923) (#34925) 2024-07-25 13:57:45 +08:00
field_value_test.go feat: Define FieldValue, FieldStats and PartitionStats (#30286) 2024-03-06 20:42:37 -08:00
gcp_native_object_storage.go enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 2024-09-30 13:23:32 +08:00
gcp_native_object_storage_test.go enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 2024-09-30 13:23:32 +08:00
index_data_codec.go enhance: support null in c data_datacodec and load null value (#32183) 2024-07-23 16:07:51 +08:00
index_data_codec_test.go enhance: Add memory size for binlog (#33025) 2024-05-15 12:59:34 +08:00
insert_data.go feat: added more checks for function creation check (#36766) 2024-10-13 17:43:22 +08:00
insert_data_test.go enhance: import supports null in parquet and json formats (#35558) 2024-08-20 16:50:55 +08:00
local_chunk_manager.go enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 2024-04-25 20:41:27 +08:00
local_chunk_manager_test.go enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 2024-04-25 20:41:27 +08:00
minio_object_storage.go enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 2024-04-25 20:41:27 +08:00
minio_object_storage_test.go fix: Removed minio bucket after use in test (#32624) 2024-04-28 13:51:26 +08:00
options.go enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 2024-09-30 13:23:32 +08:00
partition_stats.go fix: sync part stats task cannot be finished(#30376) (#34027) 2024-06-24 10:16:02 +08:00
partition_stats_test.go feat: Define FieldValue, FieldStats and PartitionStats (#30286) 2024-03-06 20:42:37 -08:00
payload.go enhance: support null in go payload (#32296) 2024-06-19 17:08:00 +08:00
payload_reader.go enhance: Update protobuf-go to protobuf-go v2 (#34394) 2024-07-29 11:31:51 +08:00
payload_reader_test.go enhance: support null in go payload (#32296) 2024-06-19 17:08:00 +08:00
payload_test.go enhance: Enable linux code checker (#35084) 2024-07-30 15:53:51 +08:00
payload_writer.go enhance: Update protobuf-go to protobuf-go v2 (#34394) 2024-07-29 11:31:51 +08:00
payload_writer_test.go enhance: binlog primary key turn off dict encoding (#34358) 2024-07-17 17:47:44 +08:00
pk_statistics.go fix: Use k locations only for basic BF test location (#35380) 2024-08-09 07:52:22 +08:00
primary_key.go enhance: Replace PrimaryKey slice with PrimaryKeys saving memory (#37127) 2024-10-28 10:29:30 +08:00
primary_key_test.go enhance: Replace PrimaryKey slice with PrimaryKeys saving memory (#37127) 2024-10-28 10:29:30 +08:00
primary_keys.go enhance: Add PrimaryKeys interface to reduce memory usage (#30405) 2024-02-01 09:57:11 +08:00
primary_keys_test.go enhance: Add PrimaryKeys interface to reduce memory usage (#30405) 2024-02-01 09:57:11 +08:00
print_binlog.go enhance: Update protobuf-go to protobuf-go v2 (#34394) 2024-07-29 11:31:51 +08:00
print_binlog_test.go enhance: Update protobuf-go to protobuf-go v2 (#34394) 2024-07-29 11:31:51 +08:00
remote_chunk_manager.go enhance: GCS native support (GCS implemented using Google Cloud Storage libraries) (#36214) 2024-09-30 13:23:32 +08:00
remote_chunk_manager_test.go enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 2024-04-25 20:41:27 +08:00
serde.go enhance: Optimize the use of locks and avoid double flush clustering buffer writer (#35486) 2024-08-16 02:24:58 +08:00
serde_events.go fix: not set nullable when stream writer write headers (#35799) 2024-08-29 20:59:00 +08:00
serde_events_test.go feat: integrate new deltalog format (#35522) 2024-08-20 19:06:56 +08:00
serde_test.go enhance: add delta log stream new format reader and writer (#34116) 2024-07-06 09:08:09 +08:00
stats.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
stats_test.go enhance: Use Blocked Bloom Filter instead of basic bloom fitler impl. (#33405) 2024-05-31 17:49:45 +08:00
storage_test.go enhance: Remove vector chunk manager (#28569) 2023-11-30 18:00:33 +08:00
types.go enhance: use WalkWithPrefix api for oss, enable piplined file gc (#31740) 2024-04-25 20:41:27 +08:00
unsafe.go enhance: support null in go payload (#32296) 2024-06-19 17:08:00 +08:00
unsafe_test.go [skip e2e]Update license for storage unsafe (#14452) 2021-12-28 20:03:56 +08:00
utils.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00
utils_test.go feat: support load and query with bm25 metric (#36071) 2024-10-11 10:23:20 +08:00