ee54a98578
enhance: Add cgo call metrics for load/write API ( #37405 )
...
Cgo API cost is not observerable since not metrics is related to them.
This PR add metrics for some sync cgo call related to load & write
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-11-07 10:06:25 +08:00
130a923dec
enhance: the estimate method when loading the collection ( #36307 )
...
- issue: #36530
---------
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: xianliang.li <xianliang.li@zilliz.com>
Co-authored-by: xianliang.li <xianliang.li@zilliz.com>
2024-10-09 17:35:19 +08:00
a773836b89
enhance: optimize milvus core building ( #35610 )
...
issue: #35549,#35611,#35633
- remove milvus_segcore milvus_indexbuilder..., add libmilvus_core
- core building only link once
- move opendal compilation into cmake
- fix odr
---------
Signed-off-by: chyezh <chyezh@outlook.com>
2024-08-23 12:35:02 +08:00
731d45abbe
enhance: provide more general configuration to control mmap behavior ( #35359 )
...
- issue: #35273
Signed-off-by: SimFG <bang.fu@zilliz.com>
2024-08-21 00:22:54 +08:00
aa247f192d
enhance: remove unused code for StorageV2 ( #35132 )
...
issue: https://github.com/milvus-io/milvus/issues/34168
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2024-08-01 12:08:13 +08:00
c45f38aa61
enhance: Update protobuf-go to protobuf-go v2 ( #34394 )
...
issue: #34252
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2024-07-29 11:31:51 +08:00
0c5d8660aa
feat: support inverted index for array ( #33452 )
...
issue: https://github.com/milvus-io/milvus/issues/27704
---------
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2024-05-31 09:47:47 +08:00
7102403a6b
fix: Add Wrapper and Keepalive for CTraceContext ids ( #32746 )
...
See also #32742
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-05-07 10:05:35 +08:00
0c868b654d
fix: fix mmap enabled check in resource estimation ( #31536 )
...
issue: https://github.com/milvus-io/milvus/issues/31569
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-25 14:29:07 +08:00
bdc70dfc6a
feat: Add global mmap enable configuration ( #31267 )
...
https://github.com/milvus-io/milvus/issues/31279
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-18 15:17:10 +08:00
17b4938985
fix: mmap does not work on index load ( #31228 )
...
issue: https://github.com/milvus-io/milvus/issues/31101
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-03-17 23:27:03 +08:00
4082315bd0
enhance: Add `ParseCTraceContext` util function for tracing ( #30883 )
...
See also #29803
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-28 18:59:00 +08:00
8a219e0102
feat: Support knowhere trace using OpenTelemetry ( #30750 )
...
Issue: #21508
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
2024-02-28 12:29:00 +08:00
5143b8c793
enhance: Use dynamic pool for `NewLoadIndexInfo` ( #30489 )
...
See also #30445
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-02-04 14:19:05 +08:00
405877c8cd
fix: Use correct pools for all CGO methods in segments pkg ( #30274 )
...
See also #30273
This PR:
- Rename confusing `LoadIndexInfo` to `UpdateIndexInfo` for LocalSegment
- Use `DynamicPool` instead of `LoadPool` for `UpdateSealedSegmentIndex`
- Fix cgo call missing pool control
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-26 10:09:00 +08:00
e1258b8cad
feat: integrate storagev2 into loading segment ( #29336 )
...
issue: #29335
---------
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2024-01-12 18:10:51 +08:00
d6429933a7
enhance: make Load process traceable in querynode & segcore ( #29858 )
...
See also #29803
This PR:
- Add trace span for `LoadIndex` & `LoadFieldData` in segment loader
- Add `TraceCtx` parameter for `Index.Load` in segcore
- Add span for ReadFiles & Engine Load for Memory/Disk Vector index
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2024-01-10 21:58:51 +08:00
b251c3a682
enhance: add ctx for HandleCStatus and callers ( #29517 )
...
See also #29516
Make `HandleCStatus` print trace id for better logging
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-27 16:10:47 +08:00
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
dc89730a50
Support collection-level mmap control ( #26901 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
c8415453a1
Support prepare config for index ( #27920 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-10-25 19:54:12 +08:00
5db4a0489e
dynamic index version control ( #27335 )
...
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-25 21:39:27 +08:00
370b6fde58
milvus support multi index engine ( #27178 )
...
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-09-22 09:59:26 +08:00
300fef446b
Enable mmap for vector index ( #25877 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-10 13:59:15 +08:00
168f346620
Remove deprecated interface ( #25864 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-07-24 17:47:00 +08:00
04082b3de2
Migrate the ability to upload and download binlog to cpp ( #22984 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
e467e0d451
Seperate reading build and load params for diskann ( #23758 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-04-27 20:30:35 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
081572d31c
Refactor QueryNode ( #21625 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
congqixia
SimFG
Zhen Ye
zhenshan.cao
wei liu
Jiquan Long
Bingyi Sun
Cai Yudong
yah01
yah01
Gao
foxspy
xige-16
jaime