b8318fcd7d
enhance: improve the handling for segcore error ( #29471 )
...
- fix lost exception details in segcore
- improve the logs of handling errors from segcore
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-26 14:06:46 +08:00
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
61fc822207
fix: creating growing segments may introduce many threads ( #29306 )
...
many growing segments may be created in a short time and there is no
restriction to the process, the CGO call will leave many threads
related: #29282
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-20 10:06:43 +08:00
dcb662d9ed
enhance: Refine C.NewSegment response and handle exception ( #28952 )
...
See also #28795
Orignal `C.NewSegment` may panic if some condition is not met, this pr
changes response struct to `CNewSegmentResult`, which contains
`C.CStatus` and may return catched exception
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-07 13:34:35 +08:00
fb089cda8b
enhance: Load raw data while scalar index doesn't have raw data ( #28888 )
...
issue: #28886
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-06 20:36:36 +08:00
d2f53aefa5
enhance: improve load speed ( #28518 )
...
This check rejects load request if running out the pool workers, but
small segment would be loaded soon, another segments would been loading
again after a check interval, which leads to slow loading for collection
Block the request by go pool
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-17 19:56:21 +08:00
d20ea061d6
Fix panic while forwarding empty deletions to growing segment ( #28213 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-08 16:42:21 +08:00
ece592a42f
Deliver L0 segments delete records ( #27722 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-07 01:44:18 +08:00
dc89730a50
Support collection-level mmap control ( #26901 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
9b6eeb46f1
Cache segment row num, size, and insert count to reduce CGO calls ( #28007 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-30 17:54:14 +08:00
af61a80f73
connect traceID for retrieve ( #27804 )
...
/kind improvement
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-10-19 19:22:10 +08:00
8f4aaa2da8
Remove ValidateIndexedFieldsData ( #27254 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-22 18:31:25 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
0a750408d0
Skip delta logs have been applied ( #26971 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-19 16:21:23 +08:00
bb6711f28c
Add ChunkCache: support get vector from storage ( #26142 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-09-15 10:21:20 +08:00
c6116d1819
Remove segment to LocalSegment type assertion ( #26931 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-09 10:35:16 +08:00
8330c18dc9
add log for loading segment( #26564 ) ( #26640 )
...
/kind improvement
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-08-31 12:03:00 +08:00
1cf6e00fa6
Improve segment manager interface ( #26637 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-29 15:46:27 +08:00
5dae6a654a
Protect segment from being released while query/search ( #26322 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 18:38:17 +08:00
3d8871b30e
Fix data race while updating segment version ( #26387 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 16:30:18 +08:00
b47a72bfcf
fix set dirty segment distribution to leader view ( #26180 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-11 11:21:32 +08:00
ca1349708b
Remove time travel ralted testcase ( #26119 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-10 18:53:17 +08:00
1055c90456
Add default retrieve limit ( #24782 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-08-10 14:11:15 +08:00
5c1f79dc54
Push down the limit operator to segcore ( #25959 )
...
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-08-01 20:29:05 +08:00
8b5e276361
Use different cgo pool for SQ and other operations ( #26021 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-01 09:19:05 +08:00
168f346620
Remove deprecated interface ( #25864 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-07-24 17:47:00 +08:00
cc71e065e3
Support load index after segment loaded ( #25567 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-18 10:51:19 +08:00
205a7c430a
Fix QueryNode panic while inserting to released segment ( #25495 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-12 19:48:29 +08:00
29b9b26de8
Improve get vector error msg ( #24838 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-27 16:34:45 +08:00
04082b3de2
Migrate the ability to upload and download binlog to cpp ( #22984 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2023-06-25 14:38:44 +08:00
a413842e38
Fix deleted data is still visible ( #24849 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-06-16 17:16:41 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
d3af451d92
Upgrade golangci-lint ( #24707 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-06-07 19:34:36 +08:00
014387fd94
Forbid to get quantized vector from ChunkManager ( #24334 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-24 23:03:27 +08:00
0c8045d66c
Fix crash caused by segment function concurrency ( #24351 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2023-05-24 19:23:27 +08:00
45aa9779e0
Retrieve segments concurrently ( #24245 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-19 18:19:24 +08:00
7e0d1492c7
Load delete from channel checkpoint ( #23961 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-05-09 19:10:41 +08:00
092d743917
Add support for getting vectors by ids ( #23450 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-04-23 09:00:32 +08:00
967a97b9bd
Support json & array types ( #23408 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
2023-04-20 11:32:31 +08:00
4fe363c4b2
Add KNN cgo pool ( #23526 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-04-19 23:56:31 +08:00
c9d0c157ec
Move some modules from internal to public package ( #22572 )
...
Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-04-06 19:14:32 +08:00
50dab2d394
Add tracing for retrieve ( #23033 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-03-29 18:10:02 +08:00
826c1fb850
Support mmap on QueryNodeV2 ( #23073 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-03-28 21:30:05 +08:00
081572d31c
Refactor QueryNode ( #21625 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
2023-03-27 00:42:00 +08:00
yah01
congqixia
cai.zhang
yah01
MrPresent-Han
yihao.dai
SimFG
wei liu
Enwei Jiao
xige-16
Jiquan Long
Bingyi Sun
jaime