a3cb8e2625
fix: Add atomic method to get collection target ( #29577 )
...
Related to #29575
Add `getCollectionTarget` method which is atomic when scope is
`CurrentTargetFirst` or `NextTargetFirst`
Also return error when executor finds no channel in target manager
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-29 09:04:46 +08:00
aa279db44c
enhance: remove flushed segmentInfo in WatchChannelRequest ( #29526 )
...
`WatchDmChannel` only need growing segment info, this PR removes fetch
segmentInfos when fill watch dml channel request.
---------
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-28 00:40:47 +08:00
1d6bcd1ded
enhance: speed up loading with many deletions ( #29455 )
...
the executor always fetches the latest segment info, so we could consume
from the latest checkpoint, which could save much time while deleted
many entities
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-25 20:58:45 +08:00
a0e1a1eb31
feat: support enable/disable mmap for index ( #29005 )
...
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
0a87724f18
enhance: remove merger for load segments ( #29062 )
...
remove merger as now QueryNode could load segments concurrently
fix #29063
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-12 10:48:37 +08:00
fab52d167b
fix: may miss stream delta while loading ( #28871 )
...
we consume the delta data from the lastest channel checkpoint while
loading segment,
this works well without level 0 segments, but now it may lead to miss
some delta data,
so we have to consume from the current target's channel checkpoint
related: #27349
---------
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-12-05 17:34:45 +08:00
043ac87be0
fix: Balance channel may cause channel not availble error ( #28829 )
...
issue: #28831
release old delegator before new delegator update it's distribution may
cause `channel not availble` error
This PR will block release old delgator before new delegator finish
`syncDistribution`
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-01 10:08:34 +08:00
70995383bf
enhance: modify log to avoid ambiguity and improve readability ( #28331 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-10 14:32:20 +08:00
ece592a42f
Deliver L0 segments delete records ( #27722 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-07 01:44:18 +08:00
90e2c63d9e
Fix getting incorrect CPU num ( #28146 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-11-06 06:02:16 +08:00
dc89730a50
Support collection-level mmap control ( #26901 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-02 23:52:16 +08:00
852be152de
Change task sourceID to stringer interface ( #27965 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-10-27 01:08:12 +08:00
e0222b2ce3
refine target manager code style ( #27883 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-25 00:44:12 +08:00
be980fbc38
Refine state check ( #27541 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-10-11 21:01:35 +08:00
a0ca982a52
Fix typo in priority name ( #27558 )
...
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-10-11 14:19:35 +08:00
41124f281a
Remove parser dependency ( #27514 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2023-10-08 15:05:31 +08:00
a8ce1b6686
Refine QueryCoord stopping ( #27371 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-27 16:27:27 +08:00
6539a5ae2c
Refine DataCoord status ( #27262 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-26 17:15:27 +08:00
26f06dd732
Format the code ( #27275 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-09-21 09:45:27 +08:00
168e82ee10
Fix panic while handling with the nil status ( #27040 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-09-15 10:09:21 +08:00
edde3cf1c7
Add tracer for querycoord tasks ( #27058 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-14 09:59:19 +08:00
fb0705df1b
Decouple basetable and componentparam ( #26725 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-09-05 10:31:48 +08:00
1a8cf5c415
Organize all mockery generation commands in Makefile ( #26826 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-09-04 21:19:48 +08:00
f9c060e0d2
Treat balance task with released source segment as stale ( #26453 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-22 17:12:22 +08:00
065d1a962e
Add sourceID output for task.String and fill reduce channel reason ( #26447 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-08-18 13:50:19 +08:00
9723787141
Calculate memory usage without page cache ( #26389 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-08-16 17:04:17 +08:00
7d61355ab0
Refactor log for Query ( #26310 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2023-08-14 18:57:32 +08:00
54c0e64059
Fix search on empty segments set bug ( #26136 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-08-08 11:17:08 +08:00
6f89620a43
remove pull target rpc from lock ( #26054 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-08-04 10:31:06 +08:00
df26b909f0
Fix panic while saw the failed task status ( #26082 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-08-02 13:47:05 +08:00
2180ef180c
Record only failed task error ( #26033 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-08-01 10:11:05 +08:00
a3e22786ed
Move meta store to kv catalog ( #25915 )
...
Signed-off-by: sunby <sunbingyi1992@gmail.com>
2023-07-31 13:57:04 +08:00
2b9ec565bb
Fix task executor can not dedup same task ( #25901 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-26 08:51:02 +08:00
a669440ee9
Allow querycoord executor load sealed segment with no index ( #25902 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-25 18:47:01 +08:00
efcaa07fe9
Implement Stringer for task related enums ( #25860 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 13:57:00 +08:00
3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map ( #25846 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-24 10:23:01 +08:00
1045c88102
Support replace indexed field in QueryCoord ( #25747 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-07-19 21:22:58 +08:00
224515eaa3
Add segment dist containing condition for loading segment ( #25736 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-19 15:02:58 +08:00
228c0df3f7
Skip record ChannelNotFound error while loading segment ( #25669 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-18 11:53:18 +08:00
49655d2f13
fix panic due to lack of indexInfo on querynode( #25362 ) ( #25613 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-07-17 19:23:19 +08:00
948d1f1f4a
Handle errors by merr for QueryCoord ( #24926 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-07-17 14:59:34 +08:00
b62c82af22
fix set target version in loading sealed segment ( #25603 )
...
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-07-14 20:00:31 +08:00
c56a26a556
Add SetMetricType and set metric only for shard leader ( #25285 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-07-04 10:20:25 +08:00
18df2ba6fd
[Cherry-Pick] Support Database ( #24769 )
...
Support Database(#23742 )
Fix db nonexists error for FlushAll (#24222 )
Fix check collection limits fails (#24235 )
backward compatibility with empty DB name (#24317 )
Fix GetFlushAllState with DB (#24347 )
Remove db from global meta cache after drop database (#24474 )
Fix db name is empty for describe collection response (#24603 )
Add RBAC for Database API (#24653 )
Fix miss load the same name collection during recover stage (#24941 )
RBAC supports Database validation (#23609 )
Fix to list grant with db return empty (#23922 )
Optimize PrivilegeAll permission check (#23972 )
Add the default db value for the rbac request (#24307 )
Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
2023-06-25 17:20:43 +08:00
c7dc1c067a
Assign metric type in load segment request ( #24917 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-16 16:50:41 +08:00
51404abe93
Add missing indexInfo in watchDmChannelsRequest ( #24696 )
...
Signed-off-by: chasingegg <chao.gao@zilliz.com>
2023-06-14 14:40:39 +08:00
41af0a98fa
Use go-api/v2 for milvus-proto ( #24770 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-06-09 01:28:37 +08:00
5f7099a9bd
Fix search failed due to metric type mismatch ( #24691 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-06 21:40:35 +08:00
b0ce4cff83
Assign metric type in search request when user has not explicitly assigned ( #24591 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-06-01 18:10:32 +08:00
b09e7aeaf7
support detailed task metrics( #23414 ) ( #24507 )
...
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-05-30 14:59:28 +08:00
congqixia
yah01
yah01
wei liu
smellthemoon
Xiaofan
SimFG
Enwei Jiao
Bingyi Sun
MrPresent-Han
yihao.dai
jaime
Gao