milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,149 Commits
141 Branches
117 Tags
560 MiB
Commit Graph

5522 Commits (8f1ba6a0bb0121a32cd5e1f4dba4a0747d4513ec)

Author SHA1 Message Date
Xiaofan e36fce1fd4
Add log for mqtt msgstream seek (#17640) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-20 15:18:13 +08:00
wayblink f2bd910df5
[skip e2e]Fix log mistake: WatchDmChannels -> WatchDeltaChannels (#17643) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-20 15:10:12 +08:00
congqixia 12b3a29db8
Fix segments info in drop virtual channel request is not set (#17641) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-20 13:40:12 +08:00
Xiaofan d70f483806
Support rocksmq dynamic lru cache size (#17626) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-20 10:56:12 +08:00
Xiaofan 516d91470f
Fix Delta log loss after compaction (#17634) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-19 20:12:11 +08:00
congqixia 50cecc65ed
Fix mock querynode server session not revoked (#17229) 
Revoke mock querynode server session when it's stopped
This PR reduces the running time of TestLoadBalanceIndexedSegmentsAfterNodeDown from 60+ seconds to less than 1+ seconds
Also related to #17212 #17215

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-18 18:22:12 +08:00
Xiaofan 144f7a2d80
Refine meta, remove redundant proxy data (#17631) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-18 12:50:11 +08:00
Letian Jiang b15c24a554
Acquire reference lock of compacted from segments during index building (#17619) 
- Accquire reference lock during index building
- Keep dropped segment record

Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-18 01:24:11 +08:00
groot 7c69f4b338
Fix bulkload failed bug caused by dropping collection (#17624) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-17 21:14:11 +08:00
groot 371a44f018
Fix a bug of bulkload (#17620) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-17 20:52:11 +08:00
congqixia aa4b42c74b
Remove wait before release collection/partition (#17623) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-17 19:00:11 +08:00
wayblink 074ec3060a
Support return dropped segments info in GetSegmentInfo rpc (#17617) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-17 18:24:12 +08:00
cai.zhang 141a3bbe00
Mark deleted before drop index (#17490) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-17 18:08:12 +08:00
bigsheeper 92d06b2e30
Purge memory by the memory state and try to purge after each search (#17565) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-17 17:46:10 +08:00
congqixia 785a5a757f
Use segment version instead of ref cnt (#17609) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-17 17:38:12 +08:00
congqixia cc3ecc4bd5
Make querycoord channel allocator respect context (#17552) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-17 16:02:12 +08:00
Jiquan Long 0e29c37499
Forbid system fields in user schema (#17613) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-17 14:10:11 +08:00
yah01 f5fa93aa0b
Return err if failed to assign segments/channels to nodes (#17616) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-17 12:42:11 +08:00
SimFG 2008f60e50
Simplify the implementations of the ToPhysicalChannel and the ConvertChannelName func (#17605) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-17 10:22:11 +08:00
yah01 0f87763682
Fix LoadBalance may not update the shard leader (#17608) 
This happens probably with concurrent updating replicas,
some goroutines modify the nodes list of replicas,
and the others modify the shard leaders of replicas

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-17 08:58:10 +08:00
cai.zhang 53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-16 20:12:11 +08:00
yah01 3f42f5f345
Set the task state to TaskFailed if error occurs (#17598) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-16 19:18:11 +08:00
Jiquan Long 8e2797fe30
Print error when failed to broadcast message (#17595) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-16 18:32:10 +08:00
shaoyue 04f836f516
Fix s3 delete file (#17574) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-16 18:14:10 +08:00
Letian Jiang be4077421e
Filter duplicated sealed segments which are both online and offline (#17593) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-06-16 17:42:11 +08:00
Xiaofan 1f6fbf91b2
Fix pulsar unsubsribe issue (#17562) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-16 17:28:11 +08:00
Jiquan Long 2f66531fdf
Fix wrong IP distances (#17590) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-16 16:34:11 +08:00
yah01 7d5c8c5f38
Fix bug not remove offline node (#17560) 
The LoadBalance task won't remove the offline node if the node never load/watch any segment/dmchannel

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-16 16:00:10 +08:00
groot b5e62023ea
Return error when bulkload pending list hit limit (#17570) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-16 13:02:10 +08:00
wayblink eb5b0b7fc8
Move SegmentInfo out of VchannelInfo, leave Id only to decrease kv size. Get complete SegmentInfo through RPC (#17441) 
Resolves: #17233 #16047

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-16 12:00:10 +08:00
Xiaofan 54aeb077e9
Refine compaction selection poliy (#17486) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-15 23:14:10 +08:00
cai.zhang ea5041aec2
Acquiring the segment reference lock on task level (#17544) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-15 21:38:10 +08:00
Jiquan Long 875b6f88b0
Add channel information to flow graph node (#17349) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-15 21:04:10 +08:00
Ten Thousand Leaves 51a9e54b21
Fix a bug that causes infinite loop (#17550) 
This will cause AddNode() to keep its lock forever so DataCoord would just hang

issue: #17366
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-15 20:50:10 +08:00
xige-16 98b32f3eba
Improve time record log in search/query (#17545) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-15 20:48:10 +08:00
xige-16 dd483beefd
Delete the log that prints a lot of deleteMsg.timestamps (#17543) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-15 20:46:14 +08:00
Bingyi Sun d4fdfab818
Fix VDC-510 (#17554) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-06-15 19:58:10 +08:00
xing.zhao 76523b313b
Fix bug #17403 by double check collmeta prefix and alias prefix (#17548) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
 2022-06-15 19:14:10 +08:00
SimFG e825215c44
Remove the redundant 'for' in the TaskCondition (#17558) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-06-15 16:32:10 +08:00
Ten Thousand Leaves 65ab847815
Add segment reference lock for bulk load (#17457) 
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-15 12:20:10 +08:00
Jiquan Long e751663fdc
Fix err msg when the dimension of query vectors mismatch field (#17553) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-15 10:54:09 +08:00
congqixia f9553970f9
Add BindContext function for querycoord task scheduler (#17531) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-14 17:12:09 +08:00
congqixia ed3a1164dd
Handle wrapped grpc error in funcutil.IsGrpcError (#17547) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-14 17:10:10 +08:00
groot ab9b186f15
Change bulkload bufsize for row-based parser (#17473) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-14 16:18:09 +08:00
bigsheeper cdcdfa1ea5
Disable jemalloc and use malloc_trim instead (#17538) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-14 11:12:09 +08:00
xing.zhao e9dcda161f
Add aliases return value in describeCollection task (#17516) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: 赵星 <zhaoxing@zhaoxings-MacBook-Pro.local>
 2022-06-13 19:22:10 +08:00
congqixia a0d3f37c0d
Sync grpc DefaultKeepAliveTimeout to milvus.yaml (#17519) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-13 16:18:09 +08:00
XuanYang-cn e6225d923b
Cancel timers while adding new timer (#17511) 
See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-13 14:46:08 +08:00
congqixia d1de8cabdd
Fix %w verb for error wrapping in grpcclient (#17484) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-13 12:48:08 +08:00
xige-16 16f0815184
Add time record log on search/query execution path (#17497) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-13 11:42:09 +08:00
xige-16 36ad989590
Fix segOffset grater than insert barrier when mark delete (#17444) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-10 20:02:08 +08:00
Enwei Jiao a5b008acec
ignore getReplica's error when handle rebalanceTask (#17469) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-06-10 14:50:08 +08:00
jaime b64a467311
Revert "Use unified catalog interface to reconstruct metastore (#17042)" (#17480) 
This reverts commit 218326bafe.

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-10 13:10:08 +08:00
bigsheeper 2d9a52206d
Use jemalloc in QueryNode, DataNode and IndexNode (#17470) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-10 11:24:08 +08:00
XuanYang-cn 2b5340581c
Skip remove if reassigns to the original node (#17450) 
Fix ut race

See also: #15966, #17432

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-10 10:54:07 +08:00
Xiaofan 66f26943f8
Add Err code when task rollback (#17472) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-09 21:08:07 +08:00
Xiaofan 05a918e0ab
Refine logs (#17438) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-09 21:04:07 +08:00
congqixia 97a871cc82
Make querycoord segment allocator respect context (#17452) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-09 20:14:06 +08:00
Letian Jiang dfaed5acdd
Add QueryServiceAvailable field in ShowCollections (#17456) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-06-09 18:20:07 +08:00
yah01 a38790f586
Predict the loading memory usage more accurate (#17451) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-09 18:16:07 +08:00
Jiquan Long 2ca81620ea
Reduce lock operations when get dml stream (#17468) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-09 17:34:09 +08:00
groot 9210299706
Fix bulkload OOM issue (#17461) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-09 16:52:07 +08:00
xige-16 6d078ebe7a
Print more information when loading fields with different lines (#17448) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-08 20:16:06 +08:00
wayblink 25883840b2
Remove delete log in datanode because it may influence datanode performance (#17415) 
Resolves #16984

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-06-08 19:10:07 +08:00
jaime dbef4c6ec2
Align kafka consume from latest with puslar and rocksmq (#17433) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-08 19:00:07 +08:00
XuanYang-cn 4f66bcbb22
Fix DataNode processes event out of order (#17440) 
The probability is low so very unlikly to reproduce

See also: #15966

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-08 17:24:07 +08:00
Xiaofan 6426e3d037
Split Etcd save operation into small txns (#17386) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-08 15:08:07 +08:00
yah01 70f8bea4b4
Avoid growing slice as deserializing binlogs (#17421) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-08 11:46:06 +08:00
XuanYang-cn 1dbfebf827
Add debug log in removing assignment (#17427) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-08 10:44:06 +08:00
bigsheeper af108b5bda
Add malloc_trim after reduce search result (#17423) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-08 00:12:05 +08:00
groot f31be941bb
Fix bulkload row count bug for multi datanodes (#17384) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-06-07 16:56:07 +08:00
xige-16 b5c11a216d
Alter varChar type params's name to max_length (#17409) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-07 15:58:06 +08:00
Xiaofan a9ec1f8864
Fix compatibility of upgrade (#17406) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-07 14:52:06 +08:00
congqixia de8f2f76c5
Fix shardLeader cache with ErrConnect check (#17402) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-07 12:20:06 +08:00
jaime 218326bafe
Use unified catalog interface to reconstruct metastore (#17042) 
Co-authored-by: kejiang ke.jiang@zilliz.com
Co-authored-by: yun.zhang yun.zhang@zilliz.com

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-07 12:00:07 +08:00
bigsheeper f38637c227
Pass PlaceholderGroup pointer to prevent memory copy in SegCore (#17389) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-06 21:34:05 +08:00
zhenshan.cao 98e95275fe
Fix several code issues caused by carelessness (#17385) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-06-06 18:58:05 +08:00
bigsheeper 9f786dd752
Remove unnecessary bitset flip in Search and Query (#17387) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-06 17:38:05 +08:00
yah01 a2d2ad88bd
Make assigning segments faster (#17377) 
This improve the Load performance,
and let the LoadBalance fails fast, which allows us to retry it timely

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-06 16:52:05 +08:00
congqixia ac6394d0fd
Fix container/list removal in iteration (#17382) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-06 16:34:05 +08:00
Xiaofan af994b5e1c
Fix etcd watch error (#17364) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-06 16:26:06 +08:00
bigsheeper 45febac298
Extract lease of cProto in QueryNode (#17373) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-06 15:56:05 +08:00
XuanYang-cn c70af73005
Fix datacoord set wrong state for node registering (#17376) 
Fix datacoord datarace

See also: #17335

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-06 15:10:05 +08:00
bigsheeper 48cf63a2d7
Wait for flow graph to exit (#17341) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-02 19:48:04 +08:00
shaoyue 76eaa3fc50
Support access s3 through AWS IAM role (#17292) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-02 19:42:03 +08:00
SimFG 0986c29d7f
proxy tls support one-way authentication (#17348) 
Signed-off-by: xiyichan <2863768433@qq.com>

Co-authored-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>

Co-authored-by: xiyichan <2863768433@qq.com>
 2022-06-02 18:56:04 +08:00
Ten Thousand Leaves f5e63177d2
Fix an issue with bulk load with multiple DataNodes (#17324) 
issue: #17300
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-02 18:54:04 +08:00
yah01 4e83f37899
Improve load performance (#17273) 
Load segment's fields concurrently

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-02 17:56:03 +08:00
shaoyue 7c52a8c535
Add RESTful API for new methods, fix err when receive empty boy (#17260) 
create collection schema use struct instead of protobuf

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-06-02 17:40:04 +08:00
xige-16 8c69790383
Fix lost delete msg caused by loadSegment after watchDeltaChannel (#17308) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-02 16:56:04 +08:00
zhenshan.cao 5f44e45480
Add metrics on QueryNode and Proxy (#17328) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-06-02 16:06:03 +08:00
Jiquan Long adf3b14027
Refine channels management in Proxy. (#17334) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-02 15:34:04 +08:00
bigsheeper 7a59a80506
Remove segmentPtr lock and use collection lock instead (#17303) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-02 14:00:04 +08:00
XuanYang-cn 42f643e727
Make DataNode release rather than delete when reassign (#17293) 
1. Reassgin now will assign to the original Node if no other nodes
   avaliable
2. Make AddNode balance async: ToRealse + Reassign

See also: #16114, #17270

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-02 13:56:04 +08:00
yah01 cc69c5cdd3
Make Cluster interface's methods called outside public (#17315) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-02 13:16:05 +08:00
congqixia c76b4ade01
Fix data race in LoadSegmentsTask (#17342) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-02 12:52:03 +08:00
zhenshan.cao 2763efc9b0
Add cache of grpc client of ShardLeader in proxy (#17301) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-06-02 12:16:03 +08:00
jaime 5698bf4236
Add sasl configurations for kafka (#17323) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-02 12:12:03 +08:00
Jiquan Long e88ffb8a57
Enable marisa trie ut on MacOS (#17316) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-02 10:48:03 +08:00
bigsheeper e0cbacba59
Check ignorable error to prevent unnecessary panic (#17317) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-06-02 10:12:03 +08:00
yah01 f5bd519e49
Add retry mechanism for NodeDown LoadBalance (#17306) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-06-01 20:00:03 +08:00
xige-16 63a31ccb38
Remove error return value when removeSegment (#17310) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-01 13:18:02 +08:00
Jiquan Long b4f797506d
Fix err msg of invalid expression (#17296) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-01 10:00:07 +08:00
Jiquan Long 72f454e054
Fix 16961, make creating dml stream idempotent (#17302) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-05-31 22:58:02 +08:00
XuanYang-cn d920e5c915
Fix DataNode panic while allocating IDs (#17294) 
See also: #17270

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-31 18:02:03 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 16:36:03 +08:00
groot affcc2acad
Fix bulkload entity number bug (#17278) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-31 15:40:04 +08:00
bigsheeper f55fe6be3c
Merge historical replica and streaming replica in QueryNode (#17217) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-31 13:42:03 +08:00
Xiaofan 9579a645c6
Support compile marisa on Macos (#17261) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-31 13:28:02 +08:00
Jiquan Long 6bd786bf50
Reduce default seal propotion (#17287) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-31 12:02:04 +08:00
cai.zhang f1bad98f00
Fix indexcoord ut (#17282) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 12:00:09 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-30 19:50:04 +08:00
congqixia cecbd40a22
Fix Data race of loadType in querynode/collection.go (#17275) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-30 19:16:02 +08:00
bigsheeper 11834a0948
Improve query log in QueryNode (#17268) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-30 17:18:03 +08:00
zhenshan.cao b0d54ed0af
Rename Client to GrpcClient in mock package (#17230) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-30 17:16:08 +08:00
Jiquan Long ac2cadc83b
Refine segment type logs to string (#17248) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-05-30 15:30:01 +08:00
yah01 b09359b12f
Remove useless collection ID in error message (#17269) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-30 14:40:01 +08:00
SimFG 731870211a
Support role-based access control (only pb) (#16987) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-05-28 00:04:01 +08:00
congqixia 55f1e03ca7
Allow DataNode clean vchannel with meta error without panicking (#17247) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-27 16:20:00 +08:00
zhenshan.cao ad9276a440
Add metrics for proxy and querynode (#17205) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-27 14:12:01 +08:00
xige-16 e196caf604
Force reclamation of memory after loading (#17244) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-27 12:50:00 +08:00
xige-16 3a63d6c98e
Fix load timeout in chaos (#17241) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-26 22:28:03 +08:00
Letian Jiang f2a27e0e64
Retry GetShardLeaders until service available or timeout (#17183) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-26 20:28:02 +08:00
yah01 5872c5afb6
Fix updating shard leaders may lost some modifications (#17218) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-26 17:14:02 +08:00
xige-16 56778787be
Reverse data from scalar index (#17145) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-26 14:58:01 +08:00
groot 3928da6493
Refine calc distance function (#17161) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-26 14:34:01 +08:00
xige-16 7d810ac159
Fix query empty when insert same pk after deletion (#17222) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-25 22:06:00 +08:00
Jiquan Long e44afc7aa3
Fix ut: wait service ready after run Proxy server (#17208) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-05-25 18:56:00 +08:00
congqixia 7409bfc56d
Make allocateNode run async in case of block offline event (#17185) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-25 18:53:59 +08:00
Jiquan Long c12ee2f5cb
Fix flow_graph_insert_buffer_node.go code style (#17216) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-05-25 17:03:59 +08:00
congqixia e06d6d7cc6
Returns collection instead of copy to make leaderMutex work (#17206) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-25 15:20:00 +08:00
yah01 de0ba6d495
Fix GetQuerySegmentInfo() returns incorrect result after LoadBalance (#17190) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-05-25 15:17:59 +08:00
Ten Thousand Leaves 5e1e7a6896
Fix an issue that causes flush hanging (#17169) 
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-25 14:34:00 +08:00
congqixia 37d7d7baf8
Fix node state data race in querycoord (#17198) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-25 10:43:59 +08:00
Jiquan Long 4293b640b2
Remove confusing log in segment id assigner (#17195) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-05-25 10:26:00 +08:00
Jiquan Long 75ca64f8c7
Refine task type logs to string (#17196) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-05-25 03:08:00 +08:00
xige-16 a8829554eb
Merge multiple sets of expr's optypes in diff dirs (#17192) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-24 21:56:00 +08:00
zhenshan.cao ec1103ca27
Simplify the merge logic of searchTask (#17194) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 21:27:59 +08:00
bigsheeper a4ea2fb18a
Handle errors in DataNode and QueryNode flow graph (#17096) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-24 21:11:59 +08:00
Bingyi Sun ffaead6ad9
Add load meta in hand off task. (#17179) 
issue: #16842

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2022-05-24 18:24:00 +08:00
congqixia a0cef3a0a2
Fix sessionutil log always datanode (#17184) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-24 16:05:59 +08:00
zhenshan.cao d05c24a0ff
Delete unused code (#17182) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 14:49:59 +08:00
zhenshan.cao 2512e668f7
Update timestmap according to gracefultime for Bounded Consistency (#17171) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 12:05:59 +08:00
zhenshan.cao ea8e1623cd
Acquire nq from placeholdergroup incase of nq is zero (#17168) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-24 10:05:58 +08:00
zhenshan.cao cce533c246
Fix unitest fail, change default index slice to 16 (#17162) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-05-23 17:44:25 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-23 16:41:58 +08:00
cai.zhang 3685ff9e0c
Mark indexing task to retry when minio occurs error (#17153) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-23 15:47:58 +08:00
Xiaofan 65269a4744
Remove rocksmq reader (#17149) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-23 13:55:59 +08:00
jaime a20e0dfc74
Refactor kafka msgstream implementation (#17069) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-05-23 12:39:58 +08:00
Enwei Jiao d28a2db46c
move arrow from storage to core (#17061) 
Signed-off-by: Enwei Jiao <jiaoew2011@gmail.com>
 2022-05-22 20:03:58 +08:00