b30c9d4f60
Support Bulk Load in Milvus 2.2 ( #18982 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-09-26 18:06:54 +08:00
d7f38a803d
Separate some proto files ( #19218 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-09-16 16:56:49 +08:00
5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector ( #18636 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-09-16 09:56:47 +08:00
626854cf0c
Refactor QueryCoord ( #18836 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Wei Liu <wei.liu@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2022-09-15 18:48:32 +08:00
c924f73105
Refactor for IndexCoord to support cloud ( #18643 )
...
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
2022-08-25 15:48:54 +08:00
d9c7519107
refactor compaction concurency logic ( #18660 )
...
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
2022-08-23 15:50:52 +08:00
c267ad612d
Support Role-Based Access Control II ( #18528 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-05 16:28:35 +08:00
ff0200210a
Support Role-Based Access Control ( #18425 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-08-04 11:04:34 +08:00
0ebe407221
Make DataNode not panic if stale segment is not found during flush ( #18322 )
...
/kind improvement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-07-20 15:06:30 +08:00
65ab847815
Add segment reference lock for bulk load ( #17457 )
...
So we can support DataQueryable and DataIndexed states
issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-06-15 12:20:10 +08:00
731870211a
Support role-based access control (only pb) ( #16987 )
...
Signed-off-by: SimFG <bang.fu@zilliz.com>
2022-05-28 00:04:01 +08:00
f2a27e0e64
Retry GetShardLeaders until service available or timeout ( #17183 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-05-26 20:28:02 +08:00
5e1e7a6896
Fix an issue that causes flush hanging ( #17169 )
...
issue: #16628
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-25 14:34:00 +08:00
08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks ( #17022 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
43b36b4e5f
Make proxy update cache with NotShardLeader status code ( #16940 )
...
See also: #16926
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-05-13 11:35:53 +08:00
63385b7c12
Make data queryable and data indexed states as independent flags ( #16837 )
...
/kind improvement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-12 19:23:53 +08:00
1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks ( #16725 )
...
issue: #16607
/kind enhancement
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-05-05 21:17:50 +08:00
804fbb6e27
Fix panic: Set Base of GetReplicas request ( #16578 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-22 14:47:42 +08:00
69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord ( #16470 )
...
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-04-20 16:15:41 +08:00
442c73a544
Support passing channel names to DataNode ( #16424 )
...
/kind feature
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-04-12 13:25:34 +08:00
4e12271d79
Support login with username and password ( #15656 ) ( #16341 )
...
Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
2022-04-11 19:49:34 +08:00
0d54697d42
Implement ReportImport related logic. ( #16312 )
...
/kind feature
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-31 13:51:28 +08:00
ba37531456
Add support for loading multiple indexes ( #16138 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-30 21:11:28 +08:00
478890a700
modify import proto ( #16155 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-23 19:29:25 +08:00
1c4b949a1d
modify import rpc interface ( #16129 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
2078b24dca
Prepare for import ( #16083 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
89356ebcb9
Modify import rpc interfaces ( #16006 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
5fdef607d5
Add rpc interfaces for import ( #15930 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
cc5bee4684
Stop handoff if segment has been compacted ( #15181 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-01-13 21:49:35 +08:00
8a63dba149
Add consistency level field in collection meta ( #13766 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-12-21 19:49:02 +08:00
6f8ea8cc7b
Fix querycoord panic at getRecoveryInfo ( #12983 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-12-13 10:29:27 +08:00
85af0b571d
Add dropped segment state ( #11499 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-09 18:41:37 +08:00
c3f0c5a3ff
Add compaction interfaces in proxy ( #11431 )
...
issue: #9904
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2021-11-09 14:47:02 +08:00
09f8fa1040
Add query coord watch delta channel ( #11218 )
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-05 14:47:19 +08:00
b567a8e5b5
Add SystemInfo msgType for getMetricRequest ( #11151 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-11-04 10:58:11 +08:00
3a4df03b70
Add SealedSegmentsChangeInfo and changeInfoMsg ( #9479 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-10-09 14:45:00 +08:00
c17f845ab3
Add StartPositions fields in proto ( #8657 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 14:13:58 +08:00
b71bffe26f
Replace ChannelID with ShardName in DML Request ( #8539 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-09-27 10:01:59 +08:00
ccb57ba4b2
add collection alias ( #7255 )
...
Signed-off-by: Yusup <yusup@lsgrep.com>
2021-09-18 11:13:51 +08:00
eb818d05ab
Remove vector fields from output fields ( #6112 )
...
* Remove vector fields from output fields
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
* Add empty_collection error_code
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-25 16:52:11 +08:00
189ac881f3
Fix bugs ( #5676 )
...
* Remove redundant session startup
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Register datanode after start success
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* fix meta snap shot
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix datanode message stream channel
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Fix bugs when drop empty collection
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Fix bug of getting pchan statistics from task scheduler
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Fix i/dist/dataservice test code
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Fix epoch lifetime not applied
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* fix datanode flowgraph dd node
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Fix handle datanode timetick bug
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Remove repack function of dml stream
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* fix proxynode
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Apply extended seal policy
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* add check for time tick
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix check
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix the repack function of dml stream
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Fix the bug when send statistics of pchan
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Fix the repack function when craete dml stream
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* fix bugs
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix describe collection
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix bug when send timestamp statistics
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* fix data node
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Add length check before flush request
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* add log for data node
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix SaveBinlog bugs
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Add more log in datanode
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Put SegmentState.Flushing as the last one in enum to fit the client
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Fix params in GetInsertBinlogPaths
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Rename policy
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Remove unused ddl functions and fields
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Remove pchan when drop collection
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Add balanced assignment policy
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* fix master ut
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Add lock in session manager
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* add log for debug
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix some logic bug and typo
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Fix recover bugs
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Get collection scheme of a specific timestamp
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Change CheckPoint to SegmentInfo in VchannelInfo
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Recover Unflushed segment numOfRows
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Fix dataservice unit tests
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:11 +08:00
3662b3f0e5
Refactor dataservice ( #5400 )
...
Pick up grpc calls to grpc_handler.go and refactor code format
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-06-15 15:59:01 +08:00
cdbc6d2c94
Refactor query node and query service ( #5751 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
f18dfb4ff6
Add RetrieveTask implementation ( #5313 )
...
Resolves : #5257
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-05-20 07:02:31 +00:00
88f5642603
Add plan proto and support basic boolean expr parser ( #5088 )
...
**What type of PR is this?**
- [x] Feature
**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
like `A > 3 && not B < 5 or C in [1, 2, 3]`.
2. Defines `plan.proto`, as Intermediate Representation (IR)
used between go and cpp.
3. Support expr parser, convert predicate expr to IR
in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
060f7ca0d2
Add get index build progress proto ( #5049 )
...
Add get index build progress proto.
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-27 07:46:45 +00:00
82ccd4cec0
Rename module ( #4988 )
...
* Rename module
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
a54097708d
Adjust the DropIndex logic of the indexservice
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-04-16 15:37:13 +08:00
f3649f0419
Refactor interface and proto
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
ca916a5c47
Add ci nightly
...
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
2021-03-11 14:14:29 +08:00
Ten Thousand Leaves
SimFG
bigsheeper
Bingyi Sun
cai.zhang
Enwei Jiao
Letian Jiang
zhenshan.cao
XuanYang-cn
yah01
congqixia
codeman
Jiquan Long
groot
xige-16
godchen
Yusup
yukun
sunby
FluorineDog
Xiangyu Wang
ThreadDao