milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
21,292 Commits
139 Branches
115 Tags
576 MiB
Commit Graph

684 Commits (74752f2ec76eb40f04a83ff484505e34806d7ffa)

Author SHA1 Message Date
aoiasd 44e5daae3a
Fix querynode read quota (#24412) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-06-08 17:24:36 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
xige-16 732fe54775
Support partition Key (#24047) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-06 10:24:34 +08:00
yihao.dai 0fb9131973
Improve rate limiter log (#24459) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-30 11:23:28 +08:00
yihao.dai 1471da846d
Fix panic due to collection not exist in quota effect (#24320) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-24 20:53:30 +08:00
cai.zhang d16e18fd34
Add output fields for search/query results (#24302) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-23 10:19:26 +08:00
cai.zhang 008285f849
Support dynamic schema for create collection (#24176) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-18 09:33:24 +08:00
yihao.dai 1a3dca9b5e
Fix dynamic partitions loading (#24112) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-18 09:17:23 +08:00
wei liu 2935e9f767
fix alter collection merge multi props policy (#24147) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-17 16:05:22 +08:00
wei liu 6cfb4559d2
fix deny writing use wrong errcode (#24152) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-17 14:23:23 +08:00
yihao.dai 7384d83d2c
Support rate limit based on growing segments size (#24121) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-17 09:57:22 +08:00
SimFG ef82e0f1a4
Fix the error when a role removes an already deleted user (#24049) (#24109) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-05-16 10:15:22 +08:00
smellthemoon 8a85dd6869
Support Default Value (#23448) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-15 16:15:21 +08:00
wei liu 4c956fab73
enable config collection level rate limit (#24012) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-12 18:13:26 +08:00
smellthemoon ab95754069
Remove debug log (#24030) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-11 14:09:19 +08:00
smellthemoon f1adb1c06d
can only use the topic in the list when preCreatedTopicEnabled (#23913) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-10 13:47:19 +08:00
smellthemoon 146050db82
Fix some wrong ut (#23990) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-10 09:31:19 +08:00
yihao.dai f1cc31ceee
Fix creating partition is not atomic (#23823) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-08 10:18:39 +08:00
Xiaofan 87d790f052
Fix upgrade casue panic (#23833) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-05-02 14:06:37 +08:00
yihao.dai 4b26c0afb3
Add collection in querynode when sync new partitions (#23825) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-01 16:36:38 +08:00
yihao.dai 5a9a86ba74
Fix quota center assignment to entry in nil map (#23819) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-29 18:22:38 +08:00
Jiquan Long c955c26656
Optimize log when rootcoord goroutine quit (#23698) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-29 16:06:42 +08:00
Jiquan Long 156eba846f
Fix synced timetick not updated properly (#23697) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-29 16:02:38 +08:00
smellthemoon 9d35a5f943
Fix num of partitions is wrong (#23780) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-28 15:20:36 +08:00
wei liu 537e068fb9
fix qupta_center calculate negative memory factor for rate limit (#23750) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-28 11:32:35 +08:00
yihao.dai 017b15af6a
Fix disk quota when there's no binlog file (#23776) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-28 11:10:35 +08:00
yihao.dai 8c060215e9
Add collection number quota per DB (#23656) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-28 11:02:35 +08:00
smellthemoon 912cf4ef0f
Change some configurations, include change the defaultChannelNum to 16 (#23617) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-27 14:26:35 +08:00
yihao.dai ed8836cd15
Add disk quota at the collection level (#23704) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-26 21:52:36 +08:00
wei liu 4fb8919a97
support collection level rate limit (#22767) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-25 15:54:35 +08:00
zhenshan.cao 03ec804e68
Change default shard number to 1 (#23586) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-21 07:08:32 +08:00
Jiquan Long adfa415b74
Optimize log trace (#23431) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-17 11:00:30 +08:00
cai.zhang 43a9e175a3
Exit component process when session key is deleted (#21658) (#22164) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-04-12 20:12:28 +08:00
Jiquan Long d85f673a95
Fix time tick lag too much (#23265) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-09 16:40:29 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
Jiquan Long 8530dfadb4
Fix metric RootCoordInsertChannelTimeTick (#23236) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-06 15:30:29 +08:00
smellthemoon ce91f61e5a
Pre-create topics (#21907) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-03 16:44:23 +08:00
zhenshan.cao 1287ca699a
Refine usage of TimeRecorder.Record (#23142) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-30 18:56:22 +08:00
yah01 081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00
XuanYang-cn 93bc805933
Enhance ID allocator in DataNode (#22905) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-23 19:43:57 +08:00
SimFG e8f8c1b445
Check the collection num when creating the collection (#22946) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-03-23 16:47:57 +08:00
yihao.dai 1f718118e9
Dynamic load/release partitions (#22655) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-20 14:55:57 +08:00
Jiquan Long 556c1cfc8a
Fix confused timetick log (#22733) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-14 09:55:54 +08:00
yah01 fa2c1c1f40
Fix RootCoord double updates TSO (#22715) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 18:01:53 +08:00
yah01 e1cb307690
Refine RootCoord services error handle (#22689) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 14:29:53 +08:00
yah01 be33911205
Refine RBAC init error handle (#22537) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 10:59:53 +08:00
wayblink 28390dfe09
Add bulkinsert integration test (#22621) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-03-10 11:03:52 +08:00
yah01 90a5aa6265
Refine errors, re-define error codes (#22501) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-09 15:47:52 +08:00
Jiquan Long e3c11069eb
Fix ddl tt go back (#22617) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-08 10:13:51 +08:00
jaime d126f06946
Decouple mq module from internal proto definition (#22536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-03-04 23:21:50 +08:00
congqixia 51cb8590a1
Use errors.Wrap and Combine so retry error could check `errors.Is` (#22520) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-02 17:25:48 +08:00
Enwei Jiao 697dedac7e
Use cockroachdb/errors to replace other error pkg (#22390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-26 11:31:49 +08:00
zhenshan.cao e768437681
Correct usage of Timer and Ticker (#22228) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-02-23 18:59:45 +08:00
Jiquan Long 3cb9a44ed8
Add license to rootcoord module (#22301) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-22 11:37:45 +08:00
Jiquan Long d9ca06eb4e
Fix max partition number don't work (#22216) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-16 16:34:38 +08:00
wei liu 13b508103e
refine mock querycoord (#22198) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-16 15:38:34 +08:00
cai.zhang 66b3566ac1
Update component state to healthy after start (#22118) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-10 17:40:32 +08:00
yah01 b1f31da77a
Fix activate standby server ignores all errors (#22073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-09 15:24:31 +08:00
Jiquan Long a548ce1cf1
Change default max shards num to 64 (#22016) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-07 14:43:57 +08:00
Jiquan Long 004a8c25ca
Remove hardcode max shards num (#22004) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-07 10:17:53 +08:00
Xiaofan 2213ed7bed
Fix bulkload segment can not be flushed (#21919) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-02 13:11:51 +08:00
Bingyi Sun b12740e108
Remove a goroutine when creating a proxy client (#21904) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-02-01 15:49:50 +08:00
Xiaofan f1daef22a1
Fix failed bulkload segment marked as sealed (#21799) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-31 12:41:53 +08:00
Jiquan Long 270c5b0502
Refine log when initialize dml channels (#21863) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-30 14:05:54 +08:00
cai.zhang bcae97b125
Register after start to prevent there are tow coordinator at the same time (#21641) (#21707) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-30 11:11:50 +08:00
jaime d7bb908148
Fix rename collection data race (#21827) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-24 10:01:46 +08:00
Xiaofan 9258f7059d
Fix Bulkload ut print too much result (#21821) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-19 21:31:44 +08:00
congqixia 5986106037
Make paramtable init only once (#21782) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 14:53:44 +08:00
jaime 1dcd9eeb79
Support rename collection (#21693) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-19 14:13:43 +08:00
groot 0af53d1e2e
Fix bulkinsert report bug (#21778) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-01-18 10:33:43 +08:00
yah01 6e66c10114
Fix check time tick lag with no channel (#21737) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-16 18:49:43 +08:00
congqixia 5f3d3dc4fc
generate-mockery for querycoordv2, querynode and rootcoort (#21714) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-16 13:59:42 +08:00
Enwei Jiao 90d9e165d4
Fix some configs not shown (#21653) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-13 15:31:41 +08:00
wayblink 6a722396bd
Integration test framework (#21283) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-12 19:49:40 +08:00
SimFG b307ee8c07
Make the rbac error message clearer (#21645) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-01-12 13:55:42 +08:00
Jiquan Long 9fd9bed2b9
Remove collection meta after GC finished (#21595) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-12 09:55:42 +08:00
groot 9f8095996b
Report bulkinsert progress (#21612) (#21638) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-11 17:37:44 +08:00
cai.zhang e127cf7b99
Reset indexpb for upgrade (#21620) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 14:35:40 +08:00
groot df0ffd08ce
Update segment id for import task (#21621) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-11 11:55:39 +08:00
Ten Thousand Leaves 23ceb9c638
Remove index checking phase for bulk insert (#21578) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-09 21:15:38 +08:00
bigsheeper 79535931ab
Return error code when deny to read/write (#21561) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-01-06 14:31:37 +08:00
cai.zhang aa203acfb3
Low IndexCoord weight (#21548) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-06 14:21:37 +08:00
jaime 58b79eb74c
Add based on timetravel GC for snapshot KV (#21417) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-04 21:37:35 +08:00
cai.zhang e5f408dceb
Merge IndexCoord and DataCoord (#21267) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-04 19:37:36 +08:00
yah01 7b39873ae0
limit the frequency of GetMetrics() log (#21514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-01-04 17:39:35 +08:00
Jiquan Long ff2a68e65a
Fix collection not exist when tried to do recovery (#21471) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-04 16:37:35 +08:00
bigsheeper c187de7754
Show trigger reason when deny to read/write (#21368) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-12-30 18:35:32 +08:00
Ten Thousand Leaves 30b86275ac
Make BulkInsert free DataNode workload when import segments are flushed (#21253) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-28 11:11:30 +08:00
Xiaofan 3667bb09b8
Avoid Sync when the segment open (#21397) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-28 10:51:34 +08:00
wayblink 28b44b781f
Update healthz to support standby state and fix rootcoord active-standby (#21078) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-12-27 12:47:34 +08:00
aoiasd de0ab9e2cf
Refactor showConfigurations to allow return global config rather than only return config of this component (#21063) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-09 14:31:21 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
groot 33a102c6fc
Refine bulkinsert (#20986) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-12-07 14:53:18 +08:00
jaime 039e9ce4bb
Refine msgstream interface (#20832) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-05 20:55:17 +08:00
Enwei Jiao 2ecdb4ba4a
Etcd config source support TLS (#20874) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-30 18:23:15 +08:00
Jiquan Long 5a6a92d603
Refine log level (#20896) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-30 14:11:15 +08:00
SimFG c6fc1511aa
Update the error message for the rabc method (#20740) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-29 20:05:14 +08:00
XuanYang-cn 7fbe0c4df2
Remove mockTestKv in metatable test (#20689) 
See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-25 11:07:12 +08:00
jaime 0a5270e8c0
Fix alter collection hang (#20694) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-21 10:09:10 +08:00
Enwei Jiao c05b9ad539
Add event dispatcher for config (#20393) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 18:59:09 +08:00
smellthemoon 7325b3e1c3
Substitute traceid for msgid in rpc (#20450) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-14 15:29:06 +08:00
Xiaofan af66a0b621
Fix Time Tick session2ChanTs map use the same nodeID (#20537) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-14 14:41:11 +08:00
groot 53850e340c
Dont expire pending import tasks (#20431) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-10 10:25:08 +08:00
Jiquan Long 0edb49209d
Fix rootcoord metrics (#20427) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 17:49:04 +08:00
Jiquan Long cc9dc0f0fa
Fix log of RootCoord (#20412) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-09 15:07:03 +08:00
groot 39a6397043
Fix an error message (#20403) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-09 14:49:03 +08:00
Jiquan Long a2bf5022cb
Fix log of alter collection (#20413) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-11-08 22:51:03 +08:00
SimFG 9c6436d72d
Add time log for methods of starting the node (#20313) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-08 20:13:03 +08:00
Enwei Jiao 7e56e7a976
Fix datarace at Setlogger (#20376) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-08 14:33:03 +08:00
groot b847c425e1
Fix listImportTasks bug (#20301) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-07 17:11:02 +08:00
jaime 709a0a94e9
Add some msgstream metrics (#20296) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-07 10:15:02 +08:00
Enwei Jiao 956c5e1b9d
Make Params singleton (#20088) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-04 14:25:38 +08:00
XuanYang-cn 22fb1d3b93
Remove dup mock_kv.go and mock_txn_kv.go (#20267) 
Replace them with mockery generated mocks in kv/mocks

See also: #19289

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-11-03 14:41:35 +08:00
bigsheeper 51acba7d99
Add role name to ttDelay metric (#20239) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-02 10:55:34 +08:00
groot cd3b3d3da6
Only allow one file for row-based import (#20234) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-02 10:23:35 +08:00
wgcn bf5fb3cd99
improve show collection (#20124) 
Signed-off-by: wgcn <1026688210@qq.com>
 2022-10-31 13:25:35 +08:00
cai.zhang bcffb3c389
Check indexes in indexcoord instead of proxy (#20193) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-10-31 11:39:33 +08:00
bigsheeper bddd47c60f
Prevent limiter burst overflow (#20153) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-28 19:05:32 +08:00
Jiquan Long 89541a6f68
Revert parallel ddl (#20118) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-28 13:25:34 +08:00
bigsheeper b949fa0ea9
Increase default ttProtection max delay (#20125) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-27 17:37:32 +08:00
groot bee66631e3
Refine bulkload (#19671) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-10-27 16:21:34 +08:00
bigsheeper f0c616050a
Reduce and improve quota center log (#20058) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-25 19:39:30 +08:00
Jiquan Long 290b902ab0
Fix rootcoord too frequent useless log (#20044) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 14:19:30 +08:00
Xiaofan 1f170ce1ba
Refine kafka consumer (#19846) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 13:23:30 +08:00
Jiquan Long db4ede217f
Fix index not exist (#20027) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 10:01:30 +08:00
Jiquan Long 7a13b1c329
Fix panic when stop rootcoord (#20019) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-25 09:51:30 +08:00
Jiquan Long 14ddfe679f
Fix failed to show partitions (#20007) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-24 15:57:29 +08:00
Xiaofan 868bdd504d
Fix flaky unit test (#20006) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-24 11:51:29 +08:00
Jiquan Long 308a30fc46
Make dropping collection atomic (#19969) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-22 15:19:28 +08:00
Jiquan Long a1a74e3891
Optimize ddl case when collection not exist (#19950) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-21 16:37:29 +08:00
smellthemoon 581e1d8c35
Fill MsgBase ALL (#19912) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-10-21 15:57:28 +08:00
Xiaofan 3d3289ce3e
Fix collection metrics leakage (#19944) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-21 10:13:28 +08:00
wayblink dbd60757c9
Support pass start_ts and end_ts through Import API (#19810) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-20 12:15:27 +08:00
bigsheeper db0486c870
Fix missing import limit and rename functions in quotaCenter (#19907) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-19 21:43:30 +08:00
bigsheeper 99df2d9b54
Add binlogSize and ttDelay metrics monitor (#19872) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-19 15:13:26 +08:00
Ten Thousand Leaves 762045adff
Fix an issue where working task might get cleaned up (#19888) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-10-19 13:23:28 +08:00
jaime 59bcbf0cf6
Support check health for the service level (#19779) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-18 13:39:26 +08:00
bigsheeper 760e9c5df2
Enable rate limit and update quota unit (#19816) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-17 16:23:25 +08:00
SimFG a55f739608
Separate public proto files (#19782) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-10-16 20:49:27 +08:00
bigsheeper a92b242ecd
Fix ttFactor in QuoterCenter (#19802) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-16 09:17:24 +08:00
bigsheeper 3a5aaeb7ad
Fix read queue metrics and memory protection (#19787) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-14 18:05:24 +08:00
xige-16 a1db9038fb
Move disk index params to config file (#19714) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-10-14 17:51:24 +08:00
Jiquan Long cb51d410a2
Limit max shard num to 64 (#19776) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-14 14:51:23 +08:00
bigsheeper efdce464f5
Fix ttProtection.enabled and memProtection (#19756) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-13 16:15:25 +08:00
bigsheeper f1e330a997
Add read result rate protection (#19728) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-13 14:57:24 +08:00
Jiquan Long a8a074162f
Add meta migration tool (#19709) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-12 11:37:23 +08:00
bigsheeper e6c76aba0e
Force deny writing if disk quota exceeded (#19680) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-12 10:03:22 +08:00
jaime 10c03de628
Refine method name from BroadCastXx to BroadcastXx (#19700) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-11 21:07:23 +08:00
jaime 31db01b6ae
Support TTL for collection level (#19383) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-10-10 20:31:22 +08:00
wayblink 85e04d845c
Move GetComponentStates to milvus.proto, make it an external interface (#19594) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-10-10 15:55:22 +08:00
congqixia d64f318a80
Add collection schema in watch channel info (#19663) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-10-10 01:37:21 +08:00
Jiquan Long bdd6171231
Make creating collection/partition follow tt mechanism (#19628) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-09 12:32:57 +08:00
Xiaofan 6826964e60
Rename some interface and files in id allocator (#19605) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-09 10:06:58 +08:00
zhenshan.cao 0ac4bc32a5
Add logic of autoindex (#19570) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-10-08 15:38:58 +08:00
Ten Thousand Leaves ccecd6a76a
Fix malformed error log (#19589) 
issue: #19553

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-30 17:22:54 +08:00
Ten Thousand Leaves d827391af1
Fix on busy node list not updated when services restart (#19561) 
issue: #19544

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-30 16:54:55 +08:00
wayblink 54e2c079c4
MEP-30:Support basic coordinators active-standby mechanism #16901 (#19166) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-09-29 18:35:02 +08:00
Jiquan Long 1817627316
Fix create collection failed to assign channels (#19551) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-29 15:54:54 +08:00
Jiquan Long 5550f883de
Fix get collection by id (#19507) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-28 19:26:54 +08:00
Jiquan Long 73463d030d
Fix dml stream leakage in Proxy (#19450) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-27 19:18:54 +08:00
Ten Thousand Leaves 67b95597ed
Deprecated old RootCoord methods in Bulk Load (#19449) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-27 10:44:53 +08:00
Ten Thousand Leaves b30c9d4f60
Support Bulk Load in Milvus 2.2 (#18982) 
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-09-26 18:06:54 +08:00
bigsheeper 804c18df68
Update quota params (#19351) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-26 16:48:53 +08:00
Jiquan Long 4e521438c3
Optimize read ddl (#19352) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-24 12:42:51 +08:00
Jiquan Long 22477d4601
UnWatch channels if failed to create collection (#19390) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-23 16:56:50 +08:00
cai.zhang 17f5e3c7fe
Drop segment indexes when drop partition (#19022) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-23 09:36:51 +08:00
Jiquan Long cfd01a188c
Fix drop collection failed due to etcd txn limit. (#19362) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-22 17:36:52 +08:00
Jiquan Long 42d371fd41
Make ddl follow tt mechanism (#19279) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-09-21 15:46:51 +08:00
SimFG d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
bigsheeper 5cdd6ca251
Add quotaCenter, rateLimiter and rateCollector (#18636) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-16 09:56:47 +08:00
Jiquan Long a5e2d6b6fb
Refactor RootCoord (#18930) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
 2022-09-05 13:29:11 +08:00
SimFG 9cd19f5396
Return nil error when handling grpc request (#18955) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-02 21:12:59 +08:00
SimFG 69c0b2fb49
Improve the storage method of the grant info in the metastore (#18817) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-26 19:22:56 +08:00
cai.zhang c924f73105
Refactor for IndexCoord to support cloud (#18643) 
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-08-25 15:48:54 +08:00
codeman 90de312d4b
Separate create & update user in mysql (#18785) (#18786) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-24 14:32:56 +08:00
SimFG 226bcc16eb
Support concurrent situations for the metastore (#18792) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-24 10:02:52 +08:00
Zach 0c9a10e8f8
1. refine logging interfaces (#18692) 
2. adjust logs for query/search requests

Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>

Signed-off-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-08-23 10:44:52 +08:00
SimFG ce434b496e
Implement the mysql metastore of the rbac (#18704) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-23 10:26:53 +08:00
Jiquan Long 641afae22f
Watch channels with start positions (#18744) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-08-22 14:42:52 +08:00
codeman 70254c3ec5
Unified catalog interface for segment (#18289) (#18290) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-20 10:24:51 +08:00
congqixia b62cb82ebe
Simplify Dml-DeltaChannel mapping logic (#18702) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-18 14:24:50 +08:00
SimFG 447ce8ee48
Handle some apis about the default role (#18647) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-15 16:40:48 +08:00
aoiasd 8a70f2badb
Add ShowConfigurations for all remaining session (#18594) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-08-12 13:20:39 +08:00
codeman 6c3dbf0a07
Support meta storing on database (#17236) (#18541) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Signed-off-by: kejiang <ke.jiang@zilliz.com>
Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-08-11 12:12:38 +08:00
SimFG 1366f130a2
Add a specific `KeyNotExistError` err for the metastore `load` interface (#18582) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-10 17:20:41 +08:00
Jiquan Long e7b3bacbec
Refine catalog of partition & alias. (#18546) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-08-10 10:22:38 +08:00
SimFG c267ad612d
Support Role-Based Access Control II (#18528) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-05 16:28:35 +08:00
SimFG ff0200210a
Support Role-Based Access Control (#18425) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-08-04 11:04:34 +08:00
congqixia 318cdbebe9
Add heap for dmlChannels to get next channel (#18451) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-01 16:36:33 +08:00
wayblink c198e6c2d9
Add some debug log to explore ocassionally channel name mismatch issue in create collection (#18414) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-07-27 13:06:30 +08:00
jaime 9672eae62c
Unify metastore catalog interface (#17772) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-07-22 10:20:29 +08:00
congqixia f37e07b0f8
Use Get method of request in GetMetrics to avoid panicking (#18356) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-21 20:52:27 +08:00
codeman 5fda8261d0
Fix: Unable to list newly created credential users (#18117) (#18153) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-07-07 21:16:22 +08:00
cai.zhang 01fc411566
Handle etcd compacted error (#17886) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-30 18:54:19 +08:00
zhenshan.cao 0baeb609dc
Fix bugs in monitoring metrics (#17894) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-06-30 17:26:19 +08:00
yah01 ef6859a4d6
Refactor handoff handler (#17806) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-29 23:26:19 +08:00
codeman 95a3830dc6
fix password comparison too slow (#17874) (#17897) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-06-29 20:02:18 +08:00
xing.zhao 3904a78211
Remove genReqID function in import manager bacause the request ID is useless (#17853) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
 2022-06-28 18:20:18 +08:00
cai.zhang b883d1f5dd
Make sure indexes are successfully dropped when DropCollection (#17780) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-28 11:24:21 +08:00
cai.zhang 4efdd0929c
Get index status based on segment generation time (#17659) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-21 18:44:12 +08:00
cai.zhang 3610cc77bf
Fix bug for align segments (#17652) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-20 20:16:12 +08:00
cai.zhang f123d657ed
Fix bug for dropping index (#17635) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-20 15:22:13 +08:00
Xiaofan 144f7a2d80
Refine meta, remove redundant proxy data (#17631) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-18 12:50:11 +08:00
Letian Jiang b15c24a554
Acquire reference lock of compacted from segments during index building (#17619) 
- Accquire reference lock during index building
- Keep dropped segment record

Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-06-18 01:24:11 +08:00
groot 7c69f4b338
Fix bulkload failed bug caused by dropping collection (#17624) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-17 21:14:11 +08:00
cai.zhang 141a3bbe00
Mark deleted before drop index (#17490) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-17 18:08:12 +08:00
Jiquan Long 0e29c37499
Forbid system fields in user schema (#17613) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-17 14:10:11 +08:00
cai.zhang 53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-16 20:12:11 +08:00
Jiquan Long 8e2797fe30
Print error when failed to broadcast message (#17595) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-16 18:32:10 +08:00
groot b5e62023ea
Return error when bulkload pending list hit limit (#17570) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-16 13:02:10 +08:00
cai.zhang ea5041aec2
Acquiring the segment reference lock on task level (#17544) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-15 21:38:10 +08:00
xing.zhao 76523b313b
Fix bug #17403 by double check collmeta prefix and alias prefix (#17548) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
 2022-06-15 19:14:10 +08:00
Ten Thousand Leaves 65ab847815
Add segment reference lock for bulk load (#17457) 
So we can support DataQueryable and DataIndexed states

issue: #16607
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-06-15 12:20:10 +08:00
groot ab9b186f15
Change bulkload bufsize for row-based parser (#17473) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-06-14 16:18:09 +08:00
jaime b64a467311
Revert "Use unified catalog interface to reconstruct metastore (#17042)" (#17480) 
This reverts commit 218326bafe.

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-10 13:10:08 +08:00
Xiaofan 05a918e0ab
Refine logs (#17438) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-09 21:04:07 +08:00
jaime 218326bafe
Use unified catalog interface to reconstruct metastore (#17042) 
Co-authored-by: kejiang ke.jiang@zilliz.com
Co-authored-by: yun.zhang yun.zhang@zilliz.com

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-06-07 12:00:07 +08:00
Xiaofan af994b5e1c
Fix etcd watch error (#17364) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-06 16:26:06 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 16:36:03 +08:00
groot affcc2acad
Fix bulkload entity number bug (#17278) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-05-31 15:40:04 +08:00
bigsheeper 9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-19 10:13:56 +08:00
cai.zhang d6bb86dd46
Return buildID when creating index if the index alreay exists (#17045) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-17 15:21:57 +08:00
Ten Thousand Leaves 63385b7c12
Make data queryable and data indexed states as independent flags (#16837) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-12 19:23:53 +08:00
Jiquan Long be688d894a
Forbid multiple indexes on same field (#16875) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-10 21:07:53 +08:00
Ten Thousand Leaves 5afdf3d49f
Move locks back to meta table methods (#16848) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-09 20:47:52 +08:00
xige-16 8fcf349609
Fix create index failed after drop index (#16836) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-09 11:35:53 +08:00
Ten Thousand Leaves 6b2d3c3ec1
Make each dataNode only accept one bulk load task simultaneously (#16820) 
issue: #16674

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-07 14:05:52 +08:00
Ten Thousand Leaves 31ddff2056
Some minor fixes and improvements (#16814) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-06 21:35:51 +08:00
xige-16 7cc995d085
Prohibit creating diff indexes with same index_name on a collection (#16774) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-05 21:33:51 +08:00
Ten Thousand Leaves 1acd256481
Add DataQueryable and DataIndexed states for bulk load tasks (#16725) 
issue: #16607

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-05 21:17:50 +08:00
cai.zhang 2be46a01c0
Reduce the number of retries and add error log (#16754) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-05-05 09:31:51 +08:00
jaime c25b337c36
Remove AddTenant from meta table (#16717) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-05-05 09:23:51 +08:00
xige-16 515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
groot 21b5973253
Refine list import task api (#16690) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-28 17:21:47 +08:00
zhenshan.cao 0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
Xiaofan b657b10eb5
Remove useless log (#16624) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-27 10:01:47 +08:00
zhenshan.cao a3f7bb5f2d
Remove the collectionID label from metrics (#16571) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-26 17:29:45 +08:00
groot bc5e9ec929
Add list import tasks api (#16605) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-25 17:37:46 +08:00
Ten Thousand Leaves e66ac6a77c
Resolve an issue with bulk load where segments in flow graph replica are not updated (#16609) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:07:47 +08:00
Ten Thousand Leaves 242f08fb67
Refine import task related logic (#16601) 
issue: #16600

/kind bug

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-24 11:29:45 +08:00
groot a6a3b69d91
Report bulk load state (#16555) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-04-21 21:37:42 +08:00
codeman 91e84ffedf
fix: not create user root if exists, optimize log & imports (#16554) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-21 19:57:42 +08:00
Ten Thousand Leaves 289e468a7a
Make bulk load fully work (#16512) 
issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-20 14:03:40 +08:00
codeman 3a1b2cedd2
create default root user for authentication (#16545) (#16549) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-20 13:03:40 +08:00
codeman f5e1ded854
fix #16504 (#16523) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-19 16:35:39 +08:00
Xiaofan fce8f6cfab
Remove Timetick channel (#16449) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-17 23:47:36 +08:00
jaime ed3c29eae7
Support Kafka (#16149) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-12 19:47:33 +08:00
Ten Thousand Leaves 442c73a544
Support passing channel names to DataNode (#16424) 
/kind feature

issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-12 13:25:34 +08:00
codeman 4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00