5c69c6902d
Refactor proxy channelTimeTick and fix search timeout ( #5760 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-06-15 16:36:36 +08:00
17b19233dc
add BroadcastMany, send DdMsg to collection channels only ( #5734 )
...
* add BroadcastMany, send DdMsg to collection channels only
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix master unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix time tick of masterservice
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix time tick unittest of masterservice
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* skip dataservice ut
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix format issue
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update timeticksync
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update timeticksync
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update SendDdXXXReq interface
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* update master unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* roll back unecessary change
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* enable ut
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:36:36 +08:00
189ac881f3
Fix bugs ( #5676 )
...
* Remove redundant session startup
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Register datanode after start success
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* fix meta snap shot
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix datanode message stream channel
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Fix bugs when drop empty collection
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Fix bug of getting pchan statistics from task scheduler
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Fix i/dist/dataservice test code
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Fix epoch lifetime not applied
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* fix datanode flowgraph dd node
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Fix handle datanode timetick bug
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Remove repack function of dml stream
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* fix proxynode
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Apply extended seal policy
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* add check for time tick
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix check
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix the repack function of dml stream
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Fix the bug when send statistics of pchan
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Fix the repack function when craete dml stream
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* fix bugs
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* fix describe collection
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix bug when send timestamp statistics
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* fix data node
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Add length check before flush request
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* add log for data node
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix SaveBinlog bugs
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Add more log in datanode
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Put SegmentState.Flushing as the last one in enum to fit the client
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Fix params in GetInsertBinlogPaths
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Rename policy
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Remove unused ddl functions and fields
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Remove pchan when drop collection
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
* Add balanced assignment policy
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* fix master ut
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Add lock in session manager
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* add log for debug
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* Fix some logic bug and typo
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
* Fix recover bugs
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Get collection scheme of a specific timestamp
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Change CheckPoint to SegmentInfo in VchannelInfo
Signed-off-by: sunby <bingyi.sun@zilliz.com>
* Recover Unflushed segment numOfRows
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
* Fix dataservice unit tests
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
2021-06-15 16:06:11 +08:00
a948b71e06
Support to send dd msg to all dml channels ( #5597 )
...
* send dd msg to dml channel
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* add dml_channels.go
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix race
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 16:04:48 +08:00
2cc330b117
Remove proxy service from master ( #5510 )
...
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove time sync
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
* remove proxy service from distributed/masterservice
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix unittest
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
* fix static-check
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 15:59:04 +08:00
bc46e4780d
Fixbug:index is not loaded in some cases ( #5633 )
...
* Fixbug:index is not created in some cases
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
* Delete unecessary print
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
* fix create index
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-06 09:41:35 +08:00
99699b2021
Support to show collections which are loaded into memory ( #5567 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-06-03 19:09:33 +08:00
f80dbab6ec
watch proxynode ( #5424 )
...
watch proxy node and,
send InvalidataCollectionMetaCache to each proxynode
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
Resolves : #5423
2021-05-26 12:14:30 +00:00
dab536a6cf
Rename some field names in master for better code readability ( #5389 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-25 06:03:06 +00:00
39f8c9dd56
Save msg start position and end position when AddSegment and AddIndex ( #5358 )
...
Resolves : #5332
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-24 06:19:52 +00:00
c0daf8e41d
set timestamp on dd requests ( #5298 )
...
- set dd request's time tick on master
- send time tick into dd channel
See also: #5291
Signed-off-by: yefu.chen yefu.chen@zilliz.com
2021-05-20 06:14:14 +00:00
d68ee3fbc2
query base on timeline ( #5279 )
...
query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection
Resolves : #5219
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 09:12:17 +00:00
49f6542b1d
query master service's with timestamp ( #5275 )
...
let meta support snapshot
so collection could query meta with timestamp
Resolves : #5219
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-18 06:18:02 +00:00
e8ceb37f81
Add ShardsNum when create collection ( #5245 )
...
Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.
Resolves : #5240
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-17 11:15:01 +00:00
001795ee70
Optimize CreateIndex routine in master service ( #5238 )
...
Remove CreateIndexTask struct and indexTaskQueue
Resolves : #5228
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-15 10:08:08 +00:00
08bb1b2ec3
Resend unsuccessful ddMsg when master start ( #5214 )
...
Resend unsuccessful ddMsg when master start
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-14 21:26:06 +08:00
7b83b11e95
Save ddmsg info and type into ETCD ( #5173 )
...
Save ddmsg info and type into ETCD to support re-send ddmsg
when system restart or recover from failure (#5172 )
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-12 07:33:53 +00:00
262c484cec
the return struct of "DescribeIndex" should contain "filed_name" ( #5055 )
...
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-04-27 02:30:55 +00:00
d7c0a72611
index name is unique in collection scope ( #5046 )
...
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-04-26 15:53:47 +08:00
82ccd4cec0
Rename module ( #4988 )
...
* Rename module
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
83647e8503
Add copyright in master service module
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-04-19 11:12:56 +08:00
8dd3e1e28f
Add unit test for masterservice
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-04-08 17:31:39 +08:00
a250eb370f
Check request type and add unittest
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-04-08 15:26:18 +08:00
7ad9b36207
Update reviewdog/action-hadolint github action version to v1.16.1
...
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-25 14:41:46 +08:00
c2734fa55f
Fix bug and enchance system
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
ba65c38d40
Fix Deadline Exceeded
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-19 17:50:26 +08:00
1d105ae451
Tag test case of test_connect.py::TestConnect::test_close_repeatedly
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-16 11:19:56 +08:00
6eb45c92eb
Fix flush segment
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-13 17:05:36 +08:00
47d9471498
Change gracefultime to 1 second
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-03-13 14:42:53 +08:00
f3649f0419
Refactor interface and proto
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-12 14:22:09 +08:00
7b2d67242d
Fix describe segment if index not exist
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-10 14:45:35 +08:00
014c4fe8ce
Add config of minSegmentSizeToEnableIndex
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-08 15:46:51 +08:00
ddddd65d10
Delete internal/errors package
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-03-05 10:15:27 +08:00
ba21b11e9e
Fix masterservice :drop index
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-04 14:54:16 +08:00
d3db2b0d82
Add zap log in query service
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-27 10:11:52 +08:00
4ea27335fd
Ignore index param from create_collection
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-26 14:40:53 +08:00
d5631b09e8
Describe collection by id
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-25 16:08:56 +08:00
e396c3f4ad
Add zap log
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-24 16:25:40 +08:00
f0c16a228e
Fix show segments
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-22 16:40:26 +08:00
1d3fdebcf4
Add drop index for proxy
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-20 15:38:44 +08:00
85c1404393
Refactor master:index ids
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-11 08:41:59 +08:00
e2d8358cb4
Refactor master:index ids
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-09 13:11:55 +08:00
4035083188
Support Index name
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-08 14:49:12 +08:00
faa23fde86
Fix getindexstate failed to fetch indexstate
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-02-08 14:20:29 +08:00
7342e0758c
Fix index and flush errors, and fix master crash error
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-02-07 17:02:13 +08:00
c231f6da79
Fix collectionName not found error
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-05 16:32:03 +08:00
e6aec3fd08
Hardcode the peer id of proxy node
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-05 14:09:55 +08:00
e384222803
Fix the logic of node id allocator
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-02-05 11:25:52 +08:00
bea1535425
Add call of release collection when drop collection
...
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-02-05 11:05:37 +08:00
6e7e0b748a
Send timetick to insert channel
...
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-02-03 20:04:29 +08:00
zhenshan.cao
Cai Yudong
sunby
dragondriver
neza2017
Xiangyu Wang
quicksilver
bigsheeper
groot
godchen