milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,924 Commits
131 Branches
115 Tags
553 MiB
Commit Graph

85 Commits (05ad242bc0b4a8e08a323bc4d77515a8ccdbf560)

Author SHA1 Message Date
yukun 9303a6414d
Format rocksmq code (#8955) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-30 21:27:55 +08:00
yukun c486007796
Rocksmq support SubscriptionPositionLatest (#8947) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-30 20:51:40 +08:00
yukun 7094722fb0
Normalize rocksmq comment (#8886) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-29 20:54:00 +08:00
yukun 0edb2db331
Destroy write batch and write options in DeleteMessages (#8884) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-29 20:46:09 +08:00
yukun e85cbea18d
Set CacheIndexAndFilterBlocks to true for rocksdb (#8851) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-29 18:45:59 +08:00
yukun 1eb3168254
Unset max open files in rocksdb (#8799) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-28 21:20:03 +08:00
yukun 174c6275ca
golint rocksmq client (#8604) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-26 19:35:57 +08:00
yukun c4c89a639a
Fix go lint in rocksmq (#8559) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-26 18:29:56 +08:00
congqixia 353eeab8b9
Add MessageID return value in producers (#8586) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-26 17:38:07 +08:00
yukun 075d6880a4
Fix rocksmq memleak (#8564) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-26 15:31:56 +08:00
yukun 5e25041254
Add rocksmq comments (#8474) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-24 21:45:56 +08:00
yukun a8a0ae27ae
Add rocksmq client comments (#8475) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-24 16:05:55 +08:00
yukun f34e5205f7
Fix some memleak in rocksmq (#8405) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-23 17:21:54 +08:00
yukun 78e8e4aa22
Add rocksmq comments (#8362) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-23 16:59:54 +08:00
yukun 9e17fdb598
Improve rocksmq codecov (#8328) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-22 20:16:04 +08:00
yukun 06c9f03c59
Refactor rocksmq close logic and improve codecov (#8278) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-22 17:21:00 +08:00
yukun e0761271dd
[skip ci]Add rocksmq_retention comment (#8016) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-15 23:29:48 +08:00
yukun 13f9b32452
Improve global_rmq code coverage (#7928) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-15 17:31:49 +08:00
yukun f1366b9451
Rename rocksmq unittests (#7846) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-14 09:55:00 +08:00
yukun 92295b77a1
Add DestroyTopic and DestroyConsumerGroup for CloseRocksMQ (#7730) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 20:54:01 +08:00
yukun d60ea8bc21
Change prefix length to FixedChannelNameLen + 1 (#7717) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 17:42:10 +08:00
yukun 5a303e7672
Fix rocksmq load with prefix (#7678) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 10:22:01 +08:00
yukun 34228eb74b
Add topic lock for DestroyConsumerGroup (#7636) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 10:20:00 +08:00
zhenshan.cao 89358ad91e
Deep copy rocksdb data to byte slice (#7605) 
Add unittest

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-09 10:56:00 +08:00
yukun fbc352263c
Improve rocksmq client code coverage (#7540) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-08 11:03:59 +08:00
yukun 7025a6e925
Improve rocksmq code coverage (#7483) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-06 19:36:42 +08:00
yukun d5d73833b2
Add some logs in rocksmq retention (#7403) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-02 20:04:10 +08:00
yukun 71aa300499
Fix rocksmq retention panic when delete by message size (#7393) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-01 12:03:58 +08:00
wt 780a4d2c8e
Update script of run_go_codecov (#7248) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-23 22:01:51 +08:00
紫晴 7233b62dad
Enable 70% of delta condition judgment for codecov (#7160) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-19 15:50:12 +08:00
紫晴 09525f5723
Update file of codecov (#7144) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-18 14:48:11 +08:00
紫晴 1b4231b4e8
Add yaml file of codecov (#7124) 
* Add yaml file of codecov

Signed-off-by: wangting0128 <ting.wang@zilliz.com>

* Update code of codecov

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-17 11:56:11 +08:00
yukun 847586eb95
Add rocksmq retention (#6617) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-08-16 18:46:10 +08:00
XuanYang-cn 6c5be83ea1
Refactor NewEtcdKV API (#7075) 
This PR mainly did the following 3 things:

- Refactor NewEtcdKV API
- Add unittests for `kv/etcd/`, raise the coverage
up to 94%
- Correct some APIs from version to revision

The old NewEtcdKV() API took `clientv3.Client` as
one of the input, making everyone who using this API
had to create a new `clientv3.Client`. So there're a lot
of repeated codes everywhere.

Meanwhile, someplace didn't even check whether the
`clientv3.New()` returned error or not, and the old API
also didn't check whether the clientv3.Client was Nil.
This caused issue#6955.

Ectd's version and revision have nothing alike, they
represent different concepts, but our APIs kind of
mixed them all up. So I correct those APIs' names.

Resolves: #6955

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-08-13 11:04:09 +08:00
yukun e94a27856e
Mkdir rocksmq_path if not exist (#6134) 
Signed-off-by: kun yu <kun.yu@zilliz.com>
 2021-06-26 12:28:07 +08:00
yukun eb818d05ab
Remove vector fields from output fields (#6112) 
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 16:52:11 +08:00
yukun 282d2624e0
Not remove rocksdb folder when close and init (#5974) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-22 16:20:10 +08:00
neza2017 71ed21f9bf
Pass smoke test (#5797) 
* pass smoke test

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
 2021-06-16 16:59:50 +08:00
Cai Yudong 0f4bd50ce3 Support ETCD endpoints array (#5755) 2021-06-15 16:36:36 +08:00
sunby 189ac881f3 Fix bugs (#5676) 
* Remove redundant session startup

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Register datanode after start success

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix meta snap shot

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix datanode message stream channel

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix bugs when drop empty collection

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix bug of getting pchan statistics from task scheduler

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix i/dist/dataservice test code

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix epoch lifetime not applied

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* fix datanode flowgraph dd node

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix handle datanode timetick bug

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix proxynode

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Apply extended seal policy

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add check for time tick

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix check

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix the repack function of dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the bug when send statistics of pchan

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Fix the repack function when craete dml stream

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix bugs

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix describe collection

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix bug when send timestamp statistics

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* fix data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add length check before flush request

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for data node

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix SaveBinlog bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Add more log in datanode

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Put SegmentState.Flushing as the last one in enum to fit the client

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Fix params in GetInsertBinlogPaths

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Rename policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Remove unused ddl functions and fields

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Remove pchan when drop collection

Signed-off-by: dragondriver <jiquan.long@zilliz.com>

* Add balanced assignment policy

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* fix master ut

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add lock in session manager

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* add log for debug

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Fix some logic bug and typo

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

* Fix recover bugs

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Get collection scheme of a specific timestamp

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Change CheckPoint to SegmentInfo in VchannelInfo

Signed-off-by: sunby <bingyi.sun@zilliz.com>

* Recover Unflushed segment numOfRows

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* Fix dataservice unit tests

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: yefu.chen <yefu.chen@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-06-15 16:06:11 +08:00
godchen 91ef35bad4 Change receive msg logic (#5605) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-06-15 16:04:48 +08:00
yukun a3fb1356de
Fix rocksmq seek error (#5432) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>

In milvus-standalone, when seek is called, rocksmq won't put a
mutex in MsgMutex channel, which cause that the consume
process hang. So the seek function in rocksmq needs to 
notify the consumer to consume.
 2021-05-26 09:31:09 +00:00
Xiangyu Wang 82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
Xiangyu Wang c731e2df78 Add license header in util 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-19 13:42:47 +08:00
godchen deba964590 Change name of kvBase 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-12 18:09:28 +08:00
yukun 964e7e0a28 Add unittest for mqclient 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-04-08 20:05:33 +08:00
yukun 18e0581d5c Destroy readOptions after use 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-31 14:10:07 +08:00
yukun 604f96ed2b Replace consumers map by sync.map 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-31 10:05:24 +08:00
sunby f8e879b423 Fix high cpu usage in singlenode 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-29 17:48:15 +08:00
yukun c4f9a6f22b Fix get position wrong 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-27 17:39:12 +08:00