milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
13,120 Commits
151 Branches
122 Tags
588 MiB
Commit Graph

199 Commits (0272ebf746f4fb17c627c3b92c2e050768f6d3c2)

Author SHA1 Message Date
yukun f51ef71b51
Change default retentionSizeInMB to 8192(8GB) (#11436) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-11 17:18:43 +08:00
Xiangyu Wang 47ff1a45fe
Improve log messages in rocksmq (#11230) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-10 22:10:47 +08:00
yukun 4c6d77201f
[skip ci]Add rmq consumer impl getExistedConsumer comment (#11594) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-10 21:50:45 +08:00
yukun 9bc7393913
[skip ci]Add rmq consumer impl Subscription comment (#11593) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-10 21:48:53 +08:00
yukun 1e86bd3038
[skip ci]Add rmq consumer impl topic comment (#11533) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-10 19:21:44 +08:00
yukun 5ec895fedc
[skip ci]Add rmq consumer impl MsgMutex comment (#11532) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-09 21:13:37 +08:00
yukun 2073ae497b
[skip ci]Add rmq consumer impl chan comment (#11531) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-09 21:11:41 +08:00
yukun 11e6a8ea50
[skip ci]Add rmq_client comment (#11458) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-08 21:09:07 +08:00
congqixia 6d3952c9bf
Fix msgs consumed before seek appear in output after seek (#11419) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-08 20:57:57 +08:00
congqixia d50543a030
Batch consume msg in rockmq internal loop (#11373) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-08 20:56:07 +08:00
yukun dfbb4c33b4
Rename newExpiredCleanUp to expiredCleanUp (#11433) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-08 19:25:00 +08:00
yukun 3a65a4517c
[skip ci]Add retention comment (#11432) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-08 18:45:14 +08:00
yukun 8ec040a35c
Add rocksmqPageSize to config (#11113) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-08 10:17:18 +08:00
yukun 425c7e09d4
[skip ci]Add rmq retention comment (#11344) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-05 21:53:22 +08:00
Xiaofan a4715996ef
Fix Pulsar seek to latest may block forever (#11128) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-05 14:55:44 +08:00
Xiangyu Wang 60848029e1
Improve warning message in rocksmq (#11227) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-05 10:58:55 +08:00
yukun c05ed4c365
[skip ci]Add rmq_imple seek comments (#11273) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-04 20:51:10 +08:00
Xiangyu Wang e8c858b545
Improve error message in util (#11137) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-04 13:36:35 +08:00
Xiangyu Wang 508f560527
Improve error message in util (#11135) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-04 13:34:45 +08:00
Xiangyu Wang cc6a08f097
Improve error message in util (#11136) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-11-04 13:32:46 +08:00
yukun 82e5d21317
Fix MB const size (#11088) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-02 18:06:28 +08:00
yukun ea6cd7d27c
Move topic lock to the front of final delete in retention expired cleanup (#11076) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-02 17:28:31 +08:00
yukun a8722797fe
Add topic lock for rocksmq Seek (#11083) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-02 17:00:31 +08:00
Xiaofan ca56290e85
Fix rocksdb retention ts not set (#11081) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-11-02 16:50:31 +08:00
congqixia 7d671b14ff
Fix SeekToLatest memory leakage and remove redundant logic (#11057) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-02 14:14:40 +08:00
yukun af7a5ca50f
[skip ci]Add rocksmq client comments (#11052) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-01 23:02:33 +08:00
congqixia be0daafdb0
Fix rocksmq SeekToLatest (#11029) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-11-01 19:35:51 +08:00
yukun 9016bd7e33
[skip ci]Add rocksmq comments (#10925) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-29 21:45:01 +08:00
yukun 552efd4b7b
[skip ci]Add comments for rocksmq producer_impl Send (#10696) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-26 21:40:24 +08:00
yukun e8651a1cd9
[skip ci]Add comments for rocksmq producer_impl (#10695) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-26 21:38:29 +08:00
yukun 5f5949cf2f
[skip ci]Remove unused code (#10597) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-25 20:30:48 +08:00
congqixia 76b64a9d01
Remove not needed ctx from rocksmq client (#10416) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-22 17:25:12 +08:00
yukun 319b3282d9
[skip ci]Add comments for rocksmq consumer_impl (#10389) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-21 22:45:22 +08:00
yukun 4a8619619b
[skip ci]Add comments for rocksmq (#10388) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-21 22:43:21 +08:00
edward.zeng 39b204374b
[skip ci] Fix rocksmq retention typo (#10376) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-10-21 19:56:10 +08:00
congqixia c2c2036154
Stop retention gracefully (#10367) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-10-21 19:50:26 +08:00
yukun f15dc150f1
[skip ci]Add comments for retention (#10316) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-21 10:59:01 +08:00
yukun 81190c573a
[skip ci]Add comments for UpdateAckedInfo (#10213) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-19 20:34:50 +08:00
yukun 7c7a66df6d
[skip ci]Add comments for rocksmq Notify (#10212) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-19 20:32:48 +08:00
Xiangyu Wang 736a2cc358
[skip ci]Improve error message in rocksmq (#10196) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-10-19 17:54:37 +08:00
Xiangyu Wang 88a2b2b07c
[skip ci]Improve error message in rocksmq (#10195) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-10-19 17:52:42 +08:00
Xiangyu Wang afa6eeecd7
[skip ci]Improve error message in rocksmq (#10194) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-10-19 17:50:49 +08:00
yukun 401753dea0
[skip ci]Add comments for rocksmq SeekToLatest (#10120) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-18 19:52:49 +08:00
yukun ec667846f4
[skip ci]Add comments for rocksmq Seek (#10119) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-18 19:50:59 +08:00
yukun 8dccd0ca70
[skip ci]Add comments for rocksmq Consume (#10118) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-18 19:48:40 +08:00
yukun 65a1ae0210
[skip ci]Add comments for rocksmq produce (#10030) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-16 22:05:04 +08:00
yukun e8d5a1ee5f
[skip ci]Add comments for DestroyConsumerGroup (#10029) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-16 22:03:09 +08:00
yukun 241acadce9
Remove error return when seek invalid messages in seekToLatest (#9967) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-16 21:49:40 +08:00
yukun 2cfdf22897
Change retention ticker time to 10 minutes (#9933) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-15 15:44:35 +08:00
yukun 83f1b75293
[skip ci]Add comments for RegisterConsumer (#9912) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-14 20:02:37 +08:00
yukun a0d9991d6f
[skip ci]Add comments for CreateConsumerGroup (#9911) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-14 20:00:35 +08:00
yukun d2e2e0a63c
[skip ci]Add comments for ExistConsumerGroup (#9832) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-13 20:44:33 +08:00
yukun 67d38d6cf1
Fix retention datarace (#9828) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-13 20:38:33 +08:00
zhenshan.cao e97b62cf67
Change RocksDBLRUCache size to 0 (#9820) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-13 20:28:34 +08:00
yukun e3ce1696e7
[skip ci]Add comments for rocksmq DestroyTopic (#9735) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-12 19:37:20 +08:00
yukun 55c615b4e2
[skip ci]Add comments for rocksmq CreateTopic (#9733) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-12 19:33:24 +08:00
yukun cdeb64d90a
Delete messages before delete meta data in rocksmq retention (#9694) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-12 19:09:22 +08:00
yukun b6b4b784cc
Delete end key after rocksdb DeleteRange (#9572) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-10 21:50:33 +08:00
yukun 12b985c32e
Add resource free in err process (#9564) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-09 19:39:16 +08:00
yukun b6ec1783e4
Remove acked infos in memory for retention (#8735) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-09 17:24:55 +08:00
yukun a6e97e0d44
Change LRUCache size of rocksdb to 512MB (#9509) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-09 11:08:56 +08:00
yukun 969c78e79d
[skip ci]Add comments for rocksmq close (#9519) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-10-09 00:11:36 +08:00
godchen f4e0736e01
Handle util rocksmq returned error (#9169) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-08 19:07:21 +08:00
Xiaofan fdbfa62b27
Add log for rocksmq (#8889) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-10-07 22:18:56 +08:00
yukun 9303a6414d
Format rocksmq code (#8955) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-30 21:27:55 +08:00
yukun c486007796
Rocksmq support SubscriptionPositionLatest (#8947) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-30 20:51:40 +08:00
yukun 7094722fb0
Normalize rocksmq comment (#8886) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-29 20:54:00 +08:00
yukun 0edb2db331
Destroy write batch and write options in DeleteMessages (#8884) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-29 20:46:09 +08:00
yukun e85cbea18d
Set CacheIndexAndFilterBlocks to true for rocksdb (#8851) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-29 18:45:59 +08:00
yukun 1eb3168254
Unset max open files in rocksdb (#8799) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-28 21:20:03 +08:00
yukun 174c6275ca
golint rocksmq client (#8604) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-26 19:35:57 +08:00
yukun c4c89a639a
Fix go lint in rocksmq (#8559) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-26 18:29:56 +08:00
congqixia 353eeab8b9
Add MessageID return value in producers (#8586) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-09-26 17:38:07 +08:00
yukun 075d6880a4
Fix rocksmq memleak (#8564) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-26 15:31:56 +08:00
yukun 5e25041254
Add rocksmq comments (#8474) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-24 21:45:56 +08:00
yukun a8a0ae27ae
Add rocksmq client comments (#8475) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-24 16:05:55 +08:00
yukun f34e5205f7
Fix some memleak in rocksmq (#8405) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-23 17:21:54 +08:00
yukun 78e8e4aa22
Add rocksmq comments (#8362) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-23 16:59:54 +08:00
yukun 9e17fdb598
Improve rocksmq codecov (#8328) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-22 20:16:04 +08:00
yukun 06c9f03c59
Refactor rocksmq close logic and improve codecov (#8278) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-22 17:21:00 +08:00
yukun e0761271dd
[skip ci]Add rocksmq_retention comment (#8016) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-15 23:29:48 +08:00
yukun 13f9b32452
Improve global_rmq code coverage (#7928) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-15 17:31:49 +08:00
yukun f1366b9451
Rename rocksmq unittests (#7846) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-14 09:55:00 +08:00
yukun 92295b77a1
Add DestroyTopic and DestroyConsumerGroup for CloseRocksMQ (#7730) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 20:54:01 +08:00
yukun d60ea8bc21
Change prefix length to FixedChannelNameLen + 1 (#7717) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 17:42:10 +08:00
yukun 5a303e7672
Fix rocksmq load with prefix (#7678) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 10:22:01 +08:00
yukun 34228eb74b
Add topic lock for DestroyConsumerGroup (#7636) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-10 10:20:00 +08:00
zhenshan.cao 89358ad91e
Deep copy rocksdb data to byte slice (#7605) 
Add unittest

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-09-09 10:56:00 +08:00
yukun fbc352263c
Improve rocksmq client code coverage (#7540) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-08 11:03:59 +08:00
yukun 7025a6e925
Improve rocksmq code coverage (#7483) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-06 19:36:42 +08:00
yukun d5d73833b2
Add some logs in rocksmq retention (#7403) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-02 20:04:10 +08:00
yukun 71aa300499
Fix rocksmq retention panic when delete by message size (#7393) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-09-01 12:03:58 +08:00
wt 780a4d2c8e
Update script of run_go_codecov (#7248) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-23 22:01:51 +08:00
紫晴 7233b62dad
Enable 70% of delta condition judgment for codecov (#7160) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-19 15:50:12 +08:00
紫晴 09525f5723
Update file of codecov (#7144) 
Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-18 14:48:11 +08:00
紫晴 1b4231b4e8
Add yaml file of codecov (#7124) 
* Add yaml file of codecov

Signed-off-by: wangting0128 <ting.wang@zilliz.com>

* Update code of codecov

Signed-off-by: wangting0128 <ting.wang@zilliz.com>
 2021-08-17 11:56:11 +08:00
yukun 847586eb95
Add rocksmq retention (#6617) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-08-16 18:46:10 +08:00
XuanYang-cn 6c5be83ea1
Refactor NewEtcdKV API (#7075) 
This PR mainly did the following 3 things:

- Refactor NewEtcdKV API
- Add unittests for `kv/etcd/`, raise the coverage
up to 94%
- Correct some APIs from version to revision

The old NewEtcdKV() API took `clientv3.Client` as
one of the input, making everyone who using this API
had to create a new `clientv3.Client`. So there're a lot
of repeated codes everywhere.

Meanwhile, someplace didn't even check whether the
`clientv3.New()` returned error or not, and the old API
also didn't check whether the clientv3.Client was Nil.
This caused issue#6955.

Ectd's version and revision have nothing alike, they
represent different concepts, but our APIs kind of
mixed them all up. So I correct those APIs' names.

Resolves: #6955

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-08-13 11:04:09 +08:00
yukun e94a27856e
Mkdir rocksmq_path if not exist (#6134) 
Signed-off-by: kun yu <kun.yu@zilliz.com>
 2021-06-26 12:28:07 +08:00
yukun eb818d05ab
Remove vector fields from output fields (#6112) 
* Remove vector fields from output fields

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

* Add empty_collection error_code

Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-06-25 16:52:11 +08:00