Commit Graph

55 Commits (f9dc87ba6cdf577539625c7f8b5bf9650572341a)

Author SHA1 Message Date
yukun 4ba974d193
Rocksmq client should be closed when close a msgstream (#13865)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-12-29 10:04:57 +08:00
congqixia 444fae87ab
Use msgstream bufsize for mqclient initialization (#14406)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-12-28 14:23:50 +08:00
jaime 83ef9a9ba1
[skip e2e] Fix golint in mq_msgstream (#14019)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-23 15:41:11 +08:00
jaime 75010777df
[skip e2e] Fix golint in mq_msgstream (#13917)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-21 21:51:10 +08:00
JackLCL 9f66b35d1f
[skip e2e]Fix error log output format in mq_msgstream.go (#13808)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-20 22:55:19 +08:00
JackLCL 46261278f0
[skip e2e]Fix error log output format in mq_msgstream.go (#13807)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-20 22:53:23 +08:00
jaime 256644002f
[skip ci] Fix golint in mq_msgstream (#13540)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: yun.zhang <yun.zhang@zilliz.com>
2021-12-16 22:25:41 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-12-14 15:31:07 +08:00
JackLCL 333f495e72
[skip ci]Fix error log output format in mq_msgstream.go (#13100)
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
2021-12-09 19:41:48 +08:00
zhenshan.cao bdd39c0623
Fix bug: check message payload before unmarshaling (#12315)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-11-29 14:31:18 +08:00
godchen f31ed089b5
Add timeout for reader next (#12308)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-26 22:45:24 +08:00
godchen e3f49858c5
Add subscription prefix (#12251)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-24 17:47:15 +08:00
Xiaofan 0aaa87a79a
Fix MsgStream inconsistent seek (#12042)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-24 10:25:15 +08:00
godchen 51b353b52c
Add msgstream reader (#12130)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-19 15:57:12 +08:00
congqixia bc765ea65b
Fix start position for seek into a mark (#12008)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-17 19:55:16 +08:00
godchen a1d1b3d0b6
Add reader interface (#11973)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-11-17 14:11:11 +08:00
congqixia 304bbd3e71
Add ConsumeAfterSeek interface (#11818)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-15 18:17:10 +08:00
Xiaofan a4715996ef
Fix Pulsar seek to latest may block forever (#11128)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-11-05 14:55:44 +08:00
XuanYang-cn 5b79c5ee92
[skip ci]Update license for ms mq (#11260)
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2021-11-04 19:12:03 +08:00
congqixia 9c53375bba
Fix MqttMsgStream skip current msg logic (#11171)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-11-03 17:11:47 +08:00
zhenshan.cao 6ae5b926c1
[skip ci]Fix golint error in mq_msgstream (#10860)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-10-28 23:44:46 +08:00
Xiaofan d845153de4
Add Buffer for consumer channel (#9578)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-15 20:31:16 +08:00
Xiaofan a0b98dacda
Skip reconsume tt msgstream when datacoord restart (#9684)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-10-15 11:46:33 +08:00
xige-16 e99ecc8cab
Add ProduceMark interface to return messageID (#9556)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-10-11 11:38:57 +08:00
xige-16 593ceb6d1d
[skip ci] Add comment for seek (#8811)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-28 23:17:53 +08:00
xige-16 002e676bd1
[skip ci] Add comment for AsProducer (#8809)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-28 23:16:14 +08:00
xige-16 72e430614c
[skip ci] Add comment for AsConsumer (#8810)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-09-28 23:08:05 +08:00
congqixia e7117f67ee
Add BroadcastMark for Msgstream returning MessageIDs (#8654)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-27 14:10:09 +08:00
Cai Yudong ece0a06798
Update repack delete msg (#8631)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-27 10:35:58 +08:00
congqixia 353eeab8b9
Add MessageID return value in producers (#8586)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-26 17:38:07 +08:00
Xiangyu Wang cbe8c03224
Fix golint warnings in msg.go (#8590)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-26 17:36:07 +08:00
Cai Yudong 4fba738bec
Update internal proto DeleteRequest (#8466)
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-09-24 15:21:55 +08:00
Xiangyu Wang eb413f1396
Add comments for exposed structures and functions (#8435)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-23 21:57:55 +08:00
Xiangyu Wang 57618ac624
Remove unused codes in msgstream (#8368)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-23 16:29:55 +08:00
Xiangyu Wang f203f750aa
Improve import statements in msgstream (#8237)
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-09-18 19:06:09 +08:00
congqixia 76f5264e79
Fix MqMsgStream Seek logic (#7536)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-07 16:35:59 +08:00
congqixia d3027c0d28
Fix possible deadlock (#7428)
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2021-09-02 14:56:10 +08:00
Xiaofan aba234fa3f
Fix Msgstream exponential retry (#6807)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2021-08-03 10:39:24 +08:00
neza2017 a518e408b1
add log when seek in msgstream (#6244)
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-07-01 13:46:16 +08:00
yukun 0f409da136
Fix standalone seek hang after restart (#6073)
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
2021-06-24 16:44:07 +08:00
xige-16 c310c70c00
Assign values to msgPack's startPosition in the form of deepCopy (#5900)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-06-21 14:16:03 +08:00
godchen c2ecce61c0
Make pulsar client singleton (#5782)
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-06-15 19:03:56 +08:00
neza2017 351d87055a fix ttmsgstream (#5689)
* fix msgstream

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* fix asconsumer

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-06-15 16:06:13 +08:00
bigsheeper cdbc6d2c94
Refactor query node and query service (#5751)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: yudong.cai <yudong.cai@zilliz.com>
2021-06-15 12:41:40 +08:00
godchen 918458a1be
Add msgstream seek (#5483)
* Add msgstream seek

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* change test name

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* fix error

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* remove file

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

* resolve conversation

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-31 17:46:30 +08:00
Cai Yudong c468481a36
Optimize MqTtMsgStream and unittest (#5498)
* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add getTsMsgFromConsumerMsg

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename some variables for better readability

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add chanWaitGroup

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add chanTtMsgTime

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* rename lastTimeStamp to currTimeStamp

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add comments

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* support unsynced ttMsg

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* code optimize

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* add TestStream_PulsarTtMsgStream_3

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix unittest

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* remove debug log

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* change ttMsg logic back

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix ci block issue

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update testcase

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix codacy

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-31 11:31:30 +08:00
Cai Yudong b414800d49
Update Seek interface (#5492)
* update Seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* update Seek for mqTtMsgStream

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* use Retry in Seek

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* fix static-check

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2021-05-29 23:21:34 +08:00
xige-16 fce792b8bf
Add historical and streaming module in querynode (#5469)
* add historical and streaming

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* fix GetSegmentInfo

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

* pass regression test

Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2021-05-28 10:26:30 +08:00
xige-16 3630eec92c
Fix receving wrong msg after seek (#5441)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-05-27 02:38:37 +00:00
zhenshan.cao 6766169878
Refactor repack logic for insertion (#5399)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-25 11:53:15 +00:00