Commit Graph

78 Commits (556c1cfc8a4d41d7b3914179d49903283a0fd199)

Author SHA1 Message Date
Bingyi Sun 24b3ad22d7
Fix msgstream close block. (#17880)
issue: #17513
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-06-29 16:24:20 +08:00
jaime 762e3b78a6
support configurable sasl mechanisms for kafka (#17542)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-06-29 10:32:17 +08:00
Xiaofan e36fce1fd4
Add log for mqtt msgstream seek (#17640)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-20 15:18:13 +08:00
Xiaofan 1f6fbf91b2
Fix pulsar unsubsribe issue (#17562)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-06-16 17:28:11 +08:00
jaime dbef4c6ec2
Align kafka consume from latest with puslar and rocksmq (#17433)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-06-08 19:00:07 +08:00
xige-16 8c69790383
Fix lost delete msg caused by loadSegment after watchDeltaChannel (#17308)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-06-02 16:56:04 +08:00
jaime 5698bf4236
Add sasl configurations for kafka (#17323)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-06-02 12:12:03 +08:00
bigsheeper e0cbacba59
Check ignorable error to prevent unnecessary panic (#17317)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-06-02 10:12:03 +08:00
zhenshan.cao 08a4dff41d
Add logic for search merging and a simple task scheduler for read tasks (#17022)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>

Co-authored-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
2022-05-23 16:41:58 +08:00
jaime a20e0dfc74
Refactor kafka msgstream implementation (#17069)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-05-23 12:39:58 +08:00
jaime 328ab7ecf6
Fix kafka client hang after killing broker pod of parition leader (#16983)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-05-14 23:05:54 +08:00
Xiaofan 000c5ff3de
Fix msgstream unsubscription (#16883)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-10 19:43:52 +08:00
Xiaofan 62658dcda6
Fix pulsar unscubsribe fail because of consumer not found (#16839)
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-05-09 12:07:52 +08:00
Eng Zer Jun 5375161777
test: use `T.TempDir` to create temporary test directory (#16758)
This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Fixes: #16759
Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-04 11:35:50 +08:00
jaime bb9ccbb7e2
Use a singleton kafka producer (#16739)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-29 17:29:47 +08:00
yah01 2022a361c0
Enable ZSTD compression for pulsar (#16014)
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-04-24 10:25:43 +08:00
jaime c7e98b922a
Fix mq factory ut (#16483)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-14 10:45:34 +08:00
jaime ed3c29eae7
Support Kafka (#16149)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-04-12 19:47:33 +08:00
godchen bb7a0766fe
Add dependency factory (#16204)
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-04-07 22:05:32 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-04-02 17:43:29 +08:00
xige-16 205c92e54b
Support insert string data (#15993)
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-25 14:27:25 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131)
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
Bingyi Sun 256ccb8f65
Fix rootcoord goroutine leak (#16071)
issue: #15658
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-16 16:47:21 +08:00
Bingyi Sun d3d5a41b76
Fix memory leak casued by unclosed channel in msgstream (#16051)
issue: #16045
Signed-off-by: sunby <bingyi.sun@zilliz.com>

Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-15 15:55:21 +08:00
jaime 97b1ed7bca
Remove reader API from Mqstream and replace it with consumer API (#15971)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-15 14:45:22 +08:00
jaime 29975a7a26
Remove Consume() API from mqstream interface (#15886)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
Jiquan Long f71651e294
Support column-based insert data in message stream (#15802)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-04 15:09:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784)
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00