Commit Graph

49 Commits (18cfd31d40364c5da07376614a0ed6772d4e5ca4)

Author SHA1 Message Date
yukun 18cfd31d40 Fix Load and Remove with prefix
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-01-11 15:17:06 +08:00
zhenshan.cao 02df310fa5 Fix data race
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-09 12:30:22 +08:00
sunby 5dfe9448ae Modify test_index
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-08 16:00:16 +08:00
GuoRentong 82ad488555 Update doc: rocksmq
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 18:37:24 +08:00
XuanYang-cn 65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-08 17:20:59 +08:00
GuoRentong c4599e26de Update doc: node api
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
2021-01-08 16:59:19 +08:00
bigsheeper e13822043d Fix search test when partition not exists, add binary vector search validation
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-08 16:41:27 +08:00
godchen fa73f17f94 Add opentracing doc
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-08 15:32:38 +08:00
XuanYang-cn e42f8bca84 Fix dd flush not completed err
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-07 16:59:05 +08:00
dragondriver ced56c84f8 Add support to build binary vector index
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-01-07 14:56:17 +08:00
godchen 8f1f118b26 Add query opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-07 14:24:35 +08:00
sunby c1a758cb58 Fix bug of segment manager
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-07 14:05:08 +08:00
XuanYang-cn b187bb289d Fix writenode flush sync test
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-07 11:47:39 +08:00
xige-16 8715cd1f0c Fix key error when loading index
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-01-06 18:19:44 +08:00
bigsheeper a1e2ff3c26 Add gcNode to write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 16:44:12 +08:00
neza2017 4015d7245d Merge operation
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-06 14:45:50 +08:00
bigsheeper 810be533ab Add gcNode, and remove msg's downstream index
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-06 14:02:25 +08:00
cai.zhang 1cfc2ff0a5 Add make clean in Makefile
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2021-01-06 13:50:48 +08:00
zhenshan.cao b06e01f523 Initmate querynode collection manager
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-01-06 11:31:36 +08:00
godchen 1c87faf8b3 Add insert request opentracing
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-01-06 11:17:35 +08:00
FluorineDog e16a7478da Add verifier
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-01-05 20:13:54 +08:00
XuanYang-cn 05f1350e19 Add writenode to ci
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-05 15:14:59 +08:00
quicksilver ca89ece576 Fix go format bug
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-01-05 14:20:45 +08:00
neza2017 8b39643e44 Fix write node crashed if collection meta not existed
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-01-05 12:20:46 +08:00
sunby 95b162ccfd Refactor flush scheduler
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-01-05 10:57:59 +08:00
bigsheeper d5d9fa03ea Disable the dropCollection filtering
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-04 20:46:00 +08:00
bigsheeper 33e1e17045 Fix writer client DescribeSegment api
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-04 20:16:15 +08:00
bigsheeper cb3ca1ab7c Refactor kvPath param and unify the segment flush kvPath in write node
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-01-04 14:17:59 +08:00
XuanYang-cn 03b5d32569 Add default fields when create collection
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-01-04 12:03:29 +08:00
XuanYang-cn 3a2e2a04b6 Fix writenode bug
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-29 21:15:00 +08:00
XuanYang-cn 9577da9fce Add writenode main
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-28 18:52:34 +08:00
FluorineDog eb8717188e Enable load index
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2020-12-28 17:55:48 +08:00
bigsheeper 5b9e8c4f60 Add write node client segment flushing
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-28 15:25:16 +08:00
XuanYang-cn aab6dede4a Add Flush for DDBuffer and insertBuffer
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-28 11:29:17 +08:00
neza2017 7e8b203fe2 Add time tick in wite node
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-26 17:05:24 +08:00
XuanYang-cn e316533a3a Add Flush Sync service
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-26 16:28:55 +08:00
XuanYang-cn c0797333ce Add ddl binlog metaTable in writenode
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-24 20:55:40 +08:00
neza2017 4dbf4ade41 Add test for master
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2020-12-24 16:19:42 +08:00
zhenshan.cao a0244b7683 Add segment expire logic for proxy
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-24 16:53:31 +08:00
bigsheeper 4ecdea698f Refactor query node and query serviceMain changes:1. Add ddBuffer and save binLog.
2. Trans to insertData.
3. Change dataFields data to Data, dim to Dim4. Add float vector and binary vector.5. Deserialize data and convert to InsertData.6. Move all data into InsertData.7. Add insert buffer and hash string.
8. Add minIOkV in insertBuffer node.
9. Init write node insertBuffer maxSize from writeNode.yaml.
10. Add ddBuffer.
11. Add ddBuffer binLog and minio.
12. Add ddNode unittest.
13. Remove redundant call.
14. Increase test time.
15. Delete ddl const, use request's timestamp instead.

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-24 15:38:29 +08:00
XuanYang-cn 7ce0f27ebc Add buffer to minIO for binlogs
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-23 18:06:04 +08:00
zhenshan.cao 11316525e5 Fix bug of running unittest in parallel
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-22 22:08:03 +08:00
XuanYang-cn 87521adfbd Add logic of deserilize rawData into InsertData
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-21 16:27:03 +08:00
zhenshan.cao 9694203f8b Add writer client and metatable for writer
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-21 10:06:46 +08:00
zhenshan.cao 3f6e926aef Refactor indexbuilder
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-13 06:48:05 +08:00
bigsheeper 585d3f9831 Refactor param table, and add ddNode
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2020-12-11 17:20:14 +08:00
XuanYang-cn d64aa37db3 Add KFlush msg type
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-11 12:01:20 +08:00
zhenshan.cao ca34353e9b Add writernode client
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2020-12-11 11:38:32 +08:00
XuanYang-cn 881be7f3e3 Impl minimum runable writenode and writenode flowgraph
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2020-12-09 20:07:27 +08:00