milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,924 Commits
130 Branches
115 Tags
562 MiB
Commit Graph

457 Commits (05ad242bc0b4a8e08a323bc4d77515a8ccdbf560)

Author SHA1 Message Date
godchen e224d1e725
Add SessionManager (#5288) 
Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-19 10:36:05 +00:00
godchen b74afd7a0e
Add service registration (#5189) 
Add service registration.
Part of Issue #5174.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-05-14 02:05:18 +00:00
dragondriver b7977698a8
Change the codec style of expr plan from text to binary (#5129) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-07 11:27:17 +00:00
dragondriver e5d4963ba0
Distinguish the dsl and expr in proxy and query node (#5118) 
Pass a parameter `IsExpr` from proxy to query node, so query 
node can create search plan according to the variable `IsExpr`.
It makes `segcore` unnecessary to distinguish `dsl` or `expr`
using dynamic exception. When `IsExpr` is set to true, query
node will call `CreatePlanByExpr` according to the passed
information about expression. Otherwise query node will keep
still to use `CreatePlan` according to the `dsl` information. At
the same time, this pr adds some unittests to `CreatePlanByExpr`,
these unittests translate already exist case with `dsl` to case with
`expr`.

Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-05-07 07:20:47 +00:00
FluorineDog 88f5642603
Add plan proto and support basic boolean expr parser (#5088) 
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
 2021-04-29 08:48:06 +00:00
bigsheeper 88e1d651cb
Remove partition and segment ids properly in query node (#5090) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

**What type of PR is this?**

- [ ] API-change
- [x] BUG
- [ ] Improvement
- [ ] Documentation
- [ ] Feature
- [ ] Test and CI
- [ ] Code Refactoring

**Which issue(s) this PR fixes:**

Fixes #

**What this PR does / why we need it:**

Not Available
 2021-04-29 07:59:08 +00:00
bigsheeper 4197b094bf
Use internal dsl and placeholder group (#5033) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-25 06:08:28 +00:00
bigsheeper c4b379be8b
Set search graceful time from config (#5014) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-24 07:53:37 +00:00
bigsheeper 2b5bedf736
Use multiple search channels (#5018) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-24 02:44:28 +00:00
bigsheeper 00bb1014cd
Add search channels to query node (#5012) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-23 10:11:26 +00:00
bigsheeper dadb02db75
Remove unused params in query node (#4987) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-23 10:07:45 +08:00
Xiangyu Wang 82ccd4cec0
Rename module (#4988) 
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-22 14:45:57 +08:00
sunby fad94fb576
Remove unsed configs (#4990) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-22 03:04:59 +00:00
xige-16 1165db75f6 Optimize search performance 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-19 19:30:36 +08:00
bigsheeper 71b8f309b2 Add licence to query node and query service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-19 13:47:10 +08:00
bigsheeper 08c5134286 Fix collection cannot be found when searching 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-16 17:37:50 +08:00
bigsheeper 9e7559b865 Make query node asynchronously load and release collection 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-16 14:40:33 +08:00
FluorineDog ce7a5ea699 Support segcoreinit 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-04-16 14:02:49 +08:00
XuanYang-cn e281f6a4a1 Remove useless configs in config files 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-04-13 10:04:39 +08:00
godchen deba964590 Change name of kvBase 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-04-12 18:09:28 +08:00
bigsheeper 39458697c7 Use asynchronous functions of load and release 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-12 09:18:43 +08:00
xige-16 22eda19bc4 Fix mem leak when load index 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-10 10:10:53 +08:00
sunby 751c9d4e81 Refactor meta.go 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-04-09 09:55:04 +08:00
xige-16 332b14c915 Delete querynode’s redundant load IO 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-04-07 18:29:19 +08:00
bigsheeper fe3fa8dd1d Fix search service unittest 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-04-01 15:39:23 -05:00
Xiangyu Wang 87a1e0b662 Reorganize msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-04-02 13:48:25 +08:00
xige-16 c10b36f188 fix getting wrong offset when segment preinsert 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-31 16:16:58 +08:00
xige-16 a6f1de036b Optimize search performance in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-30 22:16:58 +08:00
zhenshan.cao efbf4452c5 Add log for port changed or proto unmarshal error 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-30 20:19:30 +08:00
godchen 8f58d5641b Perfect trace and add trace init 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-30 16:36:34 +08:00
Xiangyu Wang 8d66388b8c Add logs for querynode 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-30 15:00:14 +08:00
yukun 432b53149b Refactor msgstream 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-03-26 20:10:11 +08:00
bigsheeper 8a4c6a039f Add search log 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-25 18:09:47 -05:00
xige-16 7e60460aee close timeTickStream in flow graph when the collection is released 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-26 18:40:04 +08:00
bigsheeper dfb68a7e46 Remove unused segment 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-25 16:08:08 -05:00
FluorineDog f39dcdb8f3 Support error code in segcore 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-26 16:18:30 +08:00
neza2017 eca1e25470 Add msg id at master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-26 15:13:33 +08:00
bigsheeper 1e4afda824 Add mutex for query node map, search buffer and dataSyncService 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-24 15:40:59 -05:00
quicksilver 7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-25 14:41:46 +08:00
bigsheeper 350e0df5f7 Refactor search service 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-22 16:59:46 -05:00
bigsheeper 1634d75980 Release flowgraph properly 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-22 12:49:50 -05:00
xige-16 1a1e864709 Fix query node crash when empty search 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-22 19:20:56 +08:00
zhenshan.cao c2734fa55f Fix bug and enchance system 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-22 16:36:10 +08:00
groot 2280791128 Add memory message stream for search 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-03-19 20:16:04 +08:00
zhenshan.cao 636755827f Fix unittest bug: query node num 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-03-17 14:19:14 +08:00
FluorineDog 2cec04ed90 Fix empty schema proto hack 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-03-17 11:35:28 +08:00
xige-16 b92ad3edd6 Fix error when search on collection without partitions 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-15 19:58:52 +08:00
groot 47d9471498 Change gracefultime to 1 second 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-03-13 14:42:53 +08:00
xige-16 8a5c039137 Fix search error when running single node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-12 19:23:06 +08:00
godchen f3649f0419 Refactor interface and proto 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-03-12 14:22:09 +08:00
ThreadDao ca916a5c47 Add ci nightly 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-03-11 14:14:29 +08:00
xige-16 d35d316056 Close seek function to pass regression test 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-11 10:21:24 +08:00
xige-16 09ae985daa Fix wrong error code in master_service_test.go 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-10 22:06:22 +08:00
del-zhenwu b1763c3690 Disable xdist in test cases 
Signed-off-by: del-zhenwu <zhenxiang.li@zilliz.com>
 2021-03-10 15:27:26 +08:00
XuanYang-cn 8567679888 Check enableIndex before load index 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-03-10 14:51:00 +08:00
neza2017 7b2d67242d Fix describe segment if index not exist 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-03-10 14:45:35 +08:00
quicksilver 455cc59dbb Update publish-test-images.yaml 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-09 13:55:35 +08:00
sunby 76480fcefe Change default log output to stdout 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 16:24:55 +08:00
sunby ab21115d54 Init rand.Seed 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-08 15:25:55 +08:00
ThreadDao 7a3223dc81 tags collection stats 
Signed-off-by: ThreadDao <yufen.zong@zilliz.com>
 2021-03-08 10:09:48 +08:00
Xiangyu Wang e5d595564a Add log for singlenode 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-03-05 18:16:50 +08:00
quicksilver 74154a11a4 Fix deploy error during regression stage 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-03-05 16:52:45 +08:00
sunby ddddd65d10 Delete internal/errors package 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-03-05 10:15:27 +08:00
bigsheeper 447a15207e Add zap log to query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-03-05 09:21:35 +08:00
xige-16 6cdd4d02fc Refactor seek function 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-03-04 14:49:51 +08:00
ophunter233 4a921ffd7e Stop listen to dd channel in query node 
Signed-off-by: ophunter233 <chengming.li@zilliz.com>
 2021-03-03 16:06:02 +08:00
godchen 7b9fdd7f29 Add opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-26 17:44:24 +08:00
xige-16 50a5f97421 Delete componentStateResponse in queryService.proto 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-26 15:26:21 +08:00
cai.zhang f5977a1302 Change retry times and add retry logic 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-02-26 15:17:47 +08:00
bigsheeper 9edbff0a6b Fix fields check 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-26 11:15:44 +08:00
Cai Yudong 84110d2684 Add tests/benchmark and tests/python_test using new python SDK 
Signed-off-by: Cai Yudong <yudong.cai@zilliz.com>
 2021-02-25 17:35:36 +08:00
bigsheeper 08a020798e Release collection and partitions 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-24 17:24:51 +08:00
XuanYang-cn fd562f9f9c Update doc: add proxy graph 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-24 09:48:17 +08:00
xige-16 d5379a82a8 Fix load binlog error when segment not flushed 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-23 14:13:33 +08:00
zhenshan.cao bbfcbbdd68 Fix ci: paramstable and startup logic refactor 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-23 11:40:30 +08:00
yukun e786ad6aa8 Fix inconsistency of produce msg and consume msg with rmq 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-02-22 16:34:15 +08:00
sunby b07a371f25 Create a goroutine to receive msg for every consumer 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-22 09:58:34 +08:00
xige-16 861576f77a Checkout field ids when load segment in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-22 10:44:38 +08:00
xige-16 f62078c027 Fix crash error when search 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-20 10:14:03 +08:00
xige-16 a6442cb600 Fix querynode memory leak 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-20 09:20:51 +08:00
sunby 5769c000a1 Find failed python tests and add skip mark "r0.3-test" 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-19 15:37:04 +08:00
bigsheeper d8f41db823 Fix search crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-19 13:56:01 +08:00
xige-16 450ea631bc Fix queryService assign queryChannel failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-18 16:26:02 +08:00
xige-16 70fe5233cf Watch queryChannel on query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-09 17:09:26 +08:00
groot 833f1d49ab Let singlenode use rocksmq 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-09 15:57:10 +08:00
zhenshan.cao 7901d98d5e Fix bug: return wrong IndexID 0 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-08 18:42:33 +08:00
groot dc3736281a Integrate message stream 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-08 14:30:54 +08:00
bigsheeper 12b2eaf196 Rename apis 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 21:26:03 +08:00
zhenshan.cao 1ba8e2448f Add tsLoop for indexservice 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-07 18:38:58 +08:00
bigsheeper a64c831ed9 Fix segment replacement bug 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 18:37:18 +08:00
bigsheeper 7342e0758c Fix index and flush errors, and fix master crash error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 17:02:13 +08:00
bigsheeper b07b2484dc Fix error when loading 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-07 10:29:58 +08:00
xige-16 cdcf006690 Fix loadCollection error when query service assign insert channels and query nodes 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-07 09:30:48 +08:00
xige-16 a12b184e02 Add field schema to LoadCollection and LoadPartition photo 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-06 17:37:10 +08:00
bigsheeper e685f1825b Fix unittest caused by param table 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-06 15:16:10 +08:00
groot b034740c6c Make paramstable initialize once by using do_once 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2021-02-06 13:39:15 +08:00
bigsheeper e9ee9a273e Refactor load service and check insertion binLog periodically 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-06 11:35:35 +08:00
sunby 0111cba8ae Fix startup bugs in some modules 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-06 10:28:53 +08:00
dragondriver 7e1887da12 Add start.sh and stop.sh, remove them later 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-05 14:31:40 +08:00
bigsheeper a7eae3a4c1 Remove consumer in query node's initialization 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 14:15:10 +08:00
bigsheeper 1578c13224 Refactor collection replica 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 10:53:11 +08:00
bigsheeper 2ee54541aa Fix query node address 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-05 10:25:47 +08:00
godchen c2ca2c276f Fix response check error 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-04 19:34:35 +08:00
dragondriver e0e8e1605e Implement the release-related task and load-related task in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-04 17:47:19 +08:00
sunby 7270d18c98 Fix timesync startup bugs 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 17:31:02 +08:00
xige-16 aaac839075 Joint debug query service success with other components 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-04 17:09:18 +08:00
zhenshan.cao b89e5a3240 Add GetQuerySegmentInfo 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-04 14:37:12 +08:00
sunby 71b6c88c6e Defer lock's unlock in meta::DropSegment 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-04 11:52:10 +08:00
Xiangyu Wang 8d35e76ad1 Change Msgstream interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-04 13:55:47 +08:00
bigsheeper c6c99ef32b Implement getSegmengInfo function 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-04 11:40:14 +08:00
XuanYang-cn 1aafe86ffe Change MsgPosition Logic 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-02-04 11:19:48 +08:00
dragondriver b92ff69cea Implement GetIndexState in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-04 11:08:36 +08:00
zhenshan.cao 1b616fb874 Add persistent_segmentinfo 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-03 18:55:00 +08:00
bigsheeper f12a0490bb Do not drop collections or partitions in flow graph 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 18:12:48 +08:00
Xiangyu Wang fd9e42d3ed Update msgstream/unmarshal interface 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-02-03 17:30:10 +08:00
bigsheeper abffdbff62 Remove partition tag 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 16:44:38 +08:00
bigsheeper 01e9dc8e3f Remove collection name 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 11:52:19 +08:00
sunby f3aad3a71c Change SegmentInfo 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 18:53:10 +08:00
bigsheeper 5e781b9370 Remove field name in query node and segCore 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-03 10:10:07 +08:00
sunby 45b99c0cf3 Save indexName and indexID in IndexCodec 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-02-02 19:56:11 +08:00
zhenshan.cao bfa14d40a8 Rename indexID to indexBuildID and add IndexID and IndexName 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-02-02 19:56:04 +08:00
godchen 66d5189601 Add get collection statistics implemetation 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-02-02 19:54:31 +08:00
dragondriver e4e3ec88d1 Add usage count of insert channels 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-02-02 15:49:13 +08:00
bigsheeper 09bdbee944 Refactor load service, remove seg manager 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-02-02 15:13:08 +08:00
xige-16 4cd315408b Refactor query service’s meta interface 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-02-02 11:52:41 +08:00
bigsheeper 3573e01093 Check index periodically and load index, add indexing segment type 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-30 16:02:10 +08:00
bigsheeper 73d2b6a101 Get index param from minio and filter by vector fields 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-29 15:22:24 +08:00
neza2017 3a866dab3d Update proto 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-28 17:25:43 +08:00
bigsheeper 68cd15af63 Get index info from master 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-27 14:41:56 +08:00
bigsheeper 4f914a2c30 Add docker file and main function of query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-27 09:50:52 +08:00
dragondriver c6a6b1436c Add soft time tick in proxy service 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-26 13:41:41 +08:00
cai.zhang f940cc455a Add dockerfile for index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-26 09:38:40 +08:00
FluorineDog a8fd6e7686 Enable drop index/field_data and extract info visitor 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-25 18:59:44 +08:00
bigsheeper 7554246ace Add segment seeking and use real client 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-25 18:33:10 +08:00
bigsheeper 445675a588 Refactor loadSegment request and load strategy 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-24 18:02:08 +08:00
zhenshan.cao aef4e41e91 Enable proxynode microbatch pulsar msg and enable parallel test 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-23 20:58:46 +08:00
XuanYang-cn b794921b7e Add enum Role in internal.proto 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-23 18:56:08 +08:00
bigsheeper 067c30c422 Add sealedSegment cgo unittest, fix growingSegment field id check 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-22 15:41:54 +08:00
xige-16 511aa419fa Refactor query service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-22 14:28:06 +08:00
bigsheeper 46a0a6133a Implement distributed api, delete partition states api 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-22 11:17:18 +08:00
dragondriver eba6bb288c Refactor the sructure of proto file 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-22 09:36:18 +08:00
neza2017 9828a52999 Add create index 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-21 10:01:29 +08:00
FluorineDog 7f044fff82 Enable search for sealed 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-21 15:29:52 +08:00
sunby be6775671b Add error returned by Init,Start,Stop functions 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-19 17:38:01 +08:00
cai.zhang 83c60fd84f Update proto about index 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-21 15:20:23 +08:00
Xiangyu Wang 7ee8623e1a Refactor msgstream 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-01-20 10:02:59 +08:00
zhenshan.cao 2ec573d2f9 Recover delete docs 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-20 09:36:50 +08:00
cai.zhang fb493cc235 Update interface name of GetIndexStates 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-19 18:32:57 +08:00
neza2017 918e279869 Set msg stream 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-19 16:14:13 +08:00
neza2017 d5e53aa28a Add Meta table implementation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-19 14:13:49 +08:00
bigsheeper 2064b014a1 Add loadFieldData, use partition id instead of tag 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-19 14:44:03 +08:00
XuanYang-cn 6a13386393 Add DataNode package 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-19 11:37:16 +08:00
neza2017 1ce32d8775 Add master service 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-19 09:45:42 +08:00
bigsheeper aaf8b0ad40 Add sealed segment, segment type and manager, send loadIndexReq to service's buffer 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-19 10:29:16 +08:00
xige-16 d22c2cd203 Add proto for query service loading collection 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-18 19:32:08 +08:00
bigsheeper 48821690b3 Rename Service interface to Component, and update segmentLoad strategy 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-18 10:38:41 +08:00
GuoRentong dd6c4bfb75 Update doc:interface definitions related to LoadCollection 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-18 10:09:17 +08:00
XuanYang-cn 37688559f1 Fix multi rowdata writenode crash bug 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-16 15:06:19 +08:00
bigsheeper 46a14da3eb Implement segment manager and loadSegment 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-16 15:04:32 +08:00
bigsheeper f02bd8c8f5 Rename query node package, implement watchDmChannel 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-16 10:12:14 +08:00
sunby 67e6ea8051 Add data service 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-15 17:09:41 +08:00
bigsheeper b98b226d8c Refactor query node and implement channelRemove, channelAdd 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-15 15:28:54 +08:00
godchen 0971cf65b2 Remove opentracing stdlog 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-15 15:28:27 +08:00
quicksilver 7fe061b846 Raneme init_devcontainer.sh to devcontainer.sh 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-01-15 14:38:36 +08:00
neza2017 56ab2bda2d Update proto:add partition id 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-15 11:53:22 +08:00
zhenshan.cao 63b21321d6 Refactor indexservice and update doc 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-15 13:50:27 +08:00
quicksilver 8e9d8e36e1 Update run_go_unittest.sh 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-01-13 10:40:46 +08:00
FluorineDog 4cd42c553f Rename field_name, make field_id strongly typed, skip multithread test 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-12 18:31:52 +08:00
bigsheeper 2d2af1505a Improve mutex usage in query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-12 18:03:24 +08:00
GuoRentong f3d2afa4f4 Update doc:detailed interface definition 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-11 18:35:54 +08:00
quicksilver a6690dbc99 Fix syntax error: Bad fd number 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2021-01-11 15:25:19 +08:00
yukun 18cfd31d40 Fix Load and Remove with prefix 
Signed-off-by: yukun <kun.yu@zilliz.com>
 2021-01-11 15:17:06 +08:00
bigsheeper 4fad3b189c Fix search without insertion, improve nil hits behavior (#622) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-09 17:58:18 +08:00
xige-16 004d0027b3 Fix error that different loadIndex messages have same indexParam 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-09 15:57:55 +08:00
zhenshan.cao 02df310fa5 Fix data race 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-09 12:30:22 +08:00
xige-16 f4566731fc Fix search error in regression test 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-09 09:47:22 +08:00
sunby 5dfe9448ae Modify test_index 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
 2021-01-08 16:00:16 +08:00
GuoRentong 82ad488555 Update doc: rocksmq 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-08 18:37:24 +08:00
XuanYang-cn 65089ea362 Fix flush sync test bug and remove unecessary lock in meta_table 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2021-01-08 17:20:59 +08:00
GuoRentong c4599e26de Update doc: node api 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2021-01-08 16:59:19 +08:00
bigsheeper e13822043d Fix search test when partition not exists, add binary vector search validation 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-08 16:41:27 +08:00
godchen fa73f17f94 Add opentracing doc 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-08 15:32:38 +08:00
zhenshan.cao 6a3ec57fef Fix docker lib dependency 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-07 21:08:23 +08:00
dragondriver a4860beae2 Parse type params and index params when buildindex is invoked 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-07 17:38:46 +08:00
xige-16 d9d2f33a23 Add binary test for loadIndex and fix loadIndexService can't close correctly 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-07 17:22:10 +08:00
dragondriver ced56c84f8 Add support to build binary vector index 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-01-07 14:56:17 +08:00
godchen 8f1f118b26 Add query opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-07 14:24:35 +08:00
xige-16 8715cd1f0c Fix key error when loading index 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-01-06 18:19:44 +08:00
bigsheeper a1e2ff3c26 Add gcNode to write node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-06 16:44:12 +08:00
neza2017 4015d7245d Merge operation 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2021-01-06 14:45:50 +08:00
bigsheeper 810be533ab Add gcNode, and remove msg's downstream index 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-06 14:02:25 +08:00
cai.zhang 1cfc2ff0a5 Add make clean in Makefile 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-01-06 13:50:48 +08:00
zhenshan.cao b06e01f523 Initmate querynode collection manager 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-01-06 11:31:36 +08:00
godchen 1c87faf8b3 Add insert request opentracing 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-01-06 11:17:35 +08:00
FluorineDog e16a7478da Add verifier 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2021-01-05 20:13:54 +08:00
bigsheeper e9dc96f931 Fix bucket name in index builder and query node 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-05 15:58:13 +08:00
bigsheeper d5d9fa03ea Disable the dropCollection filtering 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-04 20:46:00 +08:00
bigsheeper 1ba47ac433 Check system field in segCore 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-01-04 10:13:01 +08:00
quicksilver 1927eaf88b Add revive linter 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-12-31 16:38:37 +08:00
dragondriver 6cd8c2528d Add parameter check for almost all index type 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-31 15:16:03 +08:00
xige-16 29b21b4bf2 Refactor reduce strategy in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-31 14:49:27 +08:00
xige-16 be32a33eff Refactor loadIndexTest to test diffrent index params 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-29 14:43:40 +08:00
GuoRentong e8ec0424d8 Update doc: service api 
Signed-off-by: GuoRentong <rentong.guo@zilliz.com>
 2020-12-27 09:05:24 +08:00
XuanYang-cn e316533a3a Add Flush Sync service 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-26 16:28:55 +08:00
bigsheeper 5acc9681d6 Fix search error about metric type 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-26 15:29:48 +08:00
xige-16 d599407e2b Add unittest for loadIndex service 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-26 14:16:51 +08:00
XuanYang-cn c0797333ce Add ddl binlog metaTable in writenode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-24 20:55:40 +08:00
neza2017 4dbf4ade41 Add test for master 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-24 16:19:42 +08:00
xige-16 f5198bcaee Add LoadIndexService in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-24 14:51:55 +08:00
zhenshan.cao 11316525e5 Fix bug of running unittest in parallel 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-22 22:08:03 +08:00
bigsheeper 585d3f9831 Refactor param table, and add ddNode 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-11 17:20:14 +08:00
neza2017 8df05714a8 Add binlog unittest 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-11 12:01:38 +08:00
FluorineDog e54d5ccf19 Fix push failed because git hook detects "wrong error" 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-11 14:59:13 +08:00
XuanYang-cn d64aa37db3 Add KFlush msg type 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-11 12:01:20 +08:00
zhenshan.cao ca34353e9b Add writernode client 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-11 11:38:32 +08:00
xige-16 0067e2d989 Add load index client in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-11 11:29:54 +08:00
bigsheeper 762b1e1f1f Add ddService, fieldID, requestID and refactor params table, fix nil hits search result bug 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-10 16:31:09 +08:00
XuanYang-cn 32977e270c Add impl cgo of parquet 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-12-08 14:41:04 +08:00
zhenshan.cao 7bbbc14637 Fix bug: address already used 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-07 15:22:20 +08:00
xige-16 37d7526d31 Fix search test failure 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-07 14:37:42 +08:00
neza2017 aeca8f85a5 Run cwrapper unittest in ci 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 17:39:58 +08:00
cai.zhang 806a97505c Add query node for CI progress 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-05 16:12:45 +08:00
neza2017 70710dee47 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-05 16:11:03 +08:00
FluorineDog 6412ebc0d4 Add support of metric type in schema, enable binary vector, fix segfault 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-05 06:46:01 +08:00
xige-16 acdf8cc9e1 Fix memory leak in query node 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-04 19:32:30 +08:00
zhenshan.cao 0b6fe0a8cf Refactor paramtable 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-04 14:18:17 +08:00
godchen 7ab5b5d80d Add MinIO kv implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-12-03 19:00:11 +08:00
neza2017 cec903da19 Add parquet payload 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-02 18:31:56 +08:00
xige-16 36cf8a8ea7 Support return primary key when doing search 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2020-12-03 17:18:06 +08:00
bigsheeper 707a6a366e Fix issue about collection cannot be found 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-12-02 17:13:16 +08:00
zhenshan.cao 23aebe7e2a Fix proxy unittest 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-12-01 19:53:53 +08:00
neza2017 17cc7bc337 Use MetaRootPath instead 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-12-01 17:03:39 +08:00
dragondriver 84b15c1f5c Fix the bug of meta cache in Proxy 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-12-01 16:41:25 +08:00
cai.zhang c1fbc4f8fa Open the tests for insert 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-12-01 15:18:57 +08:00
FluorineDog 6b74d82259 Support some validation for dsl parser 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-12-01 02:16:53 +08:00
zhenshan.cao fda8d62b38 Fix make error 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2020-11-30 22:14:19 +08:00
neza2017 d3858500c6 Add static check 
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
 2020-11-30 19:38:23 +08:00
bigsheeper 0cd3e8d86c Use exception instead of runtime error 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-30 17:58:23 +08:00
bigsheeper d88a654db8 Fix search result 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-30 11:22:59 +08:00
quicksilver 86573d0053 Create Jenkins pipeline 
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
 2020-11-30 10:07:04 +08:00
FluorineDog 77fa75b1ec Add binary insert and warper of binary search, rename vector 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-30 05:18:44 +08:00
cai.zhang 85a544b79b Add dockerfile for each component 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 19:06:48 +08:00
cai.zhang 115d0732af Update docker-compose file for unittest 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2020-11-28 16:43:39 +08:00
dragondriver eb6c0e0a28 Fix the bug when reduce the multiple search result from query nodes 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-28 15:57:49 +08:00
godchen 92bf09c7e6 Add ddstream and scheduler implements 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2020-11-28 15:26:39 +08:00
dragondriver 568cef0730 Fix InsertTask and SearchTask 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2020-11-28 10:48:29 +08:00
FluorineDog c9fb34142c Enable primary_key switch 
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
 2020-11-28 09:16:00 +08:00
XuanYang-cn 3e605ae25f Rename reader to querynode 
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
 2020-11-27 10:56:53 +08:00
bigsheeper 1decc1a4ca Add metaRootPath and kvRootPath 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-27 09:59:40 +08:00
bigsheeper f21c0ef2e9 Reduce cpu usage 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2020-11-26 17:58:08 +08:00