Commit Graph

263 Commits (1c49ddc86aa67611a367c83154176d54fa38e88d)

Author SHA1 Message Date
neza2017 c28c34e852
let master use session.ServerID as nodeID (#5385)
let master use session.ServerID as nodeID

also see #5386 

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-25 07:06:05 +00:00
zhenshan.cao 3e73775cc1
Add ComputeProduceChannelIndexes for msgstream (#5364)
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-05-24 08:30:09 +00:00
neza2017 e326eaad1e
master client,reconnect if grpc failed (#5360)
master client,reconnect if grpc failed

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>
2021-05-24 01:44:49 +00:00
godchen 0eee7e5252
Change session (#5329)
Change session.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-21 11:28:52 +00:00
sunby c1efa53779
Remove redundant fields in SegmentInfo and related structure (#5280)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-05-21 06:51:25 +00:00
Xiangyu Wang 23c4de0eb8
Flush statistics for all int64 fields (#5318)
Resolves: #5262

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-05-20 10:38:45 +00:00
godchen e224d1e725
Add SessionManager (#5288)
Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-19 10:36:05 +00:00
godchen b74afd7a0e
Add service registration (#5189)
Add service registration.
Part of Issue #5174.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-05-14 02:05:18 +00:00
dragondriver b7977698a8
Change the codec style of expr plan from text to binary (#5129)
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-07 11:27:17 +00:00
dragondriver e5d4963ba0
Distinguish the dsl and expr in proxy and query node (#5118)
Pass a parameter `IsExpr` from proxy to query node, so query 
node can create search plan according to the variable `IsExpr`.
It makes `segcore` unnecessary to distinguish `dsl` or `expr`
using dynamic exception. When `IsExpr` is set to true, query
node will call `CreatePlanByExpr` according to the passed
information about expression. Otherwise query node will keep
still to use `CreatePlan` according to the `dsl` information. At
the same time, this pr adds some unittests to `CreatePlanByExpr`,
these unittests translate already exist case with `dsl` to case with
`expr`.

Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2021-05-07 07:20:47 +00:00
FluorineDog 88f5642603
Add plan proto and support basic boolean expr parser (#5088)
**What type of PR is this?**
- [x] Feature

**What this PR does / why we need it:**
This PR supports boolean expression as DSL.
1. The goal of this PR is to support predicates
    like `A > 3 && not B < 5 or C in [1, 2, 3]`. 
2. Defines `plan.proto`, as Intermediate Representation (IR) 
    used between go and cpp. 
3. Support expr parser, convert predicate expr to IR
    in proxynode, while doing static check there
4. Support IR to AST in cpp, enable the execution
2021-04-29 08:48:06 +00:00
bigsheeper 88e1d651cb
Remove partition and segment ids properly in query node (#5090)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

**What type of PR is this?**

- [ ] API-change
- [x] BUG
- [ ] Improvement
- [ ] Documentation
- [ ] Feature
- [ ] Test and CI
- [ ] Code Refactoring

**Which issue(s) this PR fixes:**

Fixes #

**What this PR does / why we need it:**

Not Available
2021-04-29 07:59:08 +00:00
bigsheeper 4197b094bf
Use internal dsl and placeholder group (#5033)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-25 06:08:28 +00:00
bigsheeper c4b379be8b
Set search graceful time from config (#5014)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-24 07:53:37 +00:00
bigsheeper 2b5bedf736
Use multiple search channels (#5018)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-24 02:44:28 +00:00
bigsheeper 00bb1014cd
Add search channels to query node (#5012)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-23 10:11:26 +00:00
bigsheeper dadb02db75
Remove unused params in query node (#4987)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-23 10:07:45 +08:00
Xiangyu Wang 82ccd4cec0
Rename module (#4988)
* Rename module

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-22 14:45:57 +08:00
sunby fad94fb576
Remove unsed configs (#4990)
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-22 03:04:59 +00:00
xige-16 1165db75f6 Optimize search performance
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-04-19 19:30:36 +08:00
bigsheeper 71b8f309b2 Add licence to query node and query service
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-19 13:47:10 +08:00
bigsheeper 08c5134286 Fix collection cannot be found when searching
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-16 17:37:50 +08:00
bigsheeper 9e7559b865 Make query node asynchronously load and release collection
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-16 14:40:33 +08:00
FluorineDog ce7a5ea699 Support segcoreinit
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-04-16 14:02:49 +08:00
XuanYang-cn e281f6a4a1 Remove useless configs in config files
Signed-off-by: XuanYang-cn <xuan.yang@zilliz.com>
2021-04-13 10:04:39 +08:00
godchen deba964590 Change name of kvBase
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-04-12 18:09:28 +08:00
bigsheeper 39458697c7 Use asynchronous functions of load and release
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-12 09:18:43 +08:00
xige-16 22eda19bc4 Fix mem leak when load index
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-04-10 10:10:53 +08:00
sunby 751c9d4e81 Refactor meta.go
Signed-off-by: sunby <bingyi.sun@zilliz.com>
2021-04-09 09:55:04 +08:00
xige-16 332b14c915 Delete querynode’s redundant load IO
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-04-07 18:29:19 +08:00
bigsheeper fe3fa8dd1d Fix search service unittest
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-04-01 15:39:23 -05:00
Xiangyu Wang 87a1e0b662 Reorganize msgstream
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-04-02 13:48:25 +08:00
xige-16 c10b36f188 fix getting wrong offset when segment preinsert
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-31 16:16:58 +08:00
xige-16 a6f1de036b Optimize search performance in query node
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-30 22:16:58 +08:00
zhenshan.cao efbf4452c5 Add log for port changed or proto unmarshal error
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-30 20:19:30 +08:00
godchen 8f58d5641b Perfect trace and add trace init
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
2021-03-30 16:36:34 +08:00
Xiangyu Wang 8d66388b8c Add logs for querynode
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
2021-03-30 15:00:14 +08:00
yukun 432b53149b Refactor msgstream
Signed-off-by: yukun <kun.yu@zilliz.com>
2021-03-26 20:10:11 +08:00
bigsheeper 8a4c6a039f Add search log
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-25 18:09:47 -05:00
xige-16 7e60460aee close timeTickStream in flow graph when the collection is released
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-26 18:40:04 +08:00
bigsheeper dfb68a7e46 Remove unused segment
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-25 16:08:08 -05:00
FluorineDog f39dcdb8f3 Support error code in segcore
Signed-off-by: FluorineDog <guilin.gou@zilliz.com>
2021-03-26 16:18:30 +08:00
neza2017 eca1e25470 Add msg id at master service
Signed-off-by: neza2017 <yefu.chen@zilliz.com>
2021-03-26 15:13:33 +08:00
bigsheeper 1e4afda824 Add mutex for query node map, search buffer and dataSyncService
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-24 15:40:59 -05:00
quicksilver 7ad9b36207 Update reviewdog/action-hadolint github action version to v1.16.1
Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>
2021-03-25 14:41:46 +08:00
bigsheeper 350e0df5f7 Refactor search service
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-22 16:59:46 -05:00
bigsheeper 1634d75980 Release flowgraph properly
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2021-03-22 12:49:50 -05:00
xige-16 1a1e864709 Fix query node crash when empty search
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2021-03-22 19:20:56 +08:00
zhenshan.cao c2734fa55f Fix bug and enchance system
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2021-03-22 16:36:10 +08:00
groot 2280791128 Add memory message stream for search
Signed-off-by: groot <yihua.mo@zilliz.com>
2021-03-19 20:16:04 +08:00