milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,461 Commits
130 Branches
115 Tags
556 MiB
Commit Graph

326 Commits (28200acb67c24d86e142d4064edbdecb720c2557)

Author SHA1 Message Date
wayblink 3927ae9952
Return all flushed segments and time-of-seal in Flush interface (#18882) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-09-09 09:58:37 +08:00
Jiquan Long a5e2d6b6fb
Refactor RootCoord (#18930) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
Co-authored-by: xaxys <tpnnghd@163.com>
 2022-09-05 13:29:11 +08:00
bigsheeper 8a45381e53
Make DescribeCollection return collectionName (#19000) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-09-02 19:20:59 +08:00
XuanYang-cn c054873908
Add limit to proto (#18950) 
See also: #18893

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-09-01 18:54:58 +08:00
XuanYang-cn 867ea63bdd
Move TopK check inside parseQueryInfo (#18892) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-08-30 10:32:56 +08:00
cai.zhang c924f73105
Refactor for IndexCoord to support cloud (#18643) 
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-08-25 15:48:54 +08:00
Jeng.Gwan 638f6c36e9
Support to get real row count of segment (#18115) 
Signed-off-by: xaxys <zheng.guan@zilliz.com>
 2022-07-18 09:58:28 +08:00
congqixia a9aff3a678
Allocate ID for each insertMsg (#18217) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-07-11 14:38:24 +08:00
bigsheeper 0fb5e9c1a9
Remove load collection check in Proxy (#18166) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-08 16:12:25 +08:00
bigsheeper 696d747afa
Check if loaded in Proxy when LoadCollection (#18074) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-07-05 21:16:20 +08:00
XuanYang-cn e43f487f02
Make LoadCollection compatible (#17669) 
See also: #17656

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-06-21 17:26:15 +08:00
cai.zhang 53a49a9255
GetIndexState only get the state generated call CreateIndex (#17549) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-16 20:12:11 +08:00
xing.zhao e9dcda161f
Add aliases return value in describeCollection task (#17516) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: 赵星 <zhaoxing@zhaoxings-MacBook-Pro.local>
 2022-06-13 19:22:10 +08:00
Letian Jiang dfaed5acdd
Add QueryServiceAvailable field in ShowCollections (#17456) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-06-09 18:20:07 +08:00
Jiquan Long 2ca81620ea
Reduce lock operations when get dml stream (#17468) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-09 17:34:09 +08:00
xige-16 b5c11a216d
Alter varChar type params's name to max_length (#17409) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-06-07 15:58:06 +08:00
Jiquan Long adf3b14027
Refine channels management in Proxy. (#17334) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-06-02 15:34:04 +08:00
congqixia c88514bc49
Remove not used QueryChannel in Proxy and Query Cluster (#16856) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-30 19:50:04 +08:00
bigsheeper 9eeec4a2d5
Add collection load cache and InvalidateCollMetaCache by collID (#16882) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-05-19 10:13:56 +08:00
XuanYang-cn a1480aaf76
Clear shard leaders cache when releasing (#16942) 
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-17 11:11:56 +08:00
Xiaofan 52fc9a558c
Fix valid error message (#16953) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-13 18:19:53 +08:00
xige-16 9e6d9fb7d8
Print collectionName when load failed (#16878) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-11 09:47:53 +08:00
xige-16 8fcf349609
Fix create index failed after drop index (#16836) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-09 11:35:53 +08:00
xige-16 009b773b6e
Delete set default type param max_len for varchar field (#16794) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-05-06 15:59:51 +08:00
Jiquan Long 2fe8677cbf
Enable dimension check in Proxy when create index request received (#16718) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 18:01:49 +08:00
xige-16 515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
zhenshan.cao 0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
zhenshan.cao a3f7bb5f2d
Remove the collectionID label from metrics (#16571) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-26 17:29:45 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
XuanYang-cn e9090a62ab
Rearrange search/queryTask for readability (#16325) 
`searchTask` has 683 lines of code, `queryTask` has 485 lines of code.
`task.go` contains 4k+ codes including `searchTask` and `queryTask`.

It was so much pain navigating codes of searchTask and queryTask though task.go,
task.go and task_test.go are literaly unreadable.

This PR moves
1. 650+ lines of code of `searchTask` from `task.go` to `task_search.go`.
2. 1.4k+ lines of test code of `searchTask` from `task_test.go` to
   `task_search_test.go`.
3. 450+ lines of code of `queryTask` from `task.go` to `task_query.go`.
4. 200+ lines of test code of `queryTask from `task_test.go to
   `task_query_test.go`.

This PR also rearrange methods positions of `searchTask` and
`queryTask`:
-  Putting the most important methods `PreExecute`, `Execute`, and
   `PosExecute` at the beginning of the file.
-  Moves interface methods `ID`, `SetID`, `Type`, `BeginTs`, etc.
   that nobody cares about to the bottom of the file.

See also: #16298

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-04-01 18:59:29 +08:00
XuanYang-cn a0bb758be5
Refine log in proxy searchTask (#16263) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-03-30 22:09:28 +08:00
zhenshan.cao 11cea5aad0
Refine proxy log (#16246) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-29 02:29:27 +08:00
congqixia 2c03287e79
Improve log level in proxy QueryTask (#16234) 
Use warn instead of debug in error case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-28 23:11:26 +08:00
Ten Thousand Leaves 40c703dacd
Support check index state in root coord, for bulk load feature. (#16198) 
issue: #15604

/kind feature

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-03-28 16:41:28 +08:00
xige-16 205c92e54b
Support insert string data (#15993) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-25 14:27:25 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
cai.zhang 2512dff261
Fix misuse of timerecord (#15868) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-04 15:45:56 +08:00
Jiquan Long f71651e294
Support column-based insert data in message stream (#15802) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-04 15:09:56 +08:00
Cai Yudong 57b97ccb9c
Let search fail as if one query_node return error result (#15855) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 10:43:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
congqixia 3f34bc1ab7
Ban DataType_String in CreateCollection (#15838) 
Fix #15836
/kind enhancement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-03 16:57:56 +08:00
朱亚光 27a7875a15
add array hashKeys empty check (#15796) 
fix #15791 https://github.com/milvus-io/milvus/issues/15791
Signed-off-by: zhuyaguang <zhuyaguang1368@163.com>
 2022-03-03 00:33:56 +08:00
cai.zhang d183748484
Add some prometheus metrics for proxy (#15582) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-02 16:23:55 +08:00
Filipe Caixeta dee1961b4d
Make SearchResults and QueryResults return collectionID (#15377) 
* Make SearchResults and QueryResults return collectionID

Signed-off-by: Filipe Caixeta <filipecaixeta@gmail.com>

* add return collection name from QueryRequest and SearchRequest

Signed-off-by: Filipe Caixeta <filipecaixeta@gmail.com>

* remove comment

Signed-off-by: Filipe Caixeta <filipecaixeta@gmail.com>
 2022-01-27 22:55:41 +08:00
Cai Yudong 6323907566
Use CommonConfig in GlobalParams for all components (#15106) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 19:03:35 +08:00
shaoyue 2e8dd74c7e
[skip e2e] Add comment for proxy task (#15089) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-01-10 18:33:46 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
guoxiangzhou 1807ba534d
[skip e2e] Modify log text according to the naming conventions (#14969) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-07 18:41:23 +08:00
guoxiangzhou 8ec3c69447
[skip e2e] Modify log text according to the naming conventions (#14970) 
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
 2022-01-07 18:39:22 +08:00
congqixia 6400263f3c
[skip e2e] Improve SearchTask log format (#14753) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-01-04 14:15:39 +08:00
Jiquan Long 77ef791b6e
[skip e2e] Add comment for minFloat32 (#14752) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-01-04 14:13:44 +08:00
Xiaofan 98e4ff33a8
Fix RocksDB Slow (#14614) 2021-12-30 20:08:15 +08:00
congqixia 7bb1d6267d
[skip e2e] Add comment for CreateAliasTask methods (#14539) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-29 22:40:27 +08:00
zhenshan.cao 32a5126705
Add the limitation of the traceable interval to timeTravel (#14498) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-29 21:29:45 +08:00
congqixia ffe44ae1ea
[skip e2e] Add comment for CreateAliasTask (#14463) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-28 22:22:46 +08:00
congqixia 307b07363b
[skip e2e] Add comment for IDs2Expr (#14352) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-28 09:48:15 +08:00
shaoyue 26ed17a905
[skip e2e] Add comment for AlterAliasTask (#14140) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-12-24 13:33:45 +08:00
Xiaofan 24e2faf110
Fix has collection err (#14089) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-23 21:46:10 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
shaoyue e8ebd926e0
[skip e2e] Add comment for internal/proxy/task.go (#14031) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-12-23 11:05:09 +08:00
edward.zeng 8eb4d9a2b4
[skip e2e] Add comment to proxy task (#14003) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-22 20:29:29 +08:00
Jiquan Long bdb001b20d
Add log of query to record time spent in mq (#13526) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-22 10:11:02 +08:00
Jiquan Long 8a63dba149
Add consistency level field in collection meta (#13766) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-21 19:49:02 +08:00
xige-16 dc7fb594af
[skip e2e]Improve log for QueryNode spell (#13809) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-20 23:10:51 +08:00
shaoyue 22e0fc8701
[skip e2e] Add comment for DropAliasTask (#13626) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-12-17 18:46:38 +08:00
zhenshan.cao bfdf6eb402
[skip e2e]Fix datarace in proxy (#13607) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-17 16:32:52 +08:00
Jiquan Long 00c9797d91
Add log of insert to record time spent in mq (#13527) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-16 22:27:42 +08:00
Jiquan Long cb51df5985
Add log of search to record time spent in mq (#13525) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-16 22:22:06 +08:00
Xiangyu Wang 405b3cd932
Use the same lint rules with golint in revive (#13324) 
Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>
 2021-12-14 15:31:07 +08:00
Cai Yudong b5d639a1ab
Use typeutil.ProxyRole instead of Params.RoleName in proxy (#13217) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-13 10:01:18 +08:00
bigsheeper 214f40b209
Add timeout ts for search and query (#12890) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-08 14:15:03 +08:00
dragondriver a2a1f9d08c
Disable multiple vector fields (#12691) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-06 10:03:34 +08:00
Cai Yudong 147fc41a67
Update proxy debug log (#12591) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-02 12:41:32 +08:00
dragondriver 1db1cf2104
Fix similarity correlation (#12511) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-01 15:26:02 +08:00
dragondriver 6d92b8499c
Revert #11901, Fix wrong jaccard distance (#12487) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-30 20:31:35 +08:00
Cai Yudong 74a4a4e15b
Fix golint warnings for proxy (#11895) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-29 12:25:17 +08:00
godchen 927f72d588
Add delete task channel stats interface (#12250) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-24 16:01:17 +08:00
shaoyue b231965e15
[skip ci] Add comments for exported funcs in internal/proxy/task.go (#11980) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2021-11-17 16:13:17 +08:00
dragondriver 8c951217ee
Fix wrong distances caused by metric type (#11901) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-11-16 19:11:10 +08:00
yukun 27d711e5d2
Insert task hash by primarykeys (#11819) 
Signed-off-by: fishpenguin <kun.yu@zilliz.com>
 2021-11-16 14:21:27 +08:00
XuanYang-cn 99894e451d
[skip ci]Update license for proxy task (#11611) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-11-10 23:55:48 +08:00
Cai Yudong e27283abbf
Update query reduce log (#11402) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-08 16:07:34 +08:00
Cai Yudong d32c77e99b
Update search reduce log (#11400) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-08 16:03:30 +08:00
godchen 88c9067873
Add hashvalue warning (#10668) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-05 14:20:55 +08:00
bigsheeper c99d1a56bf
Refactor proxy ticker to fix guarantee_time not work (#11248) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-11-05 09:14:02 +08:00
Cai Yudong 0fac38000c
Correct empty retrieve result handling (#11219) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-04 18:08:32 +08:00
godchen a0a3a889e7
Add common endian for global use (#11092) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-11-02 18:16:32 +08:00
Cai Yudong 70188990dc
Remove primary key duplicated query result on proxy (#10967) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-01 10:27:56 +08:00
Cai Yudong da0cb4a702
Use primary key only to check search result duplicate (#10949) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-11-01 10:06:03 +08:00
Cai Yudong 0049837dc4
Add utils API AppendFieldData (#10910) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-29 18:06:43 +08:00
Cai Yudong 5ad4cdda25
Clean search result duplicates removal debug log (#10769) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-27 20:02:26 +08:00
Cai Yudong 24f9411249
Optimize queryTask PostExecute (#10739) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-27 17:34:35 +08:00
Cai Yudong c51155a542
Optimize decodeSearchResults (#10728) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-10-27 14:10:27 +08:00
cai.zhang 3734ff49ac
Add default value for the parameter roundDeciaml (#9626) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-10-26 18:24:23 +08:00
zhenshan.cao bb08df28f3
Fix golint error in proxy (#10620) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-26 10:38:41 +08:00
zhenshan.cao 4b0fd28de7
Fix golint error in proxy (#10621) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-10-25 23:42:29 +08:00
godchen daaa9428ea
Fix delete msg hash error (#10599) 
Signed-off-by: godchen <qingxiang.chen@zilliz.com>
 2021-10-25 22:20:20 +08:00