2c03287e79
Improve log level in proxy QueryTask ( #16234 )
...
Use warn instead of debug in error case
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-28 23:11:26 +08:00
a77dd1034c
Adopt the extended state in DataCoord ( #16200 )
...
ChannelWatchInfo with ToWatch, ToRelease will trigger a timer.
ChannelManager now reacts to different ChannelWatch states.
- WatchSuccess > log this info
- WatchFailure/WatchTimeout > ToRelease
- ReleaseSuccess > Delete, reassign if not from DropCollection
- ReleaseFailure/ReleaseTimeout > Cleanup subscription and Delete,
reassgin if not from DropCollection.
Some Notes:
1. Reassignment will add this channel to buffer if there's only one node.
See also: #15846
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-28 22:33:27 +08:00
973ea406ca
Improve log of show collections ( #16233 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-28 21:47:27 +08:00
c8688a6275
Organize log for GetIndexFilePaths ( #16230 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-28 21:27:27 +08:00
4f08f6d8c6
Workaround for knowhere under windows ( #16214 )
...
issue #16205
Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-28 17:41:27 +08:00
178eb8cce1
Modify proto to support replicas ( #16218 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-28 16:43:27 +08:00
40c703dacd
Support check index state in root coord, for bulk load feature. ( #16198 )
...
issue: #15604
/kind feature
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-28 16:41:28 +08:00
e0ffe54547
Add interface to get replications by collection ID ( #16213 )
...
issue: #16158
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-28 14:35:27 +08:00
3e3756d3dc
Upgrade to knowhere v1.1.1 to support Mac M1 build ( #16210 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-28 14:33:26 +08:00
478f6ca11e
Add cache for vector chunk ( #15912 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-26 22:05:26 +08:00
205c92e54b
Support insert string data ( #15993 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-25 14:27:25 +08:00
227889b0d0
Add replica interfaces for Meta and Cluster ( #16190 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-25 14:03:25 +08:00
f4ebd3a9ce
Upgrade to knowhere v1.1.0 ( #16186 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-25 13:49:25 +08:00
a00968fdfe
Fix unregister in datacoord cluster ( #16182 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-25 12:09:25 +08:00
0d16ecf0dd
Add protobuf messages to support multi memory replica ( #16153 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-25 11:13:25 +08:00
a23ca745bb
Add Etcd support for Import Tasks. ( #16164 )
...
issue: #15604
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-25 11:03:25 +08:00
2fcf7b9760
Fix GetMemoryCount, get min as total memory ( #15931 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-24 20:41:25 +08:00
9deb245d77
Fix data race in unittest ( #16165 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-24 20:19:25 +08:00
7fee4743f5
Improve log print in OOM error info ( #16170 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-24 20:17:25 +08:00
48706f416f
Migrate scalar index from knowhere ( #16174 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-24 14:57:26 +08:00
661469445c
Copy index file content to c memory ( #16162 )
...
Might be the root cause of #15896
The index content bytes were passed via unsafe.Pointer,
which shall be copy to cpp managed memory inmediately
This solution might cause a lot of memory consumption.
Maybe we need to store the index bytes in cpp memory in the first place.
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-24 14:55:25 +08:00
63a71e247a
Loading segments concurrency level fallbacks to the value that won't cause OOM ( #16172 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-24 13:39:25 +08:00
ca129d4308
Support configurable msgstream ( #16131 )
...
Signed-off-by: xiyichan <2863768433@qq.com>
2022-03-24 10:15:25 +08:00
478890a700
modify import proto ( #16155 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-23 19:29:25 +08:00
321105cc01
Add column-based insert interface in segcore ( #16100 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-23 19:05:25 +08:00
1c4b949a1d
modify import rpc interface ( #16129 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-22 15:11:24 +08:00
2078b24dca
Prepare for import ( #16083 )
...
Signed-off-by: groot <yihua.mo@zilliz.com>
2022-03-21 15:47:23 +08:00
f8d9bc919d
Unify interface of vector index & scalar index. ( #15959 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-21 14:23:24 +08:00
ad41e2a082
Tidy getQueryChannel for readability ( #16096 )
...
- Remove useless codes in query_coord.go
- Tidy slice initialization in LoadCollection task
- Add ut for getQueryChannel
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-21 11:25:24 +08:00
97e5d77953
implements restful entities api ( #15916 )
...
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-03-21 11:21:23 +08:00
15a3fe41c5
[skip e2e]Fix comments of exposed functions in QC ( #16094 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-21 11:17:24 +08:00
787d4f8bfd
Refine prometheus metrics and etcd log ( #16084 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-18 15:51:22 +08:00
31c5b286da
Fix wrong unittest, check whether loading collection will cause OOM ( #16068 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-18 13:17:22 +08:00
78557ca6d7
Replace minio kv with minio chunk manager ( #15936 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-03-17 18:03:23 +08:00
3cd28420f1
Support compile under windows ( #15786 )
...
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
* using gofrs/flock instead of syscall.Flock
* using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`
issue #7706
Signed-off-by: Ji Bin <matrixji@live.com>
2022-03-17 17:17:22 +08:00
d65cf5309f
Remove useless LogMgr ( #16080 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-16 21:19:21 +08:00
d4ad785b76
Bump go version in documentation ( #15979 )
...
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
2022-03-16 18:51:21 +08:00
a3a29fed06
Add entry point for embedded Milvus. ( #16008 )
...
/kind feature
issue: #15711
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-16 16:51:22 +08:00
256ccb8f65
Fix rootcoord goroutine leak ( #16071 )
...
issue: #15658
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-16 16:47:21 +08:00
ffdad82b9c
Set load info in reschedule tasks ( #16065 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-16 11:43:21 +08:00
cf4c014b2e
Remove collect query result by mq ( #15988 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-16 10:19:21 +08:00
f3eeecf146
Reduce the number of metrics ( #16030 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 21:51:21 +08:00
c984c4ea73
Set load meta info in load balance task ( #16060 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-15 20:37:21 +08:00
b61e8201e7
Refactor querynode historical replica ( #15992 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-15 19:15:20 +08:00
c51c8c0a74
Fix get flused segment errror handling ( #15155 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-15 18:51:21 +08:00
d3d5a41b76
Fix memory leak casued by unclosed channel in msgstream ( #16051 )
...
issue: #16045
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-03-15 15:55:21 +08:00
0bd07daae3
Add grpc healthy check ( #16050 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-15 15:05:21 +08:00
97b1ed7bca
Remove reader API from Mqstream and replace it with consumer API ( #15971 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-15 14:45:22 +08:00
d4a0c5eba6
Alter comments in parallel.go for readability ( #16032 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-15 10:26:01 +08:00
09b2c46b61
Support create collection with VarChar field ( #15904 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-03-14 23:20:02 +08:00
caf9cbfcd4
Fix load failure and remove parition release related code ( #16038 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-14 23:18:02 +08:00
fab0263616
Use RmqFactory in DataNode unit test ( #16039 )
...
Use Rocksmq instead of Pulsar in DataNode unittest
- Avoid pulsar seek problem caused timeout issue
- Remove third party component dependency in unit test
See also: #15986
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-14 19:22:01 +08:00
d4225117ee
Enable Zstd compression for binlog ( #15840 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-14 16:00:01 +08:00
6a66faa2fb
Explictly specify Load operation meta ( #16027 )
...
Fix #15996
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-14 09:50:01 +08:00
29975a7a26
Remove Consume() API from mqstream interface ( #15886 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-11 20:09:59 +08:00
89356ebcb9
Modify import rpc interfaces ( #16006 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-11 17:13:59 +08:00
7d2934e4c2
Add methods to compress/decompress small blocks ( #15980 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-11 14:55:59 +08:00
3121619758
Chunk manager support scalar data ( #16010 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-11 14:39:59 +08:00
4f3d5fb6eb
Make load operation parallel improve performance ( #15928 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-11 14:28:01 +08:00
8f52e5b6c7
Add unit tests in segcore ( #15960 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-10 16:33:59 +08:00
d5bae6710f
Make all logs configurable for standalone/embedded Milvus. ( #15926 )
...
Resolves : #15919 , #15708
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-10 10:35:59 +08:00
5fdef607d5
Add rpc interfaces for import ( #15930 )
...
Signed-off-by: yhmo <yihua.mo@zilliz.com>
2022-03-09 18:33:59 +08:00
d013134b44
Enhance error message when coordinator not healthy ( #15950 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-09 15:57:58 +08:00
c2fe12bc97
Handle error in topic creation of rocksmq ( #15922 )
...
Reopen for: #14429
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-09 15:47:59 +08:00
6513be5c44
Extend watch/release channel logic in DataNode ( #15925 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-09 15:39:58 +08:00
9ad5c14a4a
Return error when create message stream failed in QueryNode ( #15317 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-09 15:17:59 +08:00
4f8c08f912
fix checkSegmentSize overflows int64 ( #15858 )
...
Signed-off-by: guyan <632722781@qq.com>
2022-03-09 13:29:58 +08:00
565524ed3c
Add zstd compressor in util ( #15779 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-08 18:17:58 +08:00
dff08dbf47
Refactor querynode unittest ( #15929 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-08 17:39:58 +08:00
74f66dce3b
Fix passing empty binlog content in compaction ( #15909 ) ( #15927 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-08 16:34:01 +08:00
6fec7ed6fb
Add search benchmark in QueryNode ( #15854 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-08 10:15:59 +08:00
7124e91d53
Refactor assigning loading segments task in QueryCoord ( #15895 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-03-07 19:23:58 +08:00
a5c3f4eef4
Add CompareFailErr for CAS operation error check ( #15893 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-07 18:01:58 +08:00
2512dff261
Fix misuse of timerecord ( #15868 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-04 15:45:56 +08:00
08d12e62f3
Add ByteSlice Method for embed etcd ( #15844 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-04 15:14:01 +08:00
f71651e294
Support column-based insert data in message stream ( #15802 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-04 15:09:56 +08:00
3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig ( #15843 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 11:17:56 +08:00
57b97ccb9c
Let search fail as if one query_node return error result ( #15855 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-04 10:43:56 +08:00
7a3a721380
Reconstruct mqstream module ( #15784 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-03-03 21:57:56 +08:00
54862a31b0
Extend ChannelWatchState in protobuf ( #15847 )
...
Related #15846
/kind enhancement
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-03 18:43:56 +08:00
2be217bcf4
Specify appropriate timestamp in initPKBloomFilter ( #15823 ) ( #15826 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-03 17:33:56 +08:00
3f34bc1ab7
Ban DataType_String in CreateCollection ( #15838 )
...
Fix #15836
/kind enhancement
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-03-03 16:57:56 +08:00
88527fc0a9
Fix metrics label conflict ( #15839 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-03 16:05:57 +08:00
27a7875a15
add array hashKeys empty check ( #15796 )
...
fix #15791 https://github.com/milvus-io/milvus/issues/15791
Signed-off-by: zhuyaguang <zhuyaguang1368@163.com>
2022-03-03 00:33:56 +08:00
9f4c59e621
Add prometheus metrics for RootCoord ( #15684 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 21:11:57 +08:00
09a27d1620
Add prometheus metrics for indexnode ( #15640 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 20:49:56 +08:00
92c8e32ebd
Let MemoryKV.Load return error when key not exist ( #15814 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-02 18:51:55 +08:00
442e791128
Add ByteSlice Method for memkv ( #15774 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-03-02 17:29:56 +08:00
63eec7ffc8
Refactor logs in binlog_io to reduce output ( #15827 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-02 16:49:55 +08:00
d183748484
Add some prometheus metrics for proxy ( #15582 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-03-02 16:23:55 +08:00
bd7d294c6d
Refine log level in data ( #15821 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 15:35:55 +08:00
12eb8c3da5
Add prometheus metrics for QueryNode ( #15649 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-03-02 14:49:55 +08:00
827389c704
Refine tso log print ( #15813 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-03-02 14:31:55 +08:00
f1aec49a74
Fixbug: building index return nil by mistake when error not occur ( #15809 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-03-02 09:41:56 +08:00
e394ba8c88
Refactor rocksmq comments and logs for readability ( #15801 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-03-01 18:55:55 +08:00
529098fd87
[skip ci] Update knowhere/CMakeLists.txt ( #15804 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-01 17:55:55 +08:00
001567640e
Fix shallow copy of typeutil.AppendFieldData ( #15790 )
...
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
2022-03-01 15:13:54 +08:00
503724be19
Optimize CMakeLists.txt under internal/core ( #15770 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-03-01 10:31:55 +08:00
a77e0562b2
Make C++ log level configurable and add log configuration while building embedded milvus. ( #15743 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-03-01 10:15:55 +08:00
2e4391e6c5
Add lru cache ( #15733 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-02-28 21:43:55 +08:00
13177a90af
Add prometheus metrics for DataNode ( #15650 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
2022-02-28 19:11:55 +08:00
d6857f4b24
Remove globalZapWrapper in logutil.go ( #15726 )
...
issue: #15720
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-28 17:23:54 +08:00
e1bfd14db1
Add Prometheus metrics for QueryCoord ( #15606 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
Co-authored-by: jingkl <34296482+jingkl@users.noreply.github.com>
2022-02-28 16:51:55 +08:00
b41992eda1
Fix querycoord ut bug ( #15752 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-28 11:27:54 +08:00
0cfb6a85be
Make compaction not generate empty segment ( #15707 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-28 10:17:54 +08:00
e8edaa02fa
Refine code of DataNode binlogIO ( #15759 )
...
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-28 10:09:54 +08:00
867cf620ee
Add Extract() method to take out C pointer from BytesConverter without releasing it ( #15614 )
...
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-27 09:47:53 +08:00
6a48071a5c
Set BaseMsg for LoadSegmentRequest, prevent crashing in log printing ( #15725 )
...
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2022-02-26 20:09:54 +08:00
adca79facc
Fix centos build error ( #15760 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-25 18:49:54 +08:00
d67c801511
Add bytes methods for RocksdbKV ( #15749 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-02-25 17:19:53 +08:00
1a9933a7c3
Add methods to load/save bytes for MinIOKV ( #15732 )
...
Signed-off-by: yah01 <yang.cen@zilliz.com>
2022-02-25 16:21:53 +08:00
cdcb3627b4
Make Etcd log level configurable. ( #15737 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-25 15:03:53 +08:00
a05df77e96
Remove unused ValueKV ( #15748 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-25 14:47:52 +08:00
b56ec7ea97
Add ByteSlice Method for Etcd-kv ( #15738 )
...
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
2022-02-25 11:25:53 +08:00
fcf0887d92
Add and implement chunk manager interface. ( #15541 )
...
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
2022-02-24 23:53:53 +08:00
1e8e3eb912
Fix wrong flushSegmentInfos in watchdmChannel request ( #15740 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-24 19:23:52 +08:00
6f85b49a4f
Add more compaction test cases ( #15715 )
...
Test merge compaction with 2 segments with the same PK and only 1
valid deletion.
This test can varify the compaction behavior of DataNode for the
dup PK in different segment cases.
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-24 11:27:53 +08:00
83fdc1da7c
Add web server in proxy with sample handler ( #15647 )
...
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2022-02-23 14:37:52 +08:00
b0923f1299
Fix DataCoord panics if message queue service quits before it ( #15702 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-23 11:09:52 +08:00
54b8b24151
Rename variable names for better readibility ( #15700 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-22 22:15:52 +08:00
ce9662c140
Fix data race in test ( #15690 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-22 17:43:52 +08:00
9a6ee61731
Change all fields in Expr to const to avoid modification ( #15663 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-22 14:45:51 +08:00
3eb074a9b0
Update some comments for data coord with some minor code refactoring. ( #15686 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-22 13:15:51 +08:00
782f350da5
Modify parameter spelling of task queue ( #15581 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-22 10:25:51 +08:00
b435c422c1
Add prometheus metrics for IndexCoord ( #15638 )
...
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
2022-02-21 17:15:51 +08:00
d011073269
Fix DataNode panic when compact empty segment ( #15580 )
...
remove the TODO in compactor
See also: #15573
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-18 18:47:51 +08:00
142848fcc3
Abandon using protobuf to pass binaryset parameter ( #15626 )
...
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
2022-02-18 18:39:50 +08:00
83f1358408
Update codecov.yml ( #15639 )
...
Signed-off-by: Jenny Li <jing.li@zilliz.com>
2022-02-18 18:29:50 +08:00
23dfbf1f34
Fix eventmanager retry interval ( #15631 )
...
See also: #15608
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-18 17:31:49 +08:00
2e676a4b78
Extract segments info into a new struct. ( #15537 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-18 15:27:50 +08:00
5751759c85
Implement mix compaction logic ( #15542 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-18 14:51:49 +08:00
daf25bcb81
Add log when remove keys in garbage collector. ( #15453 )
...
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
2022-02-18 11:59:49 +08:00
dc65b1aeea
Switch to knowhere v1.0.1. Also switch to forked gorocksdb v0.0.1 ( #15603 )
...
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
2022-02-17 19:05:49 +08:00
c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted ( #15497 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-15 15:07:48 +08:00
684110bc9a
Avoid to copy during converting C bytes to Go bytes ( #15551 )
...
* Avoid to copy during converting C bytes to Go bytes
Signed-off-by: yah01 <yah2er0ne@outlook.com>
* Add BytesConverter to achieve no-copy converting C bytes to Go bytes
Signed-off-by: yah01 <yah2er0ne@outlook.com>
2022-02-15 14:17:48 +08:00
7daed4ee47
Return flush state true when segment in Flushing state ( #15530 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-10 16:55:46 +08:00
d499ffc421
Fix confusing logs in rootcoord ( #15431 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-10 13:51:47 +08:00
a18d6b46a4
Fix watch dml channel fail because of no collection meta ( #15436 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-10 13:19:46 +08:00
be42627ab2
Move Logutil to Util dir ( #15447 )
...
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
2022-02-09 18:55:46 +08:00
87dd4e8f92
Fix clean up data dir failed for TestEtcdRestartLoad ( #15506 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-02-09 18:37:45 +08:00
cb9c63d039
Remove index dir ( #15467 )
...
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
2022-02-09 16:07:45 +08:00
8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig ( #15501 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-09 14:41:45 +08:00
307a8ce535
Support compile and run on Mac ( #15491 )
...
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
2022-02-09 14:27:46 +08:00
f43d96b58d
[skip e2e] Modify code comments according to the naming conventions ( #15173 )
...
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2022-02-09 10:19:56 +08:00
a157487f8c
[skip e2e] Modify code comments according to the naming conventions ( #15172 )
...
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2022-02-09 10:18:02 +08:00
c7e93f95aa
[skip e2e] Modify code comments according to the naming conventions ( #15171 )
...
Signed-off-by: guoxiangzhou <xiangzhou.guo@zilliz.com>
2022-02-09 10:14:05 +08:00
2263425cde
Fix EntriesNum is not passed in flush task ( #15482 )
...
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2022-02-09 10:05:46 +08:00
55cdc5aa35
Get indexInfo and segment size by queryCoord ( #14207 )
...
Signed-off-by: xige-16 <xi.ge@zilliz.com>
2022-02-08 21:57:46 +08:00
1f337891e1
Rename base_param to service_param, rename global_param to component_param ( #15463 )
...
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
2022-02-08 20:57:47 +08:00
43b6c295a4
Put logs of compaction elapse in defer ( #15485 )
...
Before this PR, time elapse are logged after `compact()` finished
normally. No elapse will be logged with any errors.
This PR put elapse logs in defer once time-consuming steps are finished.
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2022-02-08 19:15:45 +08:00
congqixia
XuanYang-cn
Jiquan Long
cai.zhang
Ji Bin
yah01
Ten Thousand Leaves
Bingyi Sun
Cai Yudong
godchen
xige-16
bigsheeper
xiyichan
groot
Letian Jiang
shaoyue
Xiaofan
edward.zeng
jaime
zhenshan.cao
顾炎
朱亚光
Letian Jiang
Jenny Li
yah01
guoxiangzhou