milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,151 Commits
126 Branches
113 Tags
558 MiB
Commit Graph

6862 Commits (2fdbe7f4c0543ad22f18f12777f0f13d217f11f3)

Author SHA1 Message Date
yihao.dai 092d743917
Add support for getting vectors by ids (#23450) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-23 09:00:32 +08:00
sre-ci-robot ccba1334dd
Update knowhere commit (#23595) 
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
 2023-04-21 15:34:31 +08:00
Enwei Jiao 2ced6660e2
Simplify makefile (#23583) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-04-21 14:16:33 +08:00
yah01 546080dcdd
Support to retrieve json (#23563) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-21 11:46:32 +08:00
zhenshan.cao 03ec804e68
Change default shard number to 1 (#23586) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-21 07:08:32 +08:00
yah01 71364056f9
Fix QueryNode may report succeeded to unsubscribe channel early (#23580) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-20 19:30:33 +08:00
LeoReeYang a2ab94ec3d
remove unused code. (#23572) 
Signed-off-by: LeoReeYang <LeoReeYang@outlook.com>
 2023-04-20 17:56:32 +08:00
XuanYang-cn d56771b7b7
Fix return too many nodeIDs (#23397) 
See also: #23396

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-04-20 13:50:31 +08:00
congqixia 4a6c2b78a7
Fix multiple transferNewSegment and removeSegment causes panic (#23564) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-20 13:48:31 +08:00
yah01 aa2985490c
Retrieve page size by getpagesize() (#23561) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-20 12:36:30 +08:00
Enwei Jiao 967a97b9bd
Support json & array types (#23408) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
 2023-04-20 11:32:31 +08:00
congqixia 4a22af6e1a
Unwatch channel in watch buffer (#23548) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-20 10:34:31 +08:00
Cai Yudong 2725d38b9e
Add COSINE metric type (#23350) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-04-20 10:20:31 +08:00
wei liu 4336ed8609
fix node up (#23415) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-20 09:52:31 +08:00
congqixia 4fe363c4b2
Add KNN cgo pool (#23526) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-19 23:56:31 +08:00
Enwei Jiao f14c221123
Revert "Opt compile time for Mac (#23453)" (#23533) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-04-19 16:52:30 +08:00
yah01 c7c89511b6
Fix data race of timerecord (#23524) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-19 14:18:30 +08:00
Enwei Jiao e9dcba3050
Opt compile time for Mac (#23453) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-04-19 10:24:30 +08:00
congqixia 4508786819
Fix test_watch_channel unit test unstable (#23507) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-19 10:00:31 +08:00
wei liu 3933080511
skip to balance redundant segment (#23490) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-18 18:32:32 +08:00
wei liu cbfe7a45ef
fix pull target (#23491) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-18 18:30:32 +08:00
congqixia eb690ef033
Fix querynodev2 stop logic (#23487) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-18 17:56:30 +08:00
congqixia d83654c33f
Add Close method for ChannelManager in datacoord (#23493) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-18 17:54:31 +08:00
wayblink b23723535c
Add log in AddImportSegment.getChannelLatestMsgID (#23480) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-04-18 17:50:31 +08:00
congqixia b5a73e6d1a
Add OWNERS files for querycoordv2 sub pkgs (#23489) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-18 15:52:30 +08:00
yah01 5aa0ddf77b
Refine field storage of sealed segment (#23464) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-18 11:24:30 +08:00
zhenshan.cao 4a32b842e8
Improve the check logic of channel remove (#23473) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-18 02:58:30 +08:00
wei liu 3ad9ff7a9a
fix release segment failed (#23462) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-17 18:46:30 +08:00
Bennu f63d52944f
Update knowhere version from main to commit_id (#23437) 
Signed-off-by: Bennu-Li <yunmei.li@zilliz.com>
 2023-04-17 16:32:33 +08:00
aoiasd 695cc769df
Fix watch event timeout when IO Pool busy (#23293) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-04-17 11:02:30 +08:00
Jiquan Long adfa415b74
Optimize log trace (#23431) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-17 11:00:30 +08:00
aoiasd f603456b72
remove segment from datanode meta when segment dropped (#23398) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-04-17 10:58:30 +08:00
cqy123456 7ff6a3a246
Set query node thread pool size in milvus.yaml (#23286) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-04-17 10:44:29 +08:00
yah01 7c4cafc83c
Fix task merge doesn't work (#23405) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-14 18:18:29 +08:00
Enwei Jiao 288582b2d9
Optimize mac build time (#23390) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-04-14 17:56:29 +08:00
zhenshan.cao a88178a5ea
Add slow log for read (#23272) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-14 15:30:29 +08:00
yah01 bed8d6892e
Protect the QueryCoord meta from stale data migrated from old version (#23412) 
Fix #23411

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-14 14:42:28 +08:00
congqixia ba84f52119
Fix watcher loop quit and channel shouldDrop logic (#23402) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-14 09:54:28 +08:00
congqixia a9a5f76f74
Fix AddDistribution unit test (#23418) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-13 20:46:29 +08:00
yah01 dc0eb433ae
Add IoPoolSize config (#23360) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-13 14:50:28 +08:00
cai.zhang 43a9e175a3
Exit component process when session key is deleted (#21658) (#22164) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-04-12 20:12:28 +08:00
zhenshan.cao dc5abe086c
Revert "Restrain gc files by collection (#22504)" (#23380) 
This reverts commit fa86de530d.

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-04-12 19:38:28 +08:00
yah01 296380d6e6
Support async refresh (#23107) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-12 15:06:28 +08:00
Jiquan Long a940df94c7
Forbid counting entities with pagination (#23375) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-12 14:22:34 +08:00
congqixia 6cd73d4e2a
Wait safe signal before release growing (#23358) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-11 17:56:31 +08:00
Jiquan Long ad7f3d4c3e
Support Count(*) on querynodev2 (#23321) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-11 15:40:31 +08:00
congqixia 0527d2706c
Fix offline set always nil (#23340) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-11 14:42:30 +08:00
wei liu dbbd703667
fix balance generate unexpected task (#23299) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-11 14:38:30 +08:00
congqixia f1f8ce01de
Remove querynode v1 package (#23289) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-11 11:28:31 +08:00
yihao.dai 0b354cbab1
Fix timetick block caused by dml task pop failed (#23277) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-11 11:00:31 +08:00
congqixia 03a4f60b6e
Fix knowhere cpp header files name (#23330) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-10 20:00:30 +08:00
XuanYang-cn b90fa5f459
Refine codes of datanode buffer (#23168) 
This PR refines deltabuffer of datanode:
- Add last sync time for compacted segment, see also: #23210
- Ensure all deltabuffermanager handles all delete related operations
- Change usedMemory to atomic.Int64
- Remove allocator in delete buffer

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-04-10 18:42:30 +08:00
Jiquan Long d85f673a95
Fix time tick lag too much (#23265) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-09 16:40:29 +08:00
cai.zhang 9288020da3
Fix bug for batch delete files on gcp of minio (#23052) (#23083) (#23090) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-04-09 16:22:30 +08:00
cai.zhang bd65ef8e95
Fix bug for type params when indexing (#23242) (#23250) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-04-09 16:18:29 +08:00
Xiaofan 680ad482b7
Check balance checker chore to 10s (#23304) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-04-09 16:14:32 +08:00
MrPresent-Han ba02e70301
fix bug when syncing distribution without schema and remove unused logic(#23085) (#23216) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-04-07 19:32:29 +08:00
wei liu 9f127dae47
enable balance channel (#23227) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-04-07 19:06:28 +08:00
congqixia 132c010271
Fix proxy panicking when QueryCoord offline (#23254) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-07 17:30:28 +08:00
Cai Yudong 55202ce7d9
Support show knowhere prometheus metrics (#23102) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2023-04-07 15:34:29 +08:00
Cai Yudong 5f4673fd16
Optimize unittest to save runtime (#23248) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-04-07 14:20:29 +08:00
congqixia 32be374c33
Fix MetaCache ut internal not changed (#23276) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-07 13:20:28 +08:00
jaime c9d0c157ec
Move some modules from internal to public package (#22572) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-04-06 19:14:32 +08:00
Jiquan Long 8530dfadb4
Fix metric RootCoordInsertChannelTimeTick (#23236) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-06 15:30:29 +08:00
Jiquan Long 2c1c3aecfc
Fix metric ProxySyncTimeTickLag (#23235) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-04-06 15:28:33 +08:00
congqixia 8db24e0a83
Support queryHook in querynodev2 (#23140) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-06 14:32:29 +08:00
aoiasd 1a485044a9
fix NumConsumers metric not work (#23167) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-04-05 08:18:28 +08:00
congqixia 424570b2b8
Remove annoy related unit tests (#23229) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-05 08:14:28 +08:00
congqixia 4a53018e5f
Remove legacy annoy_inner_error (#23220) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-04 18:08:24 +08:00
Cai Yudong ef63e64ded
Remove ANNOY index type (#23189) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-04-04 16:30:27 +08:00
congqixia 3af3fe52d8
Remove partition target check in querynodev2 (#23188) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-04 13:36:32 +08:00
Cai Yudong 9ad6d9f1a0
Rename RAFT_IVF_FLAT/RAFT_IVF_PQ to GPU_IVF_FLAT/GPU_IVF_PQ (#23194) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-04-04 11:44:26 +08:00
yah01 7da870f512
Remove useCustomConfig and simpilify the session type (#23166) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-03 20:10:24 +08:00
yihao.dai 17d3190fcc
Fix DescribeCollection failed with undefined msg type (#23192) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 19:52:24 +08:00
Enwei Jiao 4d2e379b09
Add prometheus-cpp (#23199) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-04-03 19:22:23 +08:00
smellthemoon ce91f61e5a
Pre-create topics (#21907) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-04-03 16:44:23 +08:00
congqixia 42168f67fc
Add OWNERS file for qnv2 (#23196) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-03 16:22:24 +08:00
congqixia e3e3ac1525
Use peek segment info in querynodev2 (#23183) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-04-03 16:10:24 +08:00
yah01 2a760c108c
Add metrics for QueryNodeV2 (#23163) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-04-03 15:24:24 +08:00
yihao.dai b6f69fe7f2
Check if loaded before delegator search/query (#23162) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 14:26:23 +08:00
yihao.dai 9cecdc4f4e
Fix querynode panic (#23181) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 14:18:24 +08:00
MrPresent-Han afd874b736
enhance segment balance by considering global rowCount(##22914) (#23056) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-04-03 14:16:25 +08:00
yihao.dai a0ca4d6108
Improve search/query logs in Proxy (#23144) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-04-03 10:54:23 +08:00
Andy Pan b8c1859aed
Bump ants to v2.7.2 for performance improvement (#22936) 
Signed-off-by: Andy Pan <i@andypan.me>
Signed-off-by: Andy Pan <panjf2000@gmail.com>
 2023-04-02 08:54:23 +08:00
Ten Thousand Leaves 1596abf6e8
Add lock for segment manager during bulk insert (#23111) 
/kind bug

issue: #23106

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-03-31 20:16:24 +08:00
XuanYang-cn be0827417a
Fix buffer calculate memorysize negative (#23152) 
Along with the following small patches:
- And fix logging dup segmentIDs for delete buffer status
- Add String method for PriorityQueue to debug

See also: #23105

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-31 18:30:23 +08:00
congqixia 127867b873
Add ratedgroup for some info/warning log (#23095) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-31 15:22:23 +08:00
aoiasd bd5fab1e53
Remove merge policy of proxy RoundRobin policy (#23021) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-31 14:36:22 +08:00
smellthemoon 22aeb72eba
Fix LoadSegment req don't fill in collection id (#23136) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-31 11:28:23 +08:00
Cai Yudong 7612c75c47
Let RAFT_IVF_PQ param accept m=0 (#23134) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-31 11:22:22 +08:00
yah01 75737c65ac
Refine error handle of QueryCoord (#23068) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-31 10:54:29 +08:00
zhenshan.cao 1287ca699a
Refine usage of TimeRecorder.Record (#23142) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-30 18:56:22 +08:00
yihao.dai a182594bf5
Add FlushAll support (#22942) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-30 11:28:25 +08:00
wei liu 74da53c027
fix update load percentage (#23054) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-30 10:48:23 +08:00
Jiquan Long 0f2c3ac832
Optimize log (#23121) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-30 10:44:22 +08:00
Enwei Jiao 50dab2d394
Add tracing for retrieve (#23033) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-29 18:10:02 +08:00
aoiasd 970dcb3161
Fix panic because timestamp was nil (#23096) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-29 17:20:04 +08:00
Xiaofan 653d9ecf91
Change RocksMQ page size to 64M (#23078) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-03-29 17:08:04 +08:00
yah01 73f1f15c1c
Fix DataCoord's meta may be broken with DataNode of prior version (#23030) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-29 16:58:02 +08:00
Cai Yudong 3febb5e45a
Remove gpu mode param check (#23074) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-29 14:24:01 +08:00
wei liu e2096965c7
fix leader view (#23038) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-29 14:06:02 +08:00
smellthemoon b1992e84cc
Delete dup check (#23076) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-29 10:06:01 +08:00
yah01 dc6d4b913a
Fix partitions may be not recovered with double load partitions (#23061) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-28 21:38:02 +08:00
smellthemoon 846de72fb3
fix to reduce memory (#23023) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-28 21:36:06 +08:00
yah01 826c1fb850
Support mmap on QueryNodeV2 (#23073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-28 21:30:05 +08:00
Xiaofan d0510210df
Add config for disable BF load (#22996) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-03-28 20:06:10 +08:00
XuanYang-cn 9691315c21
Fix flush deltabuf with segmentID=0 (#23059) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-28 19:14:01 +08:00
wei liu ed60b434fa
fix unstable ut (#23070) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-28 19:06:01 +08:00
aoiasd 5d172d0f4f
Refine errors of datanode (#22852) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-28 19:04:00 +08:00
zhenshan.cao e3c3c949c4
Fix datarace of flow_graph_manager_test.go in package datanode (#23043) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-28 10:42:00 +08:00
Bingyi Sun d6bfe8fbb9
Change frequent logs to debug level (#23012) 
Signed-off-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: sunby <bingyi.sun@zilliz.com>
 2023-03-28 07:12:01 +08:00
zhenshan.cao d55c860383
Align the maximum dim of the diskann index and collection (#23039) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-28 00:06:00 +08:00
yah01 ae0f467c02
Fix segment/channel may be re-loaded/subscribed (#22969) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-27 18:28:00 +08:00
Enwei Jiao ef57ba3bac
Remove easylogging and replace by GLog (#22979) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-27 18:22:02 +08:00
yihao.dai b21f03682a
Fix flowgraph manager test (#23009) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-27 18:00:05 +08:00
congqixia 4008ee6d60
Remove datanode ddNode forward delta logic (#23005) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-27 14:18:00 +08:00
yah01 081572d31c
Refactor QueryNode (#21625) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-27 00:42:00 +08:00
cai.zhang 977943463e
Check index params and type params (#22988) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-03-26 22:15:59 +08:00
Xiaofan 5a914aedf3
Make Datacoord TT to be configurable (#22956) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-03-24 22:24:04 +08:00
XuanYang-cn da58ba6a49
Fix endless appending sids (#22977) 
See also: #22976

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-24 18:00:03 +08:00
Cai Yudong 8aebc6f3b7
Remove faiss GPU index support (#22966) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-24 17:53:58 +08:00
Jiquan Long 0be96ab610
Check if all columns aligned with same num_rows (#22968) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-24 17:09:58 +08:00
smellthemoon 3dae84f065
Use merr in proxy (#22904) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-24 15:27:58 +08:00
Gao 8b3e5189e1
Add querynode plugin (#22379) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-03-24 15:21:59 +08:00
Jiquan Long ef93e24da7
Fix cpp lint (#22955) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-24 11:23:59 +08:00
congqixia 8f847884c2
Fix misc copylocks and remove exclude rule (#22962) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-24 11:17:58 +08:00
Cai Yudong 0e9a4478e3
Remove useless index mode (#22934) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-23 21:39:59 +08:00
XuanYang-cn 93bc805933
Enhance ID allocator in DataNode (#22905) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-23 19:43:57 +08:00
congqixia 5bf6e69381
Fix refresher copylocks and once not guaranteed (#22949) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-23 19:27:58 +08:00
congqixia 1b4e29e1ee
Fix copylocks for EnvSource (#22948) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-23 19:26:02 +08:00
SimFG e8f8c1b445
Check the collection num when creating the collection (#22946) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-03-23 16:47:57 +08:00
smellthemoon af458017f4
Add some locks when write (#22938) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-23 15:57:57 +08:00
congqixia 31c22e5d24
Fix NodeID rw race (#22945) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-23 11:45:57 +08:00
congqixia 980aaeceb9
Fill coordinators client target node id (#22899) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-22 10:07:57 +08:00
yihao.dai f2ffb5db26
Use buffer size at memory sync policy (#22825) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-21 21:37:56 +08:00
yah01 2b81933d13
Refine logs of DistHandler (#22879) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-21 18:11:57 +08:00
yah01 68b9cabb87
Fix GetShardLeader returns old leader (#22887) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-21 16:57:57 +08:00
Enwei Jiao fd974df09f
OTEL integration in cpp (#22175) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-21 16:31:57 +08:00
yah01 9efebb3df7
Optimize load segment, make load fields concurrent (#22856) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-21 16:27:57 +08:00
yah01 532d12c4cf
Fix the code lost with multi errors (#22877) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-21 14:07:57 +08:00
smellthemoon 4c603cd02c
Fix upsert msgid (#22839) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-21 14:06:01 +08:00
SimFG 8259ca6929
Add the user rpc counter (#22872) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-03-21 11:37:56 +08:00
yihao.dai 254cd4f15f
Fix MockQueryNode (#22859) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-20 17:25:56 +08:00
congqixia 0317e0458b
Use Set[string] instead of map[string]interface{} ServerType (#22853) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-20 16:21:57 +08:00
yihao.dai 1f718118e9
Dynamic load/release partitions (#22655) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-20 14:55:57 +08:00
zhagnlu 7c633e9b9d
Support aliyun OSS for chunk manager (#22663) (#22842) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-03-20 14:45:57 +08:00
yah01 3202eb0d9f
Optimize rounding distances, avoid promoting to double (#22846) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-20 14:09:56 +08:00
xige-16 9aa99aedbb
[Cherry-Pick] Remove arrow uasge in FieldData (#22726) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-03-20 10:41:56 +08:00
aoiasd b8b7d1f47e
CreateIndex should wait all segments which has data insert before CreateIndex finish (#22561) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-20 10:21:56 +08:00
yah01 005d178a0e
Optimize performance of insert & query & search (#22829) 
- Reduce 1x copy of inserting int8/int16 into growing segment
- Reduce 1x copy of retrieving primary keys
- Reduce 1x copy of inserting/loading/deleting/filtering primary keys
- Reduce 1x copy of reducing string results

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-20 10:19:56 +08:00
yah01 20974711a1
Optimize range search for nq > 1 (#22831) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-20 10:17:56 +08:00
yah01 65c58b3e41
Optimize performance of querying on InsertRecord (#22838) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-20 10:13:56 +08:00
MrPresent-Han 77c9e33e70
support dml channel balancer on datacoord (#22324) (#22377) (#22692) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-03-20 10:01:56 +08:00
wei liu 5f571ea915
remove debug log (#22828) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-17 18:05:56 +08:00
yah01 99f31dc37a
Fix data after travel timestamp is still visible (#22813) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-17 18:03:55 +08:00
Jiquan Long dff15c3488
Check dimension of inserted records (#22819) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-17 17:33:58 +08:00
SimFG 4a90490a67
Fix the `segment not found` error (#22772) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-03-17 17:27:56 +08:00
wei liu 6b5dfa6db2
deprecate shard cache immediately if query failed (#22779) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-17 16:25:55 +08:00
presburger 5841a3707b
suppport for raft gpu (#22790) 
Signed-off-by: Yusheng.Ma <Yusheng.Ma@zilliz.com>
 2023-03-17 11:59:54 +08:00
congqixia a085af931d
Fix metrics info panic when network error happens (#22802) (#22809) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-17 11:55:54 +08:00
yah01 ab1ac69712
Fix inserting into growing segment copies data (#22792) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-16 20:11:55 +08:00
Jiquan Long 8139106b51
Feat: count entities by expression (#22765) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-16 19:31:55 +08:00
yah01 3d8f0156c7
Refine scheduler & executor of QueryCoord (#22761) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-16 17:43:55 +08:00
wei liu bb5088e605
fix unassign from rg (#22747) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-16 14:27:55 +08:00
wei liu 69cc68caff
support first or random strategy (#22766) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-16 14:25:55 +08:00
zhenshan.cao ecde42a493
Fix proxy's pchan statistic min ts out of order (#22757) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-03-15 11:17:55 +08:00
Cai Yudong ab3cbdfc61
Partial change to prepare for GPU index type support (#22591) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-03-14 23:21:56 +08:00
SimFG b57e476089
Fix the nil point about the session (#22748) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-03-14 20:07:54 +08:00
congqixia 59989e2806
Fix rmq client unit test may panic (#22759) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-14 19:15:54 +08:00
smellthemoon 8799b06dbc
Fix the data is disappeared when upsert (#22762) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-03-14 19:11:54 +08:00
yah01 21ba8182ee
Refine task create errors (#22745) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-14 18:51:53 +08:00
huanghaoyuanhhy 024beddfe6
Make GCS OAuth token thread-safe (#22714) 
Signed-off-by: huanghaoyuan <haoyuan.huang@zilliz.com>
 2023-03-14 18:09:54 +08:00
congqixia 732986aa04
Remove fmt.Print from internal package (#22722) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-14 17:36:05 +08:00
aoiasd 6959332b08
Fix unittest start channel event but not close (#22662) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-03-14 17:05:54 +08:00
MrPresent-Han 89618b7e12
fix wrong sequence of timetick stats on proxy #21855 (#22685) 
Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
 2023-03-14 15:43:54 +08:00
groot 6f6bd98c27
Verify vector float data for bulkinsert and insert (#22728) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-03-14 14:03:58 +08:00
Jiquan Long 556c1cfc8a
Fix confused timetick log (#22733) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-14 09:55:54 +08:00
Enwei Jiao d5ef0448f2
Remove annoying logs when config file not found (#22717) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-03-13 18:20:02 +08:00
yah01 fa2c1c1f40
Fix RootCoord double updates TSO (#22715) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 18:01:53 +08:00
yah01 a4031da634
Refine string parameters, avoid coping or deref (#22708) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 17:53:53 +08:00
yah01 8f3d6e08df
Optimize the performance of matching prefix for string index (#22712) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 17:31:53 +08:00
yah01 e1cb307690
Refine RootCoord services error handle (#22689) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 14:29:53 +08:00
yah01 be33911205
Refine RBAC init error handle (#22537) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 10:59:53 +08:00
yah01 7bc3309918
Replace NULL with nullptr (#22701) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-13 10:35:52 +08:00
congqixia af6f0d5aa5
Make merr.Error(status) read old error code (#22698) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-10 22:37:53 +08:00
yah01 1a4732bb19
Use new errors to handle load failures cache (#22672) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-10 17:15:54 +08:00
yah01 c626b9000b
Output log if mmap enabled (#22684) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-10 15:51:52 +08:00
wayblink 28390dfe09
Add bulkinsert integration test (#22621) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-03-10 11:03:52 +08:00
congqixia 3ffa6d5302
Mark cache deprecated instead of removing it (#22674) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-10 10:19:53 +08:00
Jiquan Long a36fefb009
Fix cpplint (#22657) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-03-10 09:47:54 +08:00
yah01 e581fef115
Use the request's context to allocate ts (#22633) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-09 18:51:52 +08:00
yah01 e4120d1671
Upgrade proto to add error code field of Status (#22656) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-09 18:49:51 +08:00
congqixia d9908ff231
Fix TestMetaCache_ExpireShardLeaderCache unit test (#22651) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-09 18:25:52 +08:00
shaoyue 32581e6452
Support aliyun oss as object storage with ak or IAM (#22376) 
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-03-09 16:33:52 +08:00
congqixia 9e17e2660e
Fix lifetime race in indexnode (#22645) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-03-09 16:19:52 +08:00
yah01 90a5aa6265
Refine errors, re-define error codes (#22501) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-03-09 15:47:52 +08:00
bigsheeper f994052099
Reduce printing frequency of rate limit log (#22636) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-03-09 14:15:55 +08:00
wayblink aa6212d5b0
[Cherry-pick] Fix channel checkpoint issues and fix GC (#22559) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
Co-authored-by: Ten Thousand Leaves <69466447+soothing-rain@users.noreply.github.com>
 2023-03-09 14:13:52 +08:00
XuanYang-cn b3287ca5ec
Fix the misleading log messages (#22606) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-03-09 10:39:51 +08:00
wei liu 11f1f4226a
support replica observer assign node (#22604) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-03-08 18:57:51 +08:00