milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
19,848 Commits
136 Branches
115 Tags
566 MiB
Commit Graph

481 Commits (0f8c6f49ff19624d6cd3c4aa788bb184baa051aa)

Author SHA1 Message Date
wei liu 69a71e11d1
enhance: Add config for auto upgrade segment index (#29112) 
This PR add config for auto upgrade segment index to index engine's
version.

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-12-12 11:56:37 +08:00
Enwei Jiao 0e65e90338
enhance: Support otlp with insecure (#29115) 
issue: https://github.com/milvus-io/milvus/issues/28914

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-12-12 11:14:37 +08:00
yah01 0a87724f18
enhance: remove merger for load segments (#29062) 
remove merger as now QueryNode could load segments concurrently
fix #29063

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-12-12 10:48:37 +08:00
cai.zhang 2b05460ef9
enhance: Make import-related error message clearer (#28978) 
issue: #28976

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-12-08 10:12:38 +08:00
wayblink 6736f65345
feat: skip some empty ttMsg in Datanode flowgraph (#28756) 
/kind feature

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-12-07 01:00:37 +08:00
shaoyue 4a067a4c8c
enhance: Add proxy.ginLogSkipPaths (#28945) 
Fix #28944

/cc @xiaofan-luan

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-12-06 10:26:35 +08:00
groot ad1daebc8e
fix: Change ranges of HNSW parameters according to Knowhere (#28875) 
fix: https://github.com/milvus-io/milvus/issues/28860

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-12-05 16:20:42 +08:00
yah01 bf633bb5d7
enhance: refine the retry error (#28573) 
return the last error but not combining all errors, to improve
readability and erorr handling

resolve: #28572

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-30 18:34:32 +08:00
shaoyue 8b2b0d412c
enhance: storeageType default value change to remote (#28792) 
/kind enhancement
/cc @PowderLi

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2023-11-30 11:34:27 +08:00
congqixia f9bb8e9648
enhance: Change const magic number in querycoord to param (#28819) 
See also #28817

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-30 09:06:28 +08:00
cqy123456 3b1b14dd78
fix: update binlog index memory uasge before loading segments (#28528) 
issue: #27678 
when interimIndex = true, memory predict should be update with the
memory usage of binlog index build process.

Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-29 16:42:27 +08:00
Gao 5bd73d5503
enhance: add efficient distance computations in Go (#28657) 
Related #28656
Add more efficient calc_distance at Go side.

---------

Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-11-28 18:20:26 +08:00
XuanYang-cn 606ec77b66
enhance: Unify levelzero segment config in DN (#28720) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-28 18:04:26 +08:00
aoiasd 89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. (#28319) 
relate: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-28 15:32:31 +08:00
cai.zhang 1b7a503f89
enhance: Revert import support csv format (#28760) 
Revert import support csv format.
issue: #28778

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-28 14:32:27 +08:00
SimFG 9c46788d87
enhance: Support to trace restful request and request error (#28685) 
issue: #28348

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-27 20:14:26 +08:00
cai.zhang c29b60e18e
enhance: Support Array DataType for bulk_insert (#28341) 
issue: #28272 
Support array DataType for bulk_insert with json, binlog files.

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-11-27 13:50:27 +08:00
Bingyi Sun 2843664a20
fix: channel name can not contains '-partition-' (#28689) 
issue: https://github.com/milvus-io/milvus/issues/28675

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-24 10:14:24 +08:00
wei liu 4b8d3eaa41
enhance: Increase heartbeat timeout (#28648) 
it's easy to trigger heartbeat timeout after 100ms when standalone cpu
usage reach 100%.
This PR increase the heartbeat timeout param to 2000ms

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-23 19:50:24 +08:00
congqixia a2fe9dad49
enhance: Make etcd kv request timeout configurable (#28661) 
See also #28660
This pr add request timeout config item for etcd kv request timeout
 Sync the default timeout value to same value for etcdKV & tikv config

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-23 19:34:23 +08:00
zhagnlu bc43c5cf34
fix: Panic if minio port is empty (#28461) 
#28460

Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-11-23 19:24:25 +08:00
Bingyi Sun 4fedff6d47
feat: integrate storage v2 into the write path (#28440) 
#28378

---------

Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-11-23 17:26:24 +08:00
yah01 c96d07682e
enhance: enhance the accuracy of memory usage (#28554) 
before this, Milvus use container/system's memory info to get the memory
usage, which could be inaccurate.

we allocates the memory by private anon mmap,
then `rss - shared` would be the accurate memory usage

resolve #28553

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-23 15:12:23 +08:00
yah01 bfccfcd0ca
enhance: refine error messages (#28424) 
- Split the simple reason and full detail
- Refine existing error messages
related: #28422

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-21 17:02:24 +08:00
XuanYang-cn e88bbaac24
feat: Add universal levelzero segment switch (#28483) 
See also: #27349

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-20 19:04:22 +08:00
PowderLi 7029797452
Improve HTTP server include metrics and RESTful API (#28045) 
/kind improvement
issue: #27653

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-19 17:14:20 +08:00
yah01 e9ff7ed13d
enhance: remove legacy error related code (#28385) 
/kind improvement

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-11-14 15:02:18 +08:00
SimFG cfb6edea61
Support to trace the grpc request (#28349) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-13 20:16:18 +08:00
XuanYang-cn f8aa46419a
Add LevelZeroCompaction configs (#28190) 
See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-11-13 11:18:19 +08:00
SimFG 7dda2e8814
Change some log level in the pkg package (#28181) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-08 23:34:22 +08:00
zhenshan.cao 2057fdce43
Change default storage type to minio (#28236) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-11-07 21:15:31 +08:00
wei liu 5b45a138b1
disable auto balance when old node exists (#28191) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-07 14:02:20 +08:00
Xiaofan da19e49daf
Support purge old session for standalone (#28184) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-11-06 21:21:42 +08:00
yah01 90e2c63d9e
Fix getting incorrect CPU num (#28146) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-11-06 06:02:16 +08:00
congqixia bf2f62c1e7
Add `WriteBuffer` to provide abstraction for delta policy (#27874) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-11-04 12:10:17 +08:00
SimFG ef68680639
Fix the unstable resource manager testcase (#28033) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-11-03 11:00:17 +08:00
aoiasd d2727cc0ff
Add write cache for access log (#27792) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-11-03 10:28:17 +08:00
wei liu 3b8fbbd91d
avoid sleep to ctx done during retry (#27992) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-11-01 18:52:24 +08:00
PowderLi 0c0f012e03
add a custom http header: Accept-Type-Allow-Int64 (#27901) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-11-01 11:42:16 +08:00
Enwei Jiao 8ae9c947ae
Use OpenDAL to access object store (#25642) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-11-01 09:00:14 +08:00
cqy123456 4fbe3c9142
replace loaded binlog with binlog index for search performance (#27673) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-11-01 02:20:15 +08:00
yah01 9658367a3c
Refine chunk manager errors (#27590) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 12:18:15 +08:00
yah01 2af46d7333
Increase the ChunkManager request timeout (#28015) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-31 09:06:13 +08:00
MrPresent-Han ccaec44930
enable parameter for dataCoord broker timeout(#26972) (#27825) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-10-28 06:14:11 +08:00
yihao.dai f9c630247d
Construct plan directly when search with vector output (#27928) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-26 19:30:10 +08:00
congqixia 8ab01e5a14
Set ElementType for empty array FieldData (#27924) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-25 20:08:10 +08:00
Gao c8415453a1
Support prepare config for index (#27920) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-10-25 19:54:12 +08:00
aoiasd 53246b1b38
Set accesslog default to close and use stdout (#27891) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-10-25 10:30:10 +08:00
wei liu 40723a292e
reduce compact parallel task num (#27899) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-25 09:40:12 +08:00
jaime 4640928280
Fix initialization and backward compatibility issue for GRPC compression (#27894) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-10-24 22:14:14 +08:00
congqixia 93a877f55e
Make qcv2 target&leader observer execute in parallel (#27844) 
- Add `taskDispatcher` to submit and run task async safely
- Change `LeaderObeserver` and `TargetObserver` schedule and manual check action to submitting task into dispatcher
- Fix logic problem in collection observer when manual check return false

See also #27494

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-24 10:14:11 +08:00
Xiaofan 2ea7579dbb
Reduce rpc size for GetRecoveryInfoV2 (#27483) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-10-23 21:44:09 +08:00
zhagnlu 6060dd7ea8
Add chunk manager request timeout (#27692) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-10-23 20:08:08 +08:00
congqixia 4a6790b63a
Make KeyLock generic (#27854) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-23 17:40:11 +08:00
SimFG 9b0ecbdca7
Support to replicate the mq message (#27240) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-10-20 14:26:09 +08:00
zhenshan.cao 020ad9a6bc
Rectify wrong exception messages associated with Array datatype (#27769) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-10-19 17:24:07 +08:00
smellthemoon 4b0ec156b3
Set channel work pool size in datanode (#27728) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-10-19 08:28:08 +08:00
yah01 9467de79c3
Use pool to block exceeded tasks (#27767) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-19 03:30:06 +08:00
yihao.dai 49b3a12804
Return newly defined merr instead of grpc unimplemented err (#27751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-10-18 15:32:11 +08:00
congqixia 2f201c25e2
Remove deprecated io/ioutil usage (#27747) 
`io/ioutil` package is deprecated, use `io`,`os` package replacement
also added golangci-lint rule to block future reference

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
Co-authored-by: guoguangwu <guoguangwu@magic-shield.com>
 2023-10-17 20:32:09 +08:00
Futao Wei 599012a340
Implement SearchByPks path for Search (#25882) 
Signed-off-by: unfode <forrest.futao.wei@gmail.com>
 2023-10-16 15:34:08 +08:00
wayblink e3f2122618
Expose metrics of stanby coordinators (#27698) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-10-16 15:04:09 +08:00
wei liu 43f2452f4b
speed up shard leader cache update frequency (#27640) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-10-16 10:52:10 +08:00
yah01 be980fbc38
Refine state check (#27541) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 21:01:35 +08:00
foxspy 2b46bd1f08
fix diskann searchCache param compatibility (#27557) 
Signed-off-by: xianliang <xianliang.li@zilliz.com>
 2023-10-11 16:57:34 +08:00
yah01 2df9908c6a
Map old error code to new error (#27616) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-11 12:43:33 +08:00
yah01 3759857bc5
Refine Proxy errors (#27499) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-09 10:09:33 +08:00
XuanYang-cn 56c94cdfa7
Add channel manager in DataNode (#27308) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-10-08 21:37:33 +08:00
yah01 41495ed266
Improve the error message for getting all indexes of collection (#27389) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-08 21:23:32 +08:00
PowderLi 8d3069b1db
update openssl to 3.1.2 (#27399) 
deal with root path's normalization

Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-10-08 19:17:31 +08:00
congqixia 5d558623fe
Add revive sub-lints and fix existing problems (#27495) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-07 20:53:38 +08:00
congqixia 80eb5434d3
Fix var-naming caused by old PRs (#27501) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-10-07 14:43:32 +08:00
yah01 8394b3a1ec
Block creating new error from status reason (#27426) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-10-07 11:29:32 +08:00
yah01 63ac43a3b8
Refine errors for import (#27379) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-30 10:31:28 +08:00
yihao.dai 106c17f304
Make read ahead policy in ChunkCache configurable (#27291) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-28 15:47:27 +08:00
congqixia 258e1ccd66
Refine querynode scheduler lifetime (#26915) 
This PR refines scheduler lifetime control:
- Move private tri-state into lifetime package
- Make scheduler block incoming "Add" task
- Make scheduler Stop wait until all previously accepted task done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-28 10:21:26 +08:00
MrPresent-Han 505e8509b7
refine the default values for segcore threadpool size (#27406) 
/kind improvement

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-27 20:43:28 +08:00
wei liu 4071132f6a
reload loading collection when qc recover (#27300) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-27 11:43:28 +08:00
SimFG 9fb4c27a90
Fix rootcoord failure to start when reaching the limit of role num (#27361) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-27 11:03:26 +08:00
congqixia a3dd2756cf
Add predicates for TxnKV operations (#27365) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-27 10:21:26 +08:00
yah01 6539a5ae2c
Refine DataCoord status (#27262) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-26 17:15:27 +08:00
cai.zhang 30a6c6072e
Fix bug for loading array data panic (#27343) 
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
 2023-09-26 12:23:26 +08:00
wayblink 7dd0be1b2c
Enable bulkinsert binlog data with partitionkey (#27241) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-09-26 10:25:25 +08:00
wei liu 27046e242b
add timeout for graceful stop (#27326) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-26 10:15:25 +08:00
foxspy 5db4a0489e
dynamic index version control (#27335) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-25 21:39:27 +08:00
foxspy 370b6fde58
milvus support multi index engine (#27178) 
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-09-22 09:59:26 +08:00
congqixia 67a471e09c
Fix gofumpt issue in scann_checker_test.go (#27305) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-22 00:39:26 +08:00
Gao ab2d8dd0a7
Add ScaNN index param checker (#27268) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-09-21 14:51:25 +08:00
SimFG 26f06dd732
Format the code (#27275) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-21 09:45:27 +08:00
yihao.dai fe01d54eca
Set kafka read timeout to 10s and make it configurable (#27238) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-20 22:01:27 +08:00
smellthemoon d1f6825b86
Specify componet ip (#27161) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-09-20 11:49:25 +08:00
yah01 9baff1b81c
Rename error to match the nameing rules (#27214) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-20 10:59:24 +08:00
Xiaofan 6635398a6d
Fix Bin log concurrency by adding a pool (#27189) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-09-19 18:05:22 +08:00
cai.zhang a362bb1457
Support array datatype (#26369) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-09-19 14:23:23 +08:00
XuanYang-cn 10116b85ac
Release before remove when releasing flowgraphs (#27191) 
GetAndRemove removes the fg from manager immediately,
while the flowgraph is still releasing. This PR will remove
the fg from flowgraphManager AFTER flowgraphs released.

- Add Remove for ConcurrentMap
- Move collections() into flowgraph manager

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-19 11:53:22 +08:00
yah01 b9ab18d692
Replace error code with merr for RateLimiter (#27176) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-19 10:37:23 +08:00
congqixia cc9974979f
Add staticcheck linter and fix existing problems (#27174) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-19 10:05:22 +08:00
PowderLi 4feb3fa7c6
support azure (#26398) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-09-19 10:01:23 +08:00
yihao.dai 4b2802033d
Fix datanode panic due to concurrent compaction and delete processing (#27167) 
Co-authored-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-09-18 19:35:29 +08:00
Enwei Jiao 8e17bc3013
Use merr to handle segcore errors (#27137) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-18 14:07:22 +08:00
MrPresent-Han 7939f0e7d5
enable ctx traceId for assignsegment on dc(#26972) (#27108) (#27030) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-09-18 11:39:20 +08:00
xige-16 488b423e1b
Fix high cpu usage caused by proto.size (#27054) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-09-17 20:55:21 +08:00
yah01 0f2883ef9d
Optimize the generic zero value (#27102) 
allocate by declare,
this make it possible to allocate the value in stack,
faster and less presure for GC

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-14 19:05:20 +08:00
congqixia f0d0651989
Do not reset connection immediately if grpc code is `Canceled` or `DeadlineExceeded` (#27014) 
We found lots of connection reset & canceled due to recent retry change
Current implementation resets connection no matter what the error code is
To sync behavior to previous retry, skip reset connection only if cancel error happens too much.

Also adds a config item for minResetInterval for grpc reset connection

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-13 15:01:18 +08:00
chyezh 791e6ef6c6
[Improvement] add pulsar metrics and fix timeout (#26907) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-09-13 12:03:19 +08:00
yah01 00c65fa0d7
Refine QueryNode errors (#27013) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-12 16:07:18 +08:00
yah01 45d9fb5929
Fix wrong error codes & names (#26904) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-12 14:41:18 +08:00
congqixia 9e0977622d
Remove config content output before credential mask implemented (#26993) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-11 19:33:17 +08:00
congqixia 44f308fff5
Handle nil status in `merr.OK` (#26940) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-08 18:43:16 +08:00
Xu Tong 9166011c4a
Add float16 vector (#25852) 
Signed-off-by: Writer-X <1256866856@qq.com>
 2023-09-08 10:03:16 +08:00
XuanYang-cn 7f1ae35e72
Add timeout in dispatcher, AsConsumer and Seek (#26686) 
See also: #25309

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-08 09:51:17 +08:00
congqixia af5c01082b
Refine delegator lifetime control (#26881) 
- Add SafeChan interface in lifetime package
- Embed SafeChan into  interface
- Replace private lifetime struct in delegator package with
- Refine delegator on-going task lifetime control and wait all accepted task done
- Fix potential goroutine leakage from  if delegator closed concurrently

/kind improvement

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-07 10:11:15 +08:00
yiwangdr 337edc321b
tikv integration (#26246) 
Signed-off-by: yiwangdr <yiwangdr@gmail.com>
 2023-09-07 07:25:14 +08:00
SimFG 28681276e2
Improve the retry of the rpc client (#26795) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-09-06 17:43:14 +08:00
congqixia 7e2fa55440
Fix misuse of waitgroup in metrics mutex ut (#26865) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-06 12:39:14 +08:00
Enwei Jiao fb0705df1b
Decouple basetable and componentparam (#26725) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-05 10:31:48 +08:00
wei liu 1097776477
stop heartbeat if reach heartbeat limit (#26728) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-09-04 17:51:48 +08:00
XuanYang-cn b2e7cbdf4b
Remove TimeTravel in compactor (#26785) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-09-04 17:41:48 +08:00
zhagnlu 7056e9c0f7
Increase minio log level to avoid unnecessary log (#26776) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-09-04 17:25:48 +08:00
yihao.dai 4340cbfba2
Merge syncCPLagTooBehind policy into syncPeriodically policy (#26713) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-09-04 14:37:47 +08:00
yah01 3349db4aa7
Refine errors to remove changes breaking design (#26521) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-09-04 09:57:09 +08:00
congqixia e8f1b1736e
Remove log.Error(err.error())-style log (#26783) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-09-01 13:09:01 +08:00
smellthemoon e56b0018e4
Remove merr mask logic to make it ok to compare resp.status with origin error (#26773) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-09-01 11:09:00 +08:00
Enwei Jiao d206ebf220
Change default config files (#26780) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-09-01 09:19:00 +08:00
Cai Yudong 8dc16b599b
Add binary metric types SUBSTRUCTURE/SUPERSTRUCTURE back (#26766) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-08-31 20:07:00 +08:00
XuanYang-cn 86e9f1818f
Change default compaction timeout to 15mins (#26757) 
See also: #26566

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-08-31 18:07:00 +08:00
Gao 7e36f819b4
Make autoindex config clean (#26732) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-08-31 10:25:01 +08:00
congqixia d343888f3c
Fix pkg codecov & static-check script (#26720) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-31 09:59:01 +08:00
jaime c603f1c244
Remove mysql metastore (#26633) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-08-29 14:36:26 +08:00
congqixia ee7aef9272
Make pulsar request timeout configurable (#26525) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-23 20:46:23 +08:00
wei liu 7af0f7d90c
avoid concurrent sub/unsub on same channel (#26454) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-23 10:10:22 +08:00
jaime b82fc8d6e7
[skip e2e] Add connection log for etcd and grpc (#26536) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-08-22 21:20:24 +08:00
SimFG d13ca54414
Change the default gracefulStopTimeout and only warn when fail to refresh policy cache (#26450) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-22 09:04:21 +08:00
cai.zhang 94846995bf
Refine error message for field not exist (#26330) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-08-21 11:52:20 +08:00
yihao.dai c3128aaef3
Accelerate server id interceptor validation (#26468) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-20 21:24:19 +08:00
wayblink c5a1b41f95
Update session ttl to 60s (#26346) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-18 22:52:20 +08:00
wei liu 79ccf06cf6
refine proxy to querynode heartbeat interval (#26426) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-18 14:26:19 +08:00
cqy123456 7e3323a518
update dimension range of diskann. (#26405) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-08-17 20:26:20 +08:00
jaime 4a388cf450
[skip e2e] Align default value with milvus.yaml file (#25619) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-08-17 20:24:19 +08:00
yihao.dai 63b86b32a6
Add server id validation interceptor (#26395) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-17 20:20:20 +08:00
wayblink ab8214fef9
Add ut to ensure hash result (#26417) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-17 18:26:21 +08:00
XuanYang-cn be30838cf9
Refine some comments (#26401) 
Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2023-08-17 10:54:18 +08:00
wei liu 4aac7a6642
refine error reason of query/search failed (#26342) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-16 19:38:18 +08:00
wei liu 74133a3996
refine retry on grpc (#26360) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-16 19:36:18 +08:00
yah01 9723787141
Calculate memory usage without page cache (#26389) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-16 17:04:17 +08:00
SimFG 19f210974f
Support to update the hook config in the runtime (#26350) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-16 16:03:33 +08:00
SimFG 8b54914287
Add go.mod file for the milvus/pkg package (#26359) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-16 14:49:32 +08:00
congqixia 4742049ecf
Fix unstable TestSingleflight unit test (#26363) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-16 10:07:33 +08:00
congqixia f8bcf60e55
Adjust some confusing Warning log to INFO (#26356) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-15 17:49:33 +08:00
cqy123456 376642b933
reset diskann min dim limit (#26344) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-08-15 16:45:34 +08:00
chyezh 4c4b2903f9
[Fixup] nats log configuration, open nats log at info level by deafult (#26168) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-08-14 14:21:32 +08:00
Gao b6fcbb0998
Support ScaNN index (#26099) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-08-11 14:21:29 +08:00
zhagnlu 411f9ac823
Upgrade minio-go and add region and virtual host config for segcore chunk manager (#26194) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-08-11 10:37:36 +08:00
xige-16 1055c90456
Add default retrieve limit (#24782) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-08-10 14:11:15 +08:00
MrPresent-Han 3421956afa
modify default value for sync max parallel to mitigate oom on dn(#26763) (#26231) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-10 10:23:15 +08:00
wei liu f33a89387f
enable config shard level cost metrics whether contains worker's cost (#26132) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-09 11:37:15 +08:00
congqixia 767955ec6b
Reduce MQ buffer length and flowgraph wait queue length to 16 (#26179) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-09 10:05:14 +08:00
yihao.dai b6effd7345
Disable deny writing when the growing segment size exceeds the watermark (#26163) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-08-08 12:43:07 +08:00
xige-16 d1d0169fa3
Delete useless config (#26173) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-08-08 10:23:08 +08:00
PowderLi a7eecb1be0
support high-level RESTFUL API, listen on the same port as grpc. (#25108) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-08-08 10:15:07 +08:00
SimFG d2649b63db
Delete the user-role mapping info when deleting the user (#25988) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-08-04 18:37:08 +08:00
wayblink 775f03ed5d
Add default value for mq.type (#26064) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-08-04 18:35:07 +08:00
congqixia fcde5b8d7f
Add pursuit mode for mqtt msgstream (#26098) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-04 12:33:06 +08:00
wei liu 518b6310a2
refine retry times on replica (#26043) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-08-03 15:55:09 +08:00
MrPresent-Han 47392b0a0f
support metrics mutex to monitor cost of locks(#26102) (#26103) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-03 15:03:06 +08:00
MrPresent-Han 5634ba777d
add new threadpool with various priority to avoid deadlock(#25781) (#26028) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-08-03 09:31:07 +08:00
congqixia 0727e017c4
Add Promethues suite test utility (#26046) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-02 10:05:05 +08:00
yah01 9c55a7f422
Add worker num as one of load resource (#26045) 
Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2023-08-01 21:47:06 +08:00
yah01 2180ef180c
Record only failed task error (#26033) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-08-01 10:11:05 +08:00
congqixia 8b5e276361
Use different cgo pool for SQ and other operations (#26021) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-08-01 09:19:05 +08:00
zhagnlu 833674c1cb
add glog configurable function and redirect aws log to segcore log (#25664) 
Signed-off-by: luzhang <luzhang@zilliz.com>
Co-authored-by: luzhang <luzhang@zilliz.com>
 2023-07-27 19:49:02 +08:00
xige-16 6f18587f35
Fix small segment compaction (#21327) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-07-26 14:49:01 +08:00
chyezh e24a8b3606
[Feature] Add benchmark and retention configuration for nmq (#25768) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-07-25 19:33:01 +08:00
wei liu 302897f866
refine look aside balance logic (#25837) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-25 18:51:01 +08:00
congqixia 0bc03ede0d
Add eventlog pkg and support grpc streaming event observation (#25812) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-25 17:23:01 +08:00
jaime 8e5d6193f9
Add a timeout config for bulkinsert request (#25789) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-07-24 14:23:00 +08:00
MrPresent-Han f4e72cb170
remove sync segmentLastExpire every time when assigning(#25271) (#25316) (#25557) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-07-24 14:11:07 +08:00
Bingyi Sun 9f31fc9a31
Add broker timeout config item. (#25855) 
Signed-off-by: sunby <sunbingyi1992@gmail.com>
 2023-07-24 14:07:00 +08:00
congqixia 3c503afe7c
Use typeutil.ConcurrentMap instead of sync.Map (#25846) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-24 10:23:01 +08:00
congqixia 37636b2f4d
Add compaction worker max parallel config item (#25834) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-24 10:17:00 +08:00
cai.zhang b15e34db21
Add contraint for compaction based indexed segments (#25709) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-07-23 21:31:00 +08:00
cqy123456 0f3350ded4
set knowhere thread pool size in index node (#25804) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-07-21 18:50:59 +08:00
congqixia 1045c88102
Support replace indexed field in QueryCoord (#25747) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-19 21:22:58 +08:00
wei liu 6534396b3d
enable config different interval for different checker (#25514) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-19 16:50:57 +08:00
Cai Yudong 9a4761dcc7
Remove binary metrics TANIMOTO/SUPERSTRUCTURE/SUBSTRUCTURE (#25708) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-07-19 16:16:58 +08:00
aoiasd 3545b1a608
Refine rocksmq (#25031) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-07-18 20:18:57 +08:00
congqixia 8d343bf75a
Make compaction rpc timeout and parallel maxium configurable (#25672) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-18 14:25:20 +08:00
Cai Yudong 73512c72fd
Remove calc_distance (#25663) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-07-18 14:23:20 +08:00
aoiasd 77a9553c3f
Add some log when node being reachable or unreachable (#25572) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-07-18 10:35:20 +08:00
yah01 948d1f1f4a
Handle errors by merr for QueryCoord (#24926) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-17 14:59:34 +08:00
yihao.dai f19621fc8c
Support refund the tokens to limiter (#25598) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-07-14 15:58:32 +08:00
wayblink 50454bec4c
Allow empty minio.rootPath (#25534) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-07-13 14:46:29 +08:00
MrPresent-Han ea3817fbbc
refinement towards sync_cp_lag_too_behind_policy to avoid submit syncTasks too frequently(#25441) (#25442) (#25500) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-07-12 20:22:29 +08:00
yah01 205a7c430a
Fix QueryNode panic while inserting to released segment (#25495) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-12 19:48:29 +08:00
Xiaofan dbf0130803
Faster garbage collect on compacted data (#25088) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-07-12 10:14:29 +08:00
groot 96c987ed62
Bulkinsert supports partition keys (#25284) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-07-11 15:18:28 +08:00
yah01 8b06941da3
Reduce the load memory usage predict factor to 2 (#25469) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-07-11 13:06:28 +08:00
smellthemoon d63323d117
Add rate limit and deny write in upsert (#25351) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-07-11 11:20:34 +08:00
Cai Yudong 5aad0ceec3
HNSW support binary vector (#25417) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-07-10 18:18:28 +08:00
Xiaofan b437195e17
Set compaction default value to be true (#25424) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-07-08 11:04:25 +08:00
wei liu 342cfcad46
decrease default value of group max nq (#25380) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-07-07 15:40:25 +08:00
congqixia efdd71c640
Make cgo pool size larger than worker pool size (#25318) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-07-05 16:56:26 +08:00
foxspy 31173727b2
growing segment index memory opt & get vector bugfix (#25272) 
Signed-off-by: xianliang <xianliang.li@zilliz.com>
 2023-07-05 00:04:25 +08:00
chyezh d7d61f529c
[Feature|Pick] enable scheduler policy and add user-task-polling policy (#24839) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-07-03 18:24:25 +08:00
Gao de6e4817a2
Support dynamic tuning config (#25152) 
Signed-off-by: chasingegg <chao.gao@zilliz.com>
 2023-07-03 15:18:24 +08:00
smellthemoon b30517d303
Enlarge timeout to prevent health check failure (#25173) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-06-29 17:10:23 +08:00
yihao.dai 37fe3393d1
Remove const to enable move semantics and improve schema util functions (#25193) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-29 11:04:22 +08:00
congqixia 1357ef7043
Merge get worker process for same nodeID (#25185) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-28 18:34:24 +08:00
wei liu 7b999b42bd
enable config policy on replica selection (#25067) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-25 19:46:44 +08:00
jaime 18df2ba6fd
[Cherry-Pick] Support Database (#24769) 
Support Database(#23742)
Fix db nonexists error for FlushAll (#24222)
Fix check collection limits fails (#24235)
backward compatibility with empty DB name (#24317)
Fix GetFlushAllState with DB (#24347)
Remove db from global meta cache after drop database (#24474)
Fix db name is empty for describe collection response (#24603)
Add RBAC for Database API (#24653)
Fix miss load the same name collection during recover stage (#24941)

RBAC supports Database validation (#23609)
Fix to list grant with db return empty (#23922)
Optimize PrivilegeAll permission check (#23972)
Add the default db value for the rbac request (#24307)

Signed-off-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: SimFG <bang.fu@zilliz.com>
Co-authored-by: longjiquan <jiquan.long@zilliz.com>
 2023-06-25 17:20:43 +08:00
chyezh ccf3f0066f
[Pick] Enable max result window limit (#24986) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-06-25 14:42:43 +08:00
xige-16 04082b3de2
Migrate the ability to upload and download binlog to cpp (#22984) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-25 14:38:44 +08:00
congqixia a3630e17a0
Refine grpc Status handling: retry legacy only for Unimplemented (#25041) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-25 10:30:43 +08:00
PowderLi 3f4356df10
fix the spelling of `field` (#25008) 
Signed-off-by: PowderLi <min.li@zilliz.com>
 2023-06-21 14:00:42 +08:00
yihao.dai 9f5756f16e
Add cluster validation interceptor to resolve the ip-reuse issue (#25001) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-21 11:36:46 +08:00
Xiaofan af1d84e5e1
Support sync when cp is lagged (#24989) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-06-20 10:00:41 +08:00
cai.zhang c9e456c6eb
Remove metric_type check and fix some minor bugs (#24921) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-06-19 09:54:41 +08:00
cai.zhang c81bdda55a
Hide autoindex params details (#24851) (#24935) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
Co-authored-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-06-16 14:30:40 +08:00
cqy123456 a519213316
Update knowhere version, update diskann api and generate cache nodes in build process (#24898) 
Signed-off-by: cqy123456 <qianya.cheng@zilliz.com>
 2023-06-16 14:20:39 +08:00
yihao.dai c73219a54d
Limit the number of concurrent sync tasks and allow only one sync task for the same segment (#24881) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-16 14:14:39 +08:00
MrPresent-Han ce5cb3c0c5
[skip e2e] modify default used balancer for 2.3 (#24937) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-06-16 14:00:45 +08:00
smellthemoon db31e88a73
Add length check when insert and upsert (#24759) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-06-15 10:24:38 +08:00
wayblink bfae6b49af
Remove datanode timetick mq, use rpc to report instead (#23156) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-06-14 14:16:38 +08:00
wei liu a3437e0ab5
refactor replica selection and retry policy on channel (#24367) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-06-13 10:20:37 +08:00
congqixia 41af0a98fa
Use go-api/v2 for milvus-proto (#24770) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-09 01:28:37 +08:00
yihao.dai 2086b4f75d
Add RBAC for FlushAll (#24751) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-06-08 20:28:35 +08:00
yah01 ebd0279d3f
Check error by Error() and NoError() for better report message (#24736) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-06-08 15:36:36 +08:00
Enwei Jiao d3af451d92
Upgrade golangci-lint (#24707) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-06-07 19:34:36 +08:00
chyezh 1593278f9d
add independent nq limit option (#24632) 
Signed-off-by: chyezh <ye.zhen@zilliz.com>
 2023-06-07 10:38:36 +08:00
chyezh f97127ae55
add nats mq wrappers (#24445) 
bug fixup, configurable natsmq, add unittest, pass e2e.



move natsmq to pkg project

Signed-off-by: chyezh <ye.zhen@zilliz.com>
Co-authored-by: yiwangdr <yiwangdr@gmail.com>
 2023-06-07 10:00:37 +08:00
xige-16 732fe54775
Support partition Key (#24047) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-06-06 10:24:34 +08:00
congqixia 31880ab427
Use conc.PoolOption instead of ants.Option (#24585) 
- Add conc.PoolOption to setup conc.Pool
- Change panic default behavior
- Make future has error when job panicks

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-06-01 15:52:31 +08:00
xige-16 af42d7c2af
Fix crash when insert duplicate field datas (#24559) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2023-05-31 20:32:31 +08:00
smellthemoon bb62cf4e60
Support upsert api rbac (#24550) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2023-05-31 19:01:09 +08:00
wei liu 71f2012488
enable collection rate limit when only global limit set (#24452) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-31 16:47:29 +08:00
zhenshan.cao f7924724b0
Add consistency_level paramter in search/query request (#24499) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2023-05-30 21:01:29 +08:00
Jiquan Long 29ae1229b6
Support AutoIndex (#24387) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-29 20:35:28 +08:00
wei liu 1deac47069
reduce grpc client dial timout (#24427) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-29 10:03:28 +08:00
cai.zhang e206265d01
Fill is_dynamic after requery (#24345) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-24 11:09:26 +08:00
wei liu 8965ea2a08
refine err msg about no available node in replica (#24256) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-05-22 11:59:26 +08:00
congqixia 3a66e1de65
Use suite for integration test (#24253) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-19 15:57:24 +08:00
Jiquan Long bd343550a5
Support to manage connections (#24224) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-19 12:51:23 +08:00
MrPresent-Han 7744573d3d
support parms for import maxfilesize(#24191) (#24192) 
Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
 2023-05-18 18:59:27 +08:00
Jiquan Long a98c79b6a6
Fix flat index can be created with invalid metric type (#24180) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-05-18 11:07:26 +08:00
cai.zhang 008285f849
Support dynamic schema for create collection (#24176) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-18 09:33:24 +08:00
yah01 99cc24b283
Fix compaction doesn't support JSON data (#24151) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-05-17 19:09:22 +08:00
yihao.dai 7384d83d2c
Support rate limit based on growing segments size (#24121) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-05-17 09:57:22 +08:00
congqixia 73a181d226
Fix get vector it timeout and improve some string const usage (#24141) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-05-16 17:41:22 +08:00
cai.zhang ccd685013a
Json key must be enclosed in double quotes (#24113) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-05-16 16:19:22 +08:00
yah01 3ea4a39078
Fix insert node doesn't handle JSON data while merging (#24126) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-05-16 16:09:23 +08:00