Commit Graph

1259 Commits (632d8b3743623a68dee05be0b2094507f93e456e)

Author SHA1 Message Date
aoiasd 033456ea2c
enhance: make sure stream closed (#29456)
relate: https://github.com/milvus-io/milvus/issues/28367

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 19:56:47 +08:00
aoiasd a76e3b2813
Refine delete by expression for forbid proxy dml task scheduler hang (#29340)
relate: https://github.com/milvus-io/milvus/issues/29146

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-26 19:52:48 +08:00
MrPresent-Han bd3bde82f0
fix iterator lose data for duplicted result(#29406) (#29451)
related: #29406

Signed-off-by: MrPresent-Han <chun.han@zilliz.com>
2023-12-25 23:28:47 +08:00
aoiasd 7c916b1035
fix: Remove SetFinalizer which cause proxy msgstream memory leak (#29403)
relate: https://github.com/milvus-io/milvus/issues/28367

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-25 10:16:53 +08:00
SimFG dd9c61831d
enhance: Support to get the param value in the runtime (#29297)
/kind improvement
issue: #29299

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-12-22 18:36:44 +08:00
yah01 a0e1a1eb31
feat: support enable/disable mmap for index (#29005)
support enable/disable mmap for index, the user could alter the index's
mode by `AlterIndex` method
related: https://github.com/milvus-io/milvus/issues/21866

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
Signed-off-by: yah01 <yang.cen@zilliz.com>
2023-12-21 18:07:24 +08:00
wei liu fcba1c0d9e
fix: Rename invalid parameterutil package path (#29334)
This PR renames the invalid parameterutil package path

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-12-21 11:40:42 +08:00
yihao.dai f457b9f7c9
fix: Return time tick delay error and refine quota error messages (#29289)
This pr:
1. Handles the time tick delay error when converting old error codes to
milvus errors.
2. Enhances quota error messages by eliminating "force deny" and
substituting it with "quota exceeded."

issue: https://github.com/milvus-io/milvus/issues/29288

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-12-20 22:52:44 +08:00
cai.zhang 1b4d2674b3
fix: Set the default index name to the name of the existing index (#29275)
issue: #29269

Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-20 17:22:41 +08:00
congqixia bcf8f27aa7
enhance: refine proxy meta cache partition logic (#29315)
See also #29113

- Unify partition info refresh logic
- Prevent parse partition names for each partition key search request

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-20 10:02:43 +08:00
PowderLi bcd6865b29
enhance: add 3 builtin roles (#28961)
issue: #28960 [milvus-proto
#212](https://github.com/milvus-io/milvus-proto/issues/212)

add new configuration: builtinRoles
user can define roles in config file: `milvus.yaml`

there is an example:
1. db_ro, only have read privileges, include load
2. db_rw, read and write privileges, include create/drop/rename
collection
3. db_admin, not only read and write privileges, but also user
administration

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-18 14:28:41 +08:00
cai.zhang 2f7252b44e
enhance: Set default index name as field name (#29218)
Signed-off-by: Cai Zhang <cai.zhang@zilliz.com>
2023-12-15 12:06:39 +08:00
congqixia 8a63e53421
enhance: Add http method to control datacoord garbage collection (#29052)
See also #29051

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-14 19:26:39 +08:00
congqixia 264505b706
enhance: remove reset seed for every shuffle (#29195)
See also #29113
rand.Seed is deprecated and cost noticable CPU time during heavy payload

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-14 15:46:38 +08:00
congqixia 9407be6eb2
enhance: Add a config item for partition name as regexp feature and disable it by default (#29154)
See also #29177
Add a config item for partition name as regexp feature and disable it by
default

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-14 10:04:38 +08:00
aoiasd b5ee563914
fix: accesslog can not print search expression (#28899)
relate: https://github.com/milvus-io/milvus/issues/28893

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-13 18:50:42 +08:00
congqixia cb75e73c77
enhance: Use zap.Stringer for large log field (#29143)
See also #29113
Using zap.Stringer log field will evaluate log field value only when log
level meets the configuration, which could save some CPU time in search
route

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-12 22:52:39 +08:00
congqixia d0bac9d0bb
enhance: Avoid initializing casbin enforcer for each request (#29117)
See also #29113

This patch:
- Replace plain Enforcer with `casbin.SyncedEnforcer`
- Add implementation of persist.Adapter with `MetaCacheCasbinAdapter`
- Invoke enforcer.LoadPolicy when policy updated

---------

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-12-12 10:36:43 +08:00
jaime bb82074937
fix: Fix missing target db name for RenameCollection (#28909)
issue: #28908

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-12-05 17:12:37 +08:00
aoiasd 3cc4209d26
enhance: pack proxy connection code and support accesslog print SDK_Version (#28835)
relate: https://github.com/milvus-io/milvus/issues/28086
https://github.com/milvus-io/milvus/issues/28940

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-05 16:50:47 +08:00
PowderLi 20fc90c591
enhance: find collection schema from cache (#28782)
issue: #28781 #28329

1. There is no need to call `DescribeCollection`, if the collection's
schema is found in the globalMetaCache
2. did `GetProperties` to check the access to Azure Blob Service while
construct the ChunkManager

Signed-off-by: PowderLi <min.li@zilliz.com>
2023-12-03 19:22:33 +08:00
aoiasd 24c565e37b
fix: accesslog method status not return failed when error in response (#28824)
relate: https://github.com/milvus-io/milvus/issues/28086

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-12-01 15:30:38 +08:00
yah01 bf633bb5d7
enhance: refine the retry error (#28573)
return the last error but not combining all errors, to improve
readability and erorr handling

resolve: #28572

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-30 18:34:32 +08:00
yihao.dai 4b8bc2798e
enhance: Print nq (#28507)
Log nq in search path.

---------

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-30 10:42:27 +08:00
aoiasd 89d8ce2f73
enhance: refine access log to support format access log by yaml and print name info. (#28319)
relate: https://github.com/milvus-io/milvus/issues/28086

---------

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-28 15:32:31 +08:00
SimFG 9c46788d87
enhance: Support to trace restful request and request error (#28685)
issue: #28348

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-27 20:14:26 +08:00
jaime b1e0a27f31
enhance: Add logs for each step during service initialization (#28624)
/kind improvement

Signed-off-by: jaime <yun.zhang@zilliz.com>
2023-11-27 16:30:26 +08:00
SimFG 089e58dfbb
fix: Fix the unstable unit test `TestReplicateManager` (#28718)
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-24 17:50:30 +08:00
yah01 3ea0129eb3
enhance: improve the error messages and logs (#28684)
- better name for log fields
- make the error and log consistent

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-24 15:08:24 +08:00
smellthemoon 79c0edb1d8
enhance:Remove msgbase unnecessary assignments (#28511)
remove some unnecessary assignments, for the reason that
commonpbutil.NewMsgBase has default value.

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-24 15:02:39 +08:00
SimFG de13865769
enhance: Add load/release partitions to replicate msg stream (#28399)
/kind improvement
issue: #25655

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-23 15:38:24 +08:00
smellthemoon 73f2bab454
enhance:add some log when create client and get component states (#28160)
/kind improvement

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
2023-11-22 09:12:22 +08:00
yah01 d73dac52c7
enhance: validate JSON data while inserting (#28602)
some SDKs doesn't check the JSON data validation,
add this in server.

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-21 23:30:22 +08:00
congqixia e47fe4e4d1
enhance: Reduce ListImportTasks log content (#28605)
In proxy `ListImportTasks` may print all task information from
rootcoord, this may cause log content too large to process
This PR make this log print taskID and importState only

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
2023-11-21 20:58:23 +08:00
yah01 bfccfcd0ca
enhance: refine error messages (#28424)
- Split the simple reason and full detail
- Refine existing error messages
related: #28422

---------

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-21 17:02:24 +08:00
SimFG 899a5a32cd
Hide the password info when failing to authorize (#28428)
/kind improvement

Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-15 14:40:26 +08:00
XuanYang-cn 40d5c902b6
Enable getting multiple segments in plan result (#28350)
Compaction plan result contained one segment for one plan. For l0
compaction would write to multiple segments, this PR expand the segments
number in plan results and refactor some names for readibility.

- Name refactory: - CompactionStateResult -> CompactionPlanResult -
CompactionResult -> CompactionSegment

See also: #27606

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
2023-11-14 15:56:19 +08:00
yah01 e9ff7ed13d
enhance: remove legacy error related code (#28385)
/kind improvement

Signed-off-by: yah01 <yah2er0ne@outlook.com>
2023-11-14 15:02:18 +08:00
SimFG cfb6edea61
Support to trace the grpc request (#28349)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-13 20:16:18 +08:00
SimFG ef68680639
Fix the unstable resource manager testcase (#28033)
Signed-off-by: SimFG <bang.fu@zilliz.com>
2023-11-03 11:00:17 +08:00
aoiasd d2727cc0ff
Add write cache for access log (#27792)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-03 10:28:17 +08:00
yihao.dai 7bd44bd671
Fix unstable search task ut (#28101)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-11-02 12:52:15 +08:00
PowderLi 0252871d30
fix azure ListObjects (#27931)
Signed-off-by: PowderLi <min.li@zilliz.com>
2023-11-01 11:34:14 +08:00
aoiasd 2aa330146a
Delete by complex expression use partitionKey optimization (#27952)
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
2023-11-01 11:26:16 +08:00
wei liu 0677d2623d
remove retry in getShards (#28011)
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
2023-10-31 16:16:14 +08:00
Ikko Eltociear Ashimine ed4f20b0ed
Fix typo in util.go (#27975)
suppot -> support

Signed-off-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2023-10-30 14:40:27 +08:00
Jiquan Long 6913283716
Fill output fields for count request (#27967)
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
2023-10-30 10:00:12 +08:00
Filip Haltmayer 6b1a106a31
Moving etcd client into session (#27069)
Signed-off-by: Filip Haltmayer <filip.haltmayer@zilliz.com>
2023-10-27 07:36:12 +08:00
yihao.dai f9c630247d
Construct plan directly when search with vector output (#27928)
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
2023-10-26 19:30:10 +08:00
shaoyue 977e508a0f
Propagate task ctx done err for proxy (#27892)
Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
2023-10-26 19:06:13 +08:00