milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
17,490 Commits
146 Branches
120 Tags
570 MiB
Commit Graph

1408 Commits (b9189b9f413a95ff9a55889f7a6224ee3961baba)

Author SHA1 Message Date
SimFG 7e19a924b7
Add the rbac for the GetLoadingProgress and GetLoadState api (#22320) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-22 20:37:45 +08:00
Cai Yudong d4e0b6e91b
Remove unused index param check (#22271) 
Signed-off-by: Yudong Cai <yudong.cai@zilliz.com>
 2023-02-21 17:36:26 +08:00
wayblink b7c0d12d82
Skip file with prefix '.' may exist in filesystem (#22138) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-21 10:54:29 +08:00
bigsheeper f4e7b246f2
Fix msgdispatcher concurrency issue and add prometheus metrics (#22174) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-16 14:58:34 +08:00
congqixia f2575e5fa8
Add unconvert & durationcheck linters and fix issues (#22161) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-15 17:22:34 +08:00
aoiasd 148a024e05
Add tickle for datacoord watch event (#21193) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2023-02-15 16:20:34 +08:00
wayblink 77a27c6dfc
Support print log to file and console at the same time (#21946) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-14 16:56:34 +08:00
SimFG c2a49d5a0b
Add error log and metrics for the hook and privilege interceptor (#22111) (#22137) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2023-02-13 16:50:33 +08:00
bigsheeper d2667064bb
Add msgDispatcher to support sharing msgs for different vChannel (#21917) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2023-02-13 16:38:33 +08:00
jaime a2435cfc4f
[Cherry-Pick]Add a segment metrics for count number of binlog files (#22103) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-13 10:24:33 +08:00
wei liu d078441522
support rg api rbac (#22097) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-10 10:54:33 +08:00
Xiaofan d3f8948c17
Fix embedded milvus (#22077) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-02-09 22:06:32 +08:00
wei liu d085abbd56
fix load collection with rg (#22083) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-02-09 16:24:31 +08:00
yah01 b1f31da77a
Fix activate standby server ignores all errors (#22073) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2023-02-09 15:24:31 +08:00
Enwei Jiao 44154a828c
Forbid updating some configs (#22047) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-09 14:28:31 +08:00
groot c21e11eb8a
Fix datanode crash bug for bulkinsert (#22041) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-02-08 14:06:31 +08:00
Jiquan Long a548ce1cf1
Change default max shards num to 64 (#22016) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-02-07 14:43:57 +08:00
jaime baa035885d
Modify default value of some configurations (#22013) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-07 10:05:56 +08:00
congqixia 73b6c13245
Expand singleflight to Get and make writeEvent sync (#21993) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-02-06 15:27:54 +08:00
cai.zhang ae305a54bd
Set scheduler duration of index task can be configurable (#21944) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-02-05 17:21:53 +08:00
jaime 748a935a11
Modify segment seal policy with number of statslog files (#21957) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-02-03 17:07:52 +08:00
wayblink d41cc0b21b
Revoke session to only delete session key created by this node (#21935) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-02-02 16:37:52 +08:00
Enwei Jiao 53ae40b8c6
Make ratelimiter's config refreshable (#21757) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-02-01 16:03:51 +08:00
congqixia ec282b1958
Apply lifetime control for indexnode (#21892) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 18:37:50 +08:00
cai.zhang a04238c25f
Keep the configuration of indexCoord (#21879) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-31 12:07:50 +08:00
congqixia 8af2aebb66
Fix msg channel prefix config priority for backward compatibility (#21871) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-31 10:09:51 +08:00
jaime 5ed6ae2f27
Remove repeated etcd endpoint configuration (#21860) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-30 12:07:48 +08:00
wei liu 73c44d4b29
resource group impl (#21609) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-30 10:19:48 +08:00
congqixia 66027790a2
Implement detailed lifetime control for querynode (#21851) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-29 17:45:49 +08:00
Xiaofan 949d5d078f
Fix memory calculation in dataCodec (#21800) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-28 11:09:52 +08:00
Xiaofan 9258f7059d
Fix Bulkload ut print too much result (#21821) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-19 21:31:44 +08:00
congqixia 54ec22e879
Allow empty AK/SK for minio cfg (#21815) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 15:49:44 +08:00
congqixia 5986106037
Make paramtable init only once (#21782) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-19 14:53:44 +08:00
jaime 1dcd9eeb79
Support rename collection (#21693) 
Signed-off-by: jaime <yun.zhang@zilliz.com>
 2023-01-19 14:13:43 +08:00
Enwei Jiao cecd59d940
Disable time-travel by default (#21775) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-18 13:57:43 +08:00
Xiaofan 3e21d25de5
Add slow gc warning log (#21734) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-18 11:41:43 +08:00
wayblink de584b508e
Fix active-standby switch fail bug (#21755) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-17 11:43:43 +08:00
groot c0e7a122d4
Fix bulkinsert progress bug (#21680) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-13 16:25:40 +08:00
Enwei Jiao 90d9e165d4
Fix some configs not shown (#21653) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-13 15:31:41 +08:00
Xiaofan f8e1566b24
Support zstd compression in grpc (#21689) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-13 10:55:41 +08:00
wayblink 6a722396bd
Integration test framework (#21283) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2023-01-12 19:49:40 +08:00
Enwei Jiao fb42466c65
Use opentelemetry (#21509) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-12 16:09:39 +08:00
Jiquan Long 9fd9bed2b9
Remove collection meta after GC finished (#21595) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-12 09:55:42 +08:00
Jiquan Long 6d09bbed68
[skip e2e] Fix load meta migration (#21584) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-11 19:31:39 +08:00
groot 9f8095996b
Report bulkinsert progress (#21612) (#21638) 
Signed-off-by: groot <yihua.mo@zilliz.com>
 2023-01-11 17:37:44 +08:00
cai.zhang e127cf7b99
Reset indexpb for upgrade (#21620) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-11 14:35:40 +08:00
congqixia c550427e81
Use singleflight in localCache.load (#21606) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2023-01-10 16:21:39 +08:00
groot aab20a4031
Streaming read numpy (#21579) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2023-01-10 14:53:38 +08:00
Xiaofan dc2bfb0901
Add stack trace for core dump (#21555) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-06 16:53:35 +08:00
Xiaofan 4b4944ecee
Support sync all segments while close (#21421) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2023-01-06 14:49:36 +08:00
cai.zhang aa203acfb3
Low IndexCoord weight (#21548) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-06 14:21:37 +08:00
Ten Thousand Leaves b3da554705
Increase bulk insert file size limit (#21526) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-06 13:09:36 +08:00
Enwei Jiao 24d5faf7aa
Fix show configuration bug (#21544) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2023-01-06 09:55:36 +08:00
cai.zhang e5f408dceb
Merge IndexCoord and DataCoord (#21267) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2023-01-04 19:37:36 +08:00
Jiquan Long ff2a68e65a
Fix collection not exist when tried to do recovery (#21471) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2023-01-04 16:37:35 +08:00
Ten Thousand Leaves 3f6d139ad2
Fix bad import task expiration default (#21235) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2023-01-04 14:11:33 +08:00
wei liu e735485bb8
fix empty auth config unmarshal (#21473) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2023-01-03 10:11:34 +08:00
congqixia 8b4d60b073
Add guarantee and serviceable lag too large check (#21441) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-29 15:17:30 +08:00
SimFG 2aed48c433
Fix query node panic when watching dm channels (#21402) 
Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-28 10:19:31 +08:00
smellthemoon cded2ce77c
Fix offset panic (#21292) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-27 11:11:29 +08:00
Enwei Jiao 264f08953d
Refactor Rocksmq's config (#21304) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-26 19:11:30 +08:00
jaime d401608899
Add a segment seal policy by number of binlog files (#21263) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-20 14:09:25 +08:00
Enwei Jiao 166e9f0da5
Refactor GrpcConfig (#21142) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-16 15:59:23 +08:00
SimFG 4df91ddf60
Support the graceful stop for the indexnode (#21021) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-16 10:03:27 +08:00
jaime 0954014c6d
[skip e2e] Fix broken default parameters for seal segment policy (#21207) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-15 18:31:23 +08:00
Enwei Jiao e6374901f9
Make TopK Limit configurable (#21151) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-15 14:03:22 +08:00
aoiasd 0011b65338
Fix unit test of local cache cause data race (#21237) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-15 13:41:22 +08:00
Ten Thousand Leaves 9d8f422cb8
Reduce maxFieldNum from 256 to 64 (#21104) 
issue: #21077

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-14 10:39:22 +08:00
SimFG f31d5facff
Fix the unsafe casbin `Model` (#21129) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-14 10:29:22 +08:00
XuanYang-cn b850c4ae2c
Refine search logs in querynode (#21134) 
- Make sure all logs logging with ctx
- Add an info log for searching on not-indexed segments
- Separate log prefix of shardleader and non-shardleaders

See also: #20975

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-12-12 14:33:22 +08:00
Xiaofan e977e014a9
Fix flush didn't respect binaryvector and other schemas (#21120) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-12 10:33:26 +08:00
SimFG 4a5c282b62
Add the superuser config (#21090) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-09 16:13:20 +08:00
Ten Thousand Leaves d67db2ed9b
Make compaction target able to exceed segment max size by a bit (#21068) 
issue: #21053
Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-09 16:03:20 +08:00
aoiasd de0ab9e2cf
Refactor showConfigurations to allow return global config rather than only return config of this component (#21063) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-09 14:31:21 +08:00
aoiasd 60abe08f85
Add retention for access log on minio (#20681) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-12-08 19:39:19 +08:00
yah01 9ebaa10dec
Add more logs for GetShardLeaders (#21046) 
Also increase the heartbeatAvailableInterval from 2.5s to 10s

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-08 19:09:18 +08:00
Jiquan Long 6709ccf690
Fix empty retrieve result (#21032) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-12-08 10:15:19 +08:00
Enwei Jiao 89b810a4db
Refactor all params into ParamItem (#20987) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-12-07 18:01:19 +08:00
Jiquan Long dee426a7a5
Fix group checker ut (#21038) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-12-07 16:11:22 +08:00
Ji Bin 6692c9999e
Using first unicastable address as local ip (#20674) 
Signed-off-by: Ji Bin <matrixji@live.com>

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-12-07 15:03:19 +08:00
groot 33a102c6fc
Refine bulkinsert (#20986) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-12-07 14:53:18 +08:00
Xiaofan e131915207
Support 64M Rpc Limit (#20845) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-12-07 10:07:18 +08:00
SimFG f8cff79804
Support the graceful stop for the query node (#20851) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-12-06 22:59:19 +08:00
Ten Thousand Leaves 2f5b03fef7
Split small segment proportion and small segment compactable proportion (#20996) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-06 14:37:18 +08:00
jaime 039e9ce4bb
Refine msgstream interface (#20832) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-05 20:55:17 +08:00
jaime 548e90ec68
Support an analogous Java Guava cache implementation (#20831) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-12-05 20:29:18 +08:00
yah01 aec347e591
Check last heartbeat response time while fetching shard leaders (#20968) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-05 15:09:20 +08:00
congqixia 4238b5dbab
Make datanode unittest use embed etcd server (#20948) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-12-02 16:05:16 +08:00
yah01 ec3aef1f3e
Add time records for search path (#20924) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-12-02 11:39:17 +08:00
zhenshan.cao 9724ae5e27
Refine log level (#20904) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-12-01 20:07:15 +08:00
Ten Thousand Leaves c373b44c2c
Verify and adjust segment # of rows before saving and passing SegmentInfo (#20837) 
issue: #18120

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-12-01 16:31:16 +08:00
smellthemoon f7d603f947
Fix Group Checker test fail (#20911) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-12-01 13:39:15 +08:00
Enwei Jiao 2ecdb4ba4a
Etcd config source support TLS (#20874) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-30 18:23:15 +08:00
yah01 060649b8aa
Refactor task scheduler and executor (#20828) 
Make the performance able to scale out

Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-30 13:57:15 +08:00
yah01 18762f82aa
Refine TimeRecorder output (#20897) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-30 13:33:15 +08:00
Ten Thousand Leaves 0700e56008
Increase MaxWatchDuration and make it configurable (#20884) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-29 17:19:14 +08:00
yah01 5ffbb01a67
Fix IndexNode panic while failed to create index (#20861) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-29 15:31:15 +08:00
congqixia 938c09679c
Fix LRU cache may panic on double close (#20854) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-11-29 09:47:14 +08:00
Ten Thousand Leaves 4f9a294b2e
Refine segment stat metrics on QueryNode (#20763) 
issue: #20760

/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-28 14:53:14 +08:00
smellthemoon d9185485f1
Remove TraceID in mq (#20697) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-25 23:11:12 +08:00
yah01 f76ea292d2
Change LRU to a generic type (#20744) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-22 12:11:11 +08:00
xige-16 1ad702cc40
Reduce the memory used by load disk index (#20475) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 19:35:11 +08:00
wayblink b32f99e2e4
Update datanode.segment.insertBufSize default value to 16777216 (#20713) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-11-21 19:19:11 +08:00
jaime d84cda22ad
Fix start fail with kafka model (#20726) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-21 17:49:12 +08:00
xige-16 34c88cea32
Fix GetDiskUsedSize not work (#20474) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 17:47:17 +08:00
xige-16 82570e057c
Limit MinDim to build disk index (#20724) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 17:43:10 +08:00
xige-16 81b6703201
Fix spell PQ error (#20725) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-21 16:35:11 +08:00
smellthemoon f5ab719f21
timestamp decided if the pks were the same (#20166) 
Signed-off-by: lixinguo <xinguo.li@zilliz.com>

Signed-off-by: lixinguo <xinguo.li@zilliz.com>
Co-authored-by: lixinguo <xinguo.li@zilliz.com>
 2022-11-21 10:55:10 +08:00
groot bf2107ecf5
Fix a regression of local storage (#20653) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-21 10:19:10 +08:00
jaime 0a5270e8c0
Fix alter collection hang (#20694) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-21 10:09:10 +08:00
SimFG dffcd974ff
Make sure that the segment path matches the segment id (#20677) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-11-18 15:35:09 +08:00
Xiaofan 633a749880
Recude IndexCodec Load Memory (#20621) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-18 10:47:08 +08:00
bigsheeper fef87f879c
Disable tt delay protection (#20657) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-17 20:21:14 +08:00
Enwei Jiao c05b9ad539
Add event dispatcher for config (#20393) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 18:59:09 +08:00
Enwei Jiao 19524a5344
Fix nodeID mismatch at standalone mode (#20648) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-17 17:15:08 +08:00
groot 8da44638ac
Refine json parser for import (#20545) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-16 19:05:08 +08:00
Xiaofan 58ae746f69
Fix GC Tuner Overflow (#20497) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-15 10:43:06 +08:00
Xiaofan af66a0b621
Fix Time Tick session2ChanTs map use the same nodeID (#20537) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-14 14:41:11 +08:00
groot e217b951ee
Fix bulkinsert bug for int64 value (#20491) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-14 10:15:10 +08:00
bigsheeper fc15789da9
Ensure compatibility of channel seek position and move syncPeriod to config (#20504) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-12 21:09:04 +08:00
wei liu 701e7b5a6e
support pulsar tenant and auth (#19727) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-11 10:55:05 +08:00
bigsheeper cd19d99ad7
Add channel level checkpoint (#20350) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-11-10 22:13:04 +08:00
yah01 663bdf3399
Add RootPath field back (#20488) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-11-10 19:43:03 +08:00
aoiasd fca2b71e28
add access log for proxy (#19927) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>

Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-11-10 17:09:06 +08:00
groot 53850e340c
Dont expire pending import tasks (#20431) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-10 10:25:08 +08:00
wei liu f925fa7661
fix check segment stale task (#20401) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-08 19:43:05 +08:00
Enwei Jiao 7e56e7a976
Fix datarace at Setlogger (#20376) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-08 14:33:03 +08:00
wayblink a7bed1c927
Pass backup flag through Import request options (#20334) 
Signed-off-by: wayblink <anyang.wang@zilliz.com>

Signed-off-by: wayblink <anyang.wang@zilliz.com>
 2022-11-08 11:33:03 +08:00
wei liu c5cd92d36e
update target (#19296) 
Signed-off-by: Wei Liu <wei.liu@zilliz.com>

Signed-off-by: Wei Liu <wei.liu@zilliz.com>
 2022-11-07 19:37:04 +08:00
MrPresent-Han 21b54709a2
enable delbuf auto flush function to avoid OOM when del msg accumulating (#20213) 
issue: #19713

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>

Signed-off-by: MrPresent-Han <jamesharden11122@gmail.com>
Co-authored-by: MrPresent-Han <zilliz@zillizdeMacBook-Pro-2.local>
 2022-11-07 18:49:02 +08:00
groot b847c425e1
Fix listImportTasks bug (#20301) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-07 17:11:02 +08:00
Xiaofan 0d18336b21
Fix GCTuner calculation overflow (#20354) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-07 12:17:05 +08:00
jaime 709a0a94e9
Add some msgstream metrics (#20296) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-11-07 10:15:02 +08:00
xige-16 4a66965df4
Delete RAW_DATA copy when load IVF_FLAT index data (#20274) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-11-05 17:33:05 +08:00
Xiaofan ffd52f0d08
Add GC Tuner (#20321) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-11-04 22:25:01 +08:00
Enwei Jiao 956c5e1b9d
Make Params singleton (#20088) 
Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>

Signed-off-by: Enwei Jiao <enwei.jiao@zilliz.com>
 2022-11-04 14:25:38 +08:00
zhagnlu 15b2d1cace
Change default thread core coefficient (#20120) (#20328) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-04 12:41:35 +08:00
zhagnlu cb2591d1fe
Parallel operations of disk-ann file manager (#20120) (#20122) 
Signed-off-by: zhagnlu <lu.zhang@zilliz.com>

Signed-off-by: zhagnlu <lu.zhang@zilliz.com>
Co-authored-by: zhagnlu <lu.zhang@zilliz.com>
 2022-11-03 14:39:40 +08:00
Ten Thousand Leaves 5708352a80
Improve DataCoord and DataNode logs (#20265) 
/kind improvement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-11-03 12:15:35 +08:00
groot cd3b3d3da6
Only allow one file for row-based import (#20234) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-11-02 10:23:35 +08:00
shaoyue 9fac1476f3
Add support for GCS(GoogleCloudStorage) with IAM (#20164) 
go mod tidy

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>

Signed-off-by: shaoyue.chen <shaoyue.chen@zilliz.com>
 2022-11-01 11:07:35 +08:00
bigsheeper a2d5947fe4
Change limiter burst type to float64 (#20215) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-31 20:59:34 +08:00
groot c6151ad351
Parse utf32 string of numpy file (#20176) 
Signed-off-by: groot <yihua.mo@zilliz.com>

Signed-off-by: groot <yihua.mo@zilliz.com>
 2022-10-31 17:39:34 +08:00
Xiaofan 4136009a9a
Support load segments/channels in parallel (#20036) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-31 13:55:33 +08:00
Jiquan Long 8d0cc4226c
Fix IVF_SQ nbits check (#20183) 
Signed-off-by: longjiquan <jiquan.long@zilliz.com>

Signed-off-by: longjiquan <jiquan.long@zilliz.com>
 2022-10-31 10:13:34 +08:00
bigsheeper bddd47c60f
Prevent limiter burst overflow (#20153) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-28 19:05:32 +08:00
bigsheeper b949fa0ea9
Increase default ttProtection max delay (#20125) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-27 17:37:32 +08:00
groot bee66631e3
Refine bulkload (#19671) 
Signed-off-by: yhmo <yihua.mo@zilliz.com>

Signed-off-by: yhmo <yihua.mo@zilliz.com>
 2022-10-27 16:21:34 +08:00
bigsheeper f0c616050a
Reduce and improve quota center log (#20058) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>

Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-10-25 19:39:30 +08:00
Xiaofan 1f170ce1ba
Refine kafka consumer (#19846) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-10-25 13:23:30 +08:00