milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
16,317 Commits
139 Branches
115 Tags
576 MiB
Commit Graph

282 Commits (6d53a99171a01a9ab813d2a35f322dbe8076f2bc)

Author SHA1 Message Date
zhenshan.cao c15b880f0e
Remove redundant logs (#19412) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>

Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-09-26 15:40:53 +08:00
cai.zhang 9e18645363
Optimize IndexCoord scheduler loop (#19375) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-24 13:40:52 +08:00
Xiaofan 928a213e31
Fix Chunk Manager Path mis use (#19141) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>

Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-09-23 14:40:51 +08:00
cai.zhang 1c489a85b2
Add ut for IndexCoord (#19349) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-23 09:58:51 +08:00
cai.zhang 17f5e3c7fe
Drop segment indexes when drop partition (#19022) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-23 09:36:51 +08:00
xige-16 428840178c
Support diskann index for vector field (#19093) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-09-21 20:16:51 +08:00
cai.zhang fc3309b8b8
All indexes segments both write handoff (#19230) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-21 16:34:51 +08:00
SimFG d7f38a803d
Separate some proto files (#19218) 
Signed-off-by: SimFG <bang.fu@zilliz.com>

Signed-off-by: SimFG <bang.fu@zilliz.com>
 2022-09-16 16:56:49 +08:00
cai.zhang 525201decc
Mark index meta as deleted when segment is compacted (#19163) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-15 15:44:31 +08:00
cai.zhang e65ba6331e
Only store segmentID for flushed segments event (#19075) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-09 15:52:35 +08:00
cai.zhang d6b0fdc944
Fill index size and fail reason when query index job (#19026) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-06 17:19:11 +08:00
cai.zhang 11b4f4e901
Fix typo for index code (#18971) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-05 17:37:11 +08:00
cai.zhang 6d37b1db17
Fix bug for umarshal index model (#18959) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-09-01 17:18:57 +08:00
yah01 03a59ccbd4
Fix IndexCoord forget to fill DmChannel (#18914) 
Signed-off-by: yah01 <yang.cen@zilliz.com>

Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-08-31 11:28:58 +08:00
cai.zhang 34c9eae16f
Add IndexCoord ut and fix data race (#18897) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-08-31 10:23:04 +08:00
congqixia c91bb0b015
Unify rootPath in configs and ChunkManager (#18808) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-08-25 19:32:53 +08:00
cai.zhang c924f73105
Refactor for IndexCoord to support cloud (#18643) 
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>

Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
Co-authored-by: Zach41 <zongmei.zhang@zilliz.com>
 2022-08-25 15:48:54 +08:00
aoiasd 8a70f2badb
Add ShowConfigurations for all remaining session (#18594) 
Signed-off-by: aoiasd <zhicheng.yue@zilliz.com>
 2022-08-12 13:20:39 +08:00
jaime 0e95f8a690
Refine indexcoord logs (#18589) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-08-11 10:00:38 +08:00
xing.zhao d90313a1a8
Use chunkManager rather than minio.Client in datanode garbage collection (#18092) 
Signed-off-by: xingzhao <xing.zhao@zilliz.com>

Co-authored-by: xingzhao <xing.zhao@zilliz.com>
 2022-07-22 22:10:28 +08:00
cai.zhang 315df1ff15
Reduce the timer time of assigning index task scheduler (#18319) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-19 14:24:29 +08:00
cai.zhang a47a414dce
Fix bug for IndexNode pod kill test (#18177) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-13 16:16:25 +08:00
cai.zhang 90faf4a212
Fix bug for IndexCoord recycles index files (#18168) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-08 16:54:22 +08:00
cai.zhang 0c883b215b
Fix bug for IndexCoord ut (#18148) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 18:52:21 +08:00
cai.zhang 58a9a0b79a
Add license for new files (#18138) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 17:44:21 +08:00
cai.zhang c1a47c8941
Use struct channel instead of bool chan (#18140) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 16:20:22 +08:00
cai.zhang 0ad15a22c8
IndexCoord handle events correctly (#17878) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-07-07 14:44:21 +08:00
Xiaofan a9b1d71a8f
change the etcd compareAndSwap interface (#18068) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-07-06 13:54:21 +08:00
cai.zhang 01fc411566
Handle etcd compacted error (#17886) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-30 18:54:19 +08:00
cai.zhang 899a9bf86d
IndexNode still writes finished state to ETCD when task was deleted (#17761) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-28 12:08:17 +08:00
cai.zhang ef0fc37ad7
Fixed the bug that IndexCoord lost some meta information (#17818) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-27 21:52:17 +08:00
cai.zhang e8f53af749
When the task is canceled, reduce the load of node (#17735) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-23 19:22:15 +08:00
cai.zhang 1fbdafc943
IndexCoord does not assign task to IndexNode prematurely (#17717) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-23 10:40:13 +08:00
cai.zhang 141a3bbe00
Mark deleted before drop index (#17490) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-17 18:08:12 +08:00
cai.zhang ea5041aec2
Acquiring the segment reference lock on task level (#17544) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-06-15 21:38:10 +08:00
Xiaofan af994b5e1c
Fix etcd watch error (#17364) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-06-06 16:26:06 +08:00
cai.zhang bcf3b7426a
Add distributed lock for segment refgerence (#16782) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 16:36:03 +08:00
cai.zhang f1bad98f00
Fix indexcoord ut (#17282) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-31 12:00:09 +08:00
cai.zhang b37b87eb97
The GetIndexFilePaths interface judges whether the index is complete (#17052) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-17 20:41:56 +08:00
cai.zhang b0c9f25352
Sort out the log of IndexCoord (#16810) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2022-05-09 12:05:52 +08:00
cai.zhang 2be46a01c0
Reduce the number of retries and add error log (#16754) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-05-05 09:31:51 +08:00
cai.zhang 6440c0dd98
Add log for getting IndexNode's mertrics (#16747) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-29 19:07:47 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
cai.zhang 4267dc2efc
Handle error and oragnize log (#16367) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 14:33:30 +08:00
cai.zhang f9c6b3980f
Compressed log (#16354) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-04-06 10:11:36 +08:00
cai.zhang e2e345079f
Add indexMeta version for IndexCoord log (#16266) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-29 21:03:27 +08:00
Jiquan Long 3219b4e392
Skip dimension check for scalar index (#16257) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-29 15:35:27 +08:00
cai.zhang c8688a6275
Organize log for GetIndexFilePaths (#16230) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-28 21:27:27 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
godchen 78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
cai.zhang b435c422c1
Add prometheus metrics for IndexCoord (#15638) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-02-21 17:15:51 +08:00
congqixia c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-15 15:07:48 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-08 20:57:47 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Xiaofan f617c8887b
Time Tick Failed need log (#15225) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-01-14 23:55:34 +08:00
cai.zhang 3254d36609
[skip e2e]Add comment for msgIndexCoordIsUnhealthy (#15146) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-01-12 11:15:36 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
cai.zhang 0a209e2579
[skip e2e]Add comment for errIndexNodeIsNotOnService (#14986) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-01-07 18:47:32 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
cai.zhang 7b3b2ba94a
[skip e2e]Fix log message (#14984) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-01-07 14:03:43 +08:00
Jiquan Long 4f71fff388
Fix panic when close IndexCoord (#14611) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-01-04 10:19:18 +08:00
cxytz01 0437dadd75
[skip e2e] Grammar correction (#14514) 
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
 2021-12-29 17:33:08 +08:00
cxytz01 1f62d5205f
[skip e2e] Grammar correction (#14513) 
Signed-off-by: cxytz01 <xueyou.chen@zilliz.com>
 2021-12-29 17:30:57 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
cai.zhang 1dd909f3fa
[skip e2e]Reorder head files for task.go (#14488) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-29 12:04:03 +08:00
cai.zhang cac2ee717a
[skip e2e]Reorder head files for priority_queue_test.go (#14487) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-29 12:02:16 +08:00
cai.zhang 85e1acb64e
[skip e2e]Reorder head files for node_manager.go (#14486) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-29 11:45:29 +08:00
Xiaofan 1408926cd9
Remove logs without enough info (#14440) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 10:06:47 +08:00
cai.zhang 523dbd69c2
[skip e2e]Reorder head files for metrics_info_test.go (#14421) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-28 19:31:27 +08:00
cai.zhang c7ae7d93a3
[skip e2e]Reorder head files for meta_table_test.go (#14420) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-28 19:29:20 +08:00
cai.zhang d760a94636
[skip e2e]Reorder head files for indexcoord_test.go (#14419) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2021-12-28 19:26:56 +08:00
cai.zhang 4d21874aab
[skip e2e]Reorder head files for indexcoord/indexcoord.go (#14296) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-27 17:20:06 +08:00
cai.zhang 3b5b8194cb
[skip e2e]Reorder head files for indexcoord/index_coord_mock_test.go (#14297) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-27 14:26:20 +08:00
cai.zhang c4a4ef6f37
[skip e2e]Reorder head files for indexcoord/errors_test.go (#14295) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-27 14:24:22 +08:00
Jiquan Long e18dac16f3
Fix close indexcoord panic (#14237) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-27 14:16:38 +08:00
cai.zhang df82dcc783
[skip e2e]Add log for HasSameReq in indexcoord (#14233) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-25 23:14:50 +08:00
cai.zhang 74cf787c1d
[skip e2e]Format log for LoadMetaFromETCD in indexcoord (#14234) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-25 14:58:19 +08:00
cai.zhang 33afe1b576
[skip e2e]Add log for GetUnassignedTasks in indexcoord (#14232) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-25 14:56:24 +08:00
cai.zhang 0b77c4455e
[skip e2e]Add log for GetUnusedIndexFiles in indexcoord (#14223) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-25 11:42:21 +08:00
cai.zhang 8c0f96f8c1
[skip e2e]Add log for ListNode in indexcoord (#14222) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-25 11:40:29 +08:00
cai.zhang 410cf600be
[skip e2e]Add log for PeekClient in indexcoord (#14221) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-25 11:34:21 +08:00
cai.zhang 2e70326f31
[skip e2e]Format log for DeleteIndex in indexcoord (#14068) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-23 21:40:25 +08:00
cai.zhang 62b7e09163
[skip e2e]Add log for GetIndexFilePathInfo in indexcoord (#14067) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-23 21:38:34 +08:00
zhenshan.cao 3b7dad4486
Add node ctx mananger logic for IndexCoord (#13862) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-23 21:35:52 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
cai.zhang a53140bd39
[skip e2e]Add log for GetIndexStates in indexcoord (#14066) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-23 17:13:15 +08:00
JackLCL bf66721ea3
[skip e2e]Fix error log output format in meta_table.go (#14027) 
Signed-off-by: JackLCL <chenglong.li@zilliz.com>
 2021-12-23 14:25:53 +08:00
congqixia ef3b71f22b
Use indexInfo SerializedSize when estimating segment size (#13969) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-22 20:27:34 +08:00
edward.zeng 5c003a023e
[skip e2e] Refine indexcoord util comment (#13998) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-22 20:03:38 +08:00
cai.zhang f4ac7cbfd1
[skip e2e]Add log for indexcoord peek indexnode (#13855) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-21 16:08:59 +08:00
cai.zhang f64a1a5bae
[skip e2e]Add comment for PeekClientV1 (#13857) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-21 15:48:17 +08:00
cai.zhang 1eb5645762
[skip e2e]Add comment for PeekClientV0 (#13856) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-21 15:46:13 +08:00
cai.zhang 04c210f4dd
[skip e2e]Add comment for PeekClientPolicy in indexcoord (#13759) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-20 18:09:06 +08:00
cai.zhang 2da7f528e8
[skip e2e]Add comment for getDimension in indexcoord (#13761) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-20 17:47:23 +08:00
cai.zhang 05919ed346
[skip e2e]Add comment for estimate in indexcoord (#13760) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-20 17:44:06 +08:00
cai.zhang 2ef2228ad0
Check component state before get index files (#13578) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-17 23:10:42 +08:00
cai.zhang 037b9b0d4e
[skip e2e]Format log (#13579) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-17 22:17:03 +08:00
cai.zhang 461130e933
[skip e2e]Add log for BuildIndex (#13577) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-17 22:15:08 +08:00