milvus-io
/
milvus
mirror of https://github.com/milvus-io/milvus.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
15,845 Commits
127 Branches
115 Tags
560 MiB
Commit Graph

1185 Commits (5429a973b4b72cbcc37a3581f579a8e3b24b4120)

Author SHA1 Message Date
Xiaofan 338edcd3cc
Load binlog for different field in parallel (#17005) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-15 16:11:54 +08:00
congqixia 23f7ef931c
Remove not used QueryService (#16973) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-13 20:27:54 +08:00
congqixia ec66ac69ac
Simplify chunk manager in QueryShard (#16976) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-13 19:55:54 +08:00
Letian Jiang f0dee539bb
Improve query shard log (#16950) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-13 13:45:53 +08:00
XuanYang-cn 43b36b4e5f
Make proxy update cache with NotShardLeader status code (#16940) 
See also: #16926

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2022-05-13 11:35:53 +08:00
Jiquan Long c5149c7e85
Replace text codec of pb with binary codec (#16955) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-05-13 09:53:53 +08:00
Letian Jiang b3eb2b1d0d
Support deltaLog loading on growing segment (#16903) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-12 11:59:53 +08:00
congqixia a8b81e215b
Fix deadlock in release replica (#16925) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-11 21:57:53 +08:00
congqixia a0614a3ff8
Use NodeIds instead of NodeID in segment info (#16912) 
The NodeID field is deprecated in current design
Query Node segment detector shall read NodeIds instead of NodeID

Fix #16911

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-11 17:39:53 +08:00
congqixia 34833680da
Add lock before query release collection from replica (#16880) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-10 18:09:53 +08:00
Ten Thousand Leaves 216c45fbd6
Add tests where segments load delta logs (#16825) 
issue: #16821

/kind improvements

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-05-10 17:59:52 +08:00
congqixia c7935d1a86
Add legacySegments for load balance segments rc track (#16831) 
- Add legacySegments in ShardCluster tracking old segment inUse
- Apply node id check for segment online/inUse check
- Add unit test simulating load balance case

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-07 18:25:52 +08:00
Xiaofan 92b6293be4
Fix QueryNode log level (#16604) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-07 10:27:51 +08:00
Letian Jiang b0053b7a45
Add isShardLeader field in search/query request (#16798) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-05-06 16:55:51 +08:00
congqixia 8c004fa981
Fix replicaID is allow zero from watch channel task (#16788) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-05-05 18:11:51 +08:00
Xiaofan 337ad53613
Support config small index nprobe and nlist (#16731) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-05-05 09:49:50 +08:00
Cai Yudong 6a62ff18bf
Support easylogging config for segcore and knowhere (#16751) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-05-03 08:39:49 +08:00
bigsheeper 1a29337e6b
Print unflushed segmentIDs instead of whole SegmentInfo (#16301) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-29 19:13:47 +08:00
bigsheeper f18b3c44ee
Print msgID instead of msg (#16268) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-29 19:11:47 +08:00
congqixia 9e09434a4b
Remove not used QueryService in querynode (#16735) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-29 18:15:48 +08:00
zhenshan.cao 97757405de
Fix bug: wrong metrics name and forget to register metrics (#16714) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-29 15:35:47 +08:00
yah01 415a0e5978
Fix miss segments/DmChannel when query by nodeID (#16719) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-29 14:11:47 +08:00
Letian Jiang a38c84e7e7
Lock meta-replica during search/query (#16703) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-29 13:37:47 +08:00
xige-16 515d0369de
Support string type in segcore (#16546) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
Co-authored-by: dragondriver <jiquan.long@zilliz.com>

Co-authored-by: dragondriver <jiquan.long@zilliz.com>
 2022-04-29 13:35:49 +08:00
jaime 68b1b82faf
Remove DataKV interface (#16692) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 21:03:47 +08:00
jaime 2205b878b1
Refine log in query_shard.go (#16693) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-04-28 17:13:48 +08:00
zhenshan.cao 0a953948af
Simplify monitoring metrics (#16687) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-04-27 23:03:47 +08:00
congqixia 3a6db2faeb
Fix handling segment change logic (#16695) 
Dispatch segmentChangeInfo to ShardCluster leader
Hold segment remove before search is done

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-27 22:23:46 +08:00
Letian Jiang 2879796b42
Validate partitionIDs & segmentIDs in query request (#16684) 
* check partitions if they are released or unloaded
    * check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-27 19:23:48 +08:00
congqixia 0bf176a7e1
Add SyncReplicaSegments interface for QueryNode (#16672) 
Add SyncReplicaSegments for LoadCollection/LoadPartition child task
This API notifies related ShardCluster to record segment Loaded
Preventing the Etcd events is delayed, which will cause some segment not search

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-27 10:41:46 +08:00
Xiaofan b657b10eb5
Remove useless log (#16624) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-27 10:01:47 +08:00
Letian Jiang f75dedb317
Validate partitionIDs & segmentIDs in search request (#16639) 
* check partitions if they are released or unloaded
* check segments if their collection/partition(s) are released or unloaded

Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-26 14:19:45 +08:00
yah01 80ae6de323
Add comments for concurrency package (#16654) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-26 11:33:45 +08:00
congqixia bb69e09736
Use fmt.Errorf instead of string concat in querynode/segment.go (#16422) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-25 16:11:48 +08:00
yah01 7af02fa531
Improve load performance, load binlogs concurrently per file, deserialize binlogs concurrently per field/segment (#16514) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-04-25 15:57:47 +08:00
congqixia b99b65c26e
Add segment reference count and handles change info in ShardCluster (#16620) 
Resolves #16619
Add reference count for each search/query request
For SegmentChangeInfo
- Wait all segments in OnlineList to be loaded
- Add handoff event into pending list
- Wait all segments in OfflineList is not used (reference count = 0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-25 11:51:46 +08:00
Ten Thousand Leaves 4ef2df8cb9
Fix an issue where query coord calls MinioChunkManager during init (#16629) 
It causes Milvus not runnable with local storage.

issue: #15604

/kind enhancement

Signed-off-by: Yuchen Gao <yuchen.gao@zilliz.com>
 2022-04-25 11:11:46 +08:00
Xiaofan 89b4a34892
Fix Data race in NodeID (#16603) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-04-24 22:03:44 +08:00
congqixia bd29518b6e
Remove debug log with no meaning (#16583) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-22 13:11:42 +08:00
congqixia 6de1d223a6
Remove debug log in querynode segment/node detector (#16572) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-22 11:07:42 +08:00
congqixia 69252f812d
Implement memory replica in Proxy, QueryNode and QueryCoord (#16470) 
Related to #16298 #16291 #16154
Co-authored-by: sunby <bingyi.sun@zilliz.com>
Co-authored-by: yangxuan <xuan.yang@zilliz.com>
Co-authored-by: yah01 <yang.cen@zilliz.com>
Co-authored-by: Letian Jiang <letian.jiang@zilliz.com>

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-20 16:15:41 +08:00
codeman 4e12271d79
Support login with username and password (#15656) (#16341) 
Signed-off-by: kejiang <ke.jiang@zilliz.com>

Co-authored-by: kejiang <ke.jiang@zilliz.com>
 2022-04-11 19:49:34 +08:00
Cai Yudong 71cd7ba67a
Add configuration common.indexSliceSize (#16438) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-04-08 20:29:33 +08:00
xige-16 27b4cbc098
Cherry pick remove translateHits commit to mater (#16436) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>

Co-authored-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-08 20:27:31 +08:00
godchen bb7a0766fe
Add dependency factory (#16204) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-04-07 22:05:32 +08:00
congqixia 4cb7c28bfa
Fix ReplicaInfo package name (#16400) 
Fix #16399
PR 16373 breaks the build after ReplicaInfo moved from querypb to milvuspb
Also fix  problem in Makefile
Limit action/setup-go version to ~1.17.1(>=1.17.1 <1.18.0)

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-06 19:47:31 +08:00
congqixia 18d76261a0
Add ShardNodeDetector implementation (#16373) 
Use etcd as a ShardNodeDetector
Unit tests is based on embedded etcd

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-06 15:43:31 +08:00
bigsheeper 2b27fa4609
Print numTimestamps instead of Timestamps (#16369) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-06 14:47:31 +08:00
bigsheeper 4f4a417c8f
Unify in log print (#16368) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-04-06 14:09:35 +08:00
Letian Jiang 09b90fd8bf
Fix QueryCollection retrieve unit test (#16376) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-04 21:07:30 +08:00
xige-16 99984b88e1
Support delete varChar value (#16229) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-04-02 17:43:29 +08:00
congqixia aa1c26de77
Add ShardCluster implementation (#16360) 
ShardCluster maintains shard replica meta information
It watches node & segment change events
Provides shard replica search/query services

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-02 17:35:29 +08:00
Letian Jiang 028c5cb882
Modify grpc interface for replica Search/Query in QueryNode (#16326) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-02 14:15:31 +08:00
congqixia 7a44fff8cd
Use fmt.Errorf instead of string concat in tsafe_replica (#16344) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-04-01 18:37:28 +08:00
Letian Jiang cb65860e22
Add implementation for QueryShardService (#16317) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-04-01 14:19:29 +08:00
Letian Jiang ad9a2217ae
Add interface for QueryShardService and QueryShard (#16309) 
Signed-off-by: Letian Jiang <letian.jiang@zilliz.com>
 2022-03-31 14:05:28 +08:00
congqixia 899638810a
Add error log when tsafe register watcher more than once (#16294) 
Add some test case to cover the error case
Add log in error branch

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 22:07:28 +08:00
Jiquan Long ba37531456
Add support for loading multiple indexes (#16138) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-30 21:11:28 +08:00
Cai Yudong 5a9e1af3bb
Update create plan CGO interface (#16271) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-30 17:13:27 +08:00
congqixia 18a3e9f265
Add grpc interface for replica Search/Query in QueryNode (#16197) 
Resolves #16195
Add Search and Query grpc interface in query proto and types/types.go

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-30 12:03:27 +08:00
jaime d5d6e115bd
Refine delete segment data log (#16241) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-29 18:41:27 +08:00
congqixia 91d5436807
Improve log level and param function in QueryCollection (#16254) 
Use debug level for all debug information
Use specific parameter function for appliable ones

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-29 12:31:27 +08:00
bigsheeper 9d673006e0
Improve primary keys print (#16232) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-28 23:15:27 +08:00
xige-16 d2424fc50a
Print msgID instead of loadInfo when load segment (#16238) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-28 23:13:27 +08:00
godchen 478f6ca11e
Add cache for vector chunk (#15912) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-26 22:05:26 +08:00
xige-16 205c92e54b
Support insert string data (#15993) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-03-25 14:27:25 +08:00
bigsheeper 7fee4743f5
Improve log print in OOM error info (#16170) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-24 20:17:25 +08:00
yah01 63a71e247a
Loading segments concurrency level fallbacks to the value that won't cause OOM (#16172) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-24 13:39:25 +08:00
xiyichan ca129d4308
Support configurable msgstream (#16131) 
Signed-off-by: xiyichan <2863768433@qq.com>
 2022-03-24 10:15:25 +08:00
Jiquan Long f8d9bc919d
Unify interface of vector index & scalar index. (#15959) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-21 14:23:24 +08:00
yah01 31c5b286da
Fix wrong unittest, check whether loading collection will cause OOM (#16068) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-18 13:17:22 +08:00
godchen 78557ca6d7
Replace minio kv with minio chunk manager (#15936) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-03-17 18:03:23 +08:00
Ji Bin 3cd28420f1
Support compile under windows (#15786) 
This patch makes compile milvus under windows(MSYS), including:
- some cpp adaptation for compile under msys/gcc-10.3
- install toolchain scripts for setup from MinGW/MSYS `scripts/install_deps_msys.sh`
- adaptation for POSIX API use in golang
  * using gofrs/flock instead of syscall.Flock
  * using x/exp/mmap instead of syscall.Mmap
- introducing github actions for build milvus.exe under windows/MSYS
- rocksdb's patch for MSYS
- adaptation for compile knowhere under windows
- a windows package script for pack zip file, `scripts/package_windows.sh`

issue #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-03-17 17:17:22 +08:00
Jiquan Long ffdad82b9c
Set load info in reschedule tasks (#16065) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-16 11:43:21 +08:00
zhenshan.cao f3eeecf146
Reduce the number of metrics (#16030) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-15 21:51:21 +08:00
congqixia b61e8201e7
Refactor querynode historical replica (#15992) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-15 19:15:20 +08:00
jaime 97b1ed7bca
Remove reader API from Mqstream and replace it with consumer API (#15971) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-15 14:45:22 +08:00
bigsheeper caf9cbfcd4
Fix load failure and remove parition release related code (#16038) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-14 23:18:02 +08:00
congqixia 6a66faa2fb
Explictly specify Load operation meta (#16027) 
Fix #15996

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-03-14 09:50:01 +08:00
jaime 29975a7a26
Remove Consume() API from mqstream interface (#15886) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-11 20:09:59 +08:00
yah01 4f3d5fb6eb
Make load operation parallel improve performance (#15928) 
Signed-off-by: yah01 <yang.cen@zilliz.com>
 2022-03-11 14:28:01 +08:00
bigsheeper 9ad5c14a4a
Return error when create message stream failed in QueryNode (#15317) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-09 15:17:59 +08:00
顾炎 4f8c08f912
fix checkSegmentSize overflows int64 (#15858) 
Signed-off-by: guyan <632722781@qq.com>
 2022-03-09 13:29:58 +08:00
zhenshan.cao dff08dbf47
Refactor querynode unittest (#15929) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-03-08 17:39:58 +08:00
bigsheeper 6fec7ed6fb
Add search benchmark in QueryNode (#15854) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-08 10:15:59 +08:00
Jiquan Long f71651e294
Support column-based insert data in message stream (#15802) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-03-04 15:09:56 +08:00
Cai Yudong 3e862ecdfb
Merge msgChannelConfig and knowhereConfig into commonConfig (#15843) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-04 11:17:56 +08:00
jaime 7a3a721380
Reconstruct mqstream module (#15784) 
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>
 2022-03-03 21:57:56 +08:00
cai.zhang 88527fc0a9
Fix metrics label conflict (#15839) 
Signed-off-by: cai.zhang <cai.zhang@zilliz.com>
 2022-03-03 16:05:57 +08:00
Cai Yudong 92c8e32ebd
Let MemoryKV.Load return error when key not exist (#15814) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-03-02 18:51:55 +08:00
bigsheeper 12eb8c3da5
Add prometheus metrics for QueryNode (#15649) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-03-02 14:49:55 +08:00
bigsheeper 6a48071a5c
Set BaseMsg for LoadSegmentRequest, prevent crashing in log printing (#15725) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-02-26 20:09:54 +08:00
godchen fcf0887d92
Add and implement chunk manager interface. (#15541) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-02-24 23:53:53 +08:00
zhenshan.cao 142848fcc3
Abandon using protobuf to pass binaryset parameter (#15626) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-02-18 18:39:50 +08:00
congqixia c7f55c2e34
Make SessionWatch keep watch even no Rewatch func when find ErrCompacted (#15497) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-15 15:07:48 +08:00
yah01 684110bc9a
Avoid to copy during converting C bytes to Go bytes (#15551) 
* Avoid to copy during converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>

* Add BytesConverter to achieve no-copy converting C bytes to Go bytes

Signed-off-by: yah01 <yah2er0ne@outlook.com>
 2022-02-15 14:17:48 +08:00
Xiaofan a18d6b46a4
Fix watch dml channel fail because of no collection meta (#15436) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-02-10 13:19:46 +08:00
Cai Yudong 8255b713ac
Move ProxySubName/QueryNodeSubName/DataNodeSubName to msgChannelConfig (#15501) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-09 14:41:45 +08:00
jaime 307a8ce535
Support compile and run on Mac (#15491) 
Co-authored-by: jaime <yun.zhang@zilliz.com>
Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
Signed-off-by: yun.zhang <yun.zhang@zilliz.com>

Co-authored-by: Cai Yudong <yudong.cai@zilliz.com>
Co-authored-by: Jenny Li <jing.li@zilliz.com>
Co-authored-by: Nemo <yuchen.gao@zilliz.com>
 2022-02-09 14:27:46 +08:00
xige-16 55cdc5aa35
Get indexInfo and segment size by queryCoord (#14207) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-02-08 21:57:46 +08:00
Cai Yudong 1f337891e1
Rename base_param to service_param, rename global_param to component_param (#15463) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-08 20:57:47 +08:00
Ji Bin d83f69bb21
Support windows compliation (#15448) 
- remove dl dependency while compiling some lib under MSYS
- int types aligement in golang
- core/src/utils adaptation for compile under MSYS

issue: #7706

Signed-off-by: Ji Bin <matrixji@live.com>
 2022-02-07 23:43:46 +08:00
congqixia d830791e4f
Respect SegmentIDs in querynode.GetSegmentInfo (#15455) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2022-02-07 15:35:44 +08:00
Cai Yudong f1ea9613ae
Add EtcdConfig and move PulsarConfig/RocksdbConfig/MinioConfig into BaseParamTable (#15434) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-07 10:09:45 +08:00
Cai Yudong 4bccc85534
Use MsgChannelConfig in GlobalParams for all components (#15163) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-02-02 00:35:43 +08:00
zhenshan.cao 5bc6ec7250
Fixbug: QueryNode search exceptions will cause memory leak (#15305) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2022-01-24 21:19:39 +08:00
xige-16 1cd2363bd8
Delete usless params SkipQueryChannelRecovery (#15289) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-21 21:43:40 +08:00
xige-16 57383c9f76
Delete Useless timetick channel (#15296) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-21 14:31:39 +08:00
xige-16 c132302129
Clean flowGraph if watchChannel failed (#15303) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-20 10:01:38 +08:00
xige-16 bbb672cb00
Remove state service in querynode (#15281) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-18 18:43:37 +08:00
xige-16 dfc6670f0f
Unsubscribe channel after query node down (#15230) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-17 17:37:37 +08:00
Jiquan Long 675e6d352b
QueryNodes send search/query results by rpc (#15223) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2022-01-17 14:41:35 +08:00
Xiaofan f617c8887b
Time Tick Failed need log (#15225) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2022-01-14 23:55:34 +08:00
bigsheeper 0c98f21d4d
Remove global sealed segments in historical (#15177) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-12 17:01:38 +08:00
bigsheeper ebed1a68ff
Add log for segcore search (#15159) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-11 18:07:34 +08:00
Cai Yudong 951249c6c1
Use KnowhereConfig for indexnode and querynode (#15112) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-11 09:35:34 +08:00
xige-16 99315ce00e
Optimize recentlyModified's lock (#15120) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-10 23:17:36 +08:00
bigsheeper 018f32847f
[skip e2e] Add comment for createSearchPlan (#15111) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-10 22:36:06 +08:00
bigsheeper 4174daee80
[skip e2e] Add comment for newPartition (#15110) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-10 22:34:09 +08:00
bigsheeper 67e8c09df5
[skip e2e] Add comment for removeSegmentID (#15109) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-10 22:32:14 +08:00
Cai Yudong b6a48817b0
Add MinioConfig in GlobalParams for all components (#15099) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-10 17:29:34 +08:00
xige-16 97ea9786e1
[skip e2e]Improve getSegmentByIDPrivate log format (#15028) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-10 10:11:34 +08:00
xige-16 43b09c83d1
[skip e2e]Improve getPartitionByIDPrivate log format (#15027) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-10 10:09:39 +08:00
Cai Yudong 7efb02a4c5
Use PulsarConfig in GlobalParams for all components (#15046) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-09 23:45:33 +08:00
xige-16 708dfddba6
[skip e2e]Improve getSegmentInfosByColID log format (#15026) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-07 17:55:23 +08:00
bigsheeper 8d24428c4e
[skip e2e] Add comment for addSegmentID (#15020) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-07 17:47:48 +08:00
bigsheeper a7ad3f2a76
[skip e2e] Add comment for getSystemInfoMetrics (#15019) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-07 17:45:56 +08:00
bigsheeper 36b9c498ab
[skip e2e] Add comment for appendIndex (#15018) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-07 17:44:04 +08:00
Cai Yudong f8f61fb116
Use BaseParams etcd configuration for all components (#15012) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2022-01-07 16:15:26 +08:00
xige-16 f808390f32
[skip e2e]Improve getCollectionID error log (#14964) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-06 23:43:22 +08:00
xige-16 9dbb37c7ea
[skip e2e]Improve addCollection error log (#14963) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-06 23:41:28 +08:00
bigsheeper f90d5395a1
[skip e2e] Add comment for appendFieldInfo (#14958) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-06 23:35:47 +08:00
bigsheeper d38ad2f511
[skip e2e] Add comment for appendIndexParam (#14957) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-06 23:33:55 +08:00
xige-16 f7368e35a4
Add log for LoadSegmentsTask (#14703) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-06 23:26:18 +08:00
bigsheeper c79cbd341a
[skip e2e] Add comment for deleteLoadIndexInfo (#14956) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-06 23:22:03 +08:00
xige-16 0004238669
[skip e2e]Improve AddQueryChannel log (#14962) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-06 23:17:26 +08:00
xige-16 851794598e
[skip e2e]Improve log format (#14886) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-05 21:17:21 +08:00
xige-16 b3d33a78f5
[skip e2e]Improve log format (#14887) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-05 21:15:20 +08:00
xige-16 d751357be1
[skip e2e]Improve log format (#14885) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-05 21:13:20 +08:00
bigsheeper 64a5db66e9
[skip e2e] Add comment for newLoadIndexInfo (#14883) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-05 21:05:20 +08:00
bigsheeper fe730024ab
[skip e2e] Add comment for newIndexLoader (#14882) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-05 21:03:26 +08:00
bigsheeper 8b9ce5780d
[skip e2e] Add comment for setIndexInfo (#14881) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-05 20:59:30 +08:00
bigsheeper 12f37ffb57
Improve timeout log when query in QueryNode (#14766) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-05 10:35:31 +08:00
xige-16 abf1f58abe
Add log for watchDeltaChannelTask's nodeID (#14702) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2022-01-04 20:45:24 +08:00
godchen 9e51591b3a
Fix pulsar reader not close (#14543) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2022-01-04 14:45:19 +08:00
bigsheeper bd7a1cd3d1
[skip e2e] Add comment for getIndexInfo (#14745) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-04 14:06:02 +08:00
bigsheeper cda172ede2
[skip e2e] Add comment for estimateIndexBinlogSize (#14744) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-04 14:04:03 +08:00
bigsheeper 6bace04dd3
[skip e2e] Add comment for getIndexBinlog (#14743) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2022-01-04 14:02:12 +08:00
bigsheeper 2e6fe7d815
[skip e2e] Add comment for printIndexParams (#14694) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-31 19:09:19 +08:00
bigsheeper c95deb2266
[skip e2e] Add comment for loadIndex (#14693) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-31 19:07:23 +08:00
bigsheeper 6b806fc850
[skip e2e] Add comment for getReadyLoad (#14692) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-31 19:05:24 +08:00
xige-16 148dad23ac
Format watchDmChannel's log (#14633) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-30 20:27:20 +08:00
bigsheeper 8a3a1b556a
Free OS memory in QueryNode and IndexNode (#14305) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-30 20:03:29 +08:00
bigsheeper 983d9b4b73
[skip e2e] Add comment for getIndexParams (#14580) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-30 16:57:33 +08:00
bigsheeper 26a8bac54c
[skip e2e] Add comment for getIndexPaths (#14579) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-30 16:56:04 +08:00
bigsheeper 168f05de3a
[skip e2e] Add comment for getFieldID (#14578) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-30 16:39:04 +08:00
Jiquan Long ebb9b24b47
Improve name of flowgraph node (#14538) 
Signed-off-by: dragondriver <jiquan.long@zilliz.com>
 2021-12-30 10:33:46 +08:00
xige-16 8b919f50c1
[skip e2e] Add debug log for watchDmChannelReq (#14551) 
Signed-off-by: xige-16 <xi.ge@zilliz.com>
 2021-12-29 21:51:45 +08:00
godchen eb7dcf9ed3
Fix do unsolved msg goroutine not release (#14536) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-29 21:06:03 +08:00
bigsheeper 20178360ba
[skip e2e] Add comment for getBuildID (#14520) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-29 17:39:04 +08:00
bigsheeper 0415618f81
[skip e2e] Add comment for isHealthy (#14519) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-29 17:37:25 +08:00
bigsheeper 6bd640c130
[skip e2e] Add comment for historical retrieve (#14518) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-29 17:35:27 +08:00
Xiaofan 69087ff8dd
Support embedded etcd (#14333) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 14:35:21 +08:00
Xiaofan 1408926cd9
Remove logs without enough info (#14440) 
Signed-off-by: xiaofan-luan <xiaofan.luan@zilliz.com>
 2021-12-29 10:06:47 +08:00
bigsheeper 33b68a11fd
[skip e2e] Add comment for globalSealedSegmentManager (#14444) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-28 19:44:09 +08:00
bigsheeper b4a04a36ab
[skip e2e] Add comment for removeGlobalSegmentIDsByPartitionIds (#14443) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-28 19:40:22 +08:00
bigsheeper 56d41eaf36
[skip e2e] Add comment for removeGlobalSealedSegmentInfo (#14442) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-28 19:38:36 +08:00
bigsheeper 025fff0bbb
[skip e2e] Add comment for getGlobalSegmentIDs (#14357) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-28 18:48:42 +08:00
bigsheeper c0ba4c32af
[skip e2e] Add comment for getGlobalSegmentIDsByPartitionIds (#14358) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-28 13:19:40 +08:00
bigsheeper b20d3e9d13
[skip e2e] Add comment for hasGlobalSealedSegment (#14359) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-27 22:58:17 +08:00
bigsheeper d384bb1a83
[skip e2e] Add comment for addGlobalSegmentInfo (#14244) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-25 23:20:24 +08:00
bigsheeper 551c685df3
[skip e2e] Add comment for newGlobalSealedSegmentManager (#14243) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-25 23:18:30 +08:00
bigsheeper d5321f54c6
[skip e2e] Add comment for globalSealedSegmentManager (#14242) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-25 23:16:40 +08:00
godchen 5edc4903d6
[skip e2e]Sort imports (#14182) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-25 19:22:19 +08:00
zhenshan.cao 684a63b698
Fix bug:wait for search finish when stop querynode (#14228) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-25 13:36:19 +08:00
zhenshan.cao 9a2c91d326
Fix bug: refresh config after session init (#14194) 
Signed-off-by: zhenshan.cao <zhenshan.cao@zilliz.com>
 2021-12-24 22:22:17 +08:00
bigsheeper b22cf68c36
[skip e2e] Add comment for newServiceTimeNode (#14205) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-24 19:35:20 +08:00
bigsheeper 84ac0a8b7f
[skip e2e] Add comment for serviceTimeNode operate (#14204) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-24 19:33:39 +08:00
bigsheeper e5e5937409
[skip e2e] Add comment for serviceTimeNode (#14203) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-24 19:31:31 +08:00
godchen 381698fc18
[skip e2e]Sort imports (#14181) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 18:10:29 +08:00
godchen b2d4ca8fac
[skip e2e]Sort imports (#14180) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 18:08:39 +08:00
godchen 99798e1318
[skip e2e]Sort imports (#14179) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 18:06:30 +08:00
godchen 08102b8945
[skip e2e]Fix comment typo (#14178) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:28:20 +08:00
godchen 817182f48f
[skip e2e]Fix comment typo (#14184) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:18:28 +08:00
godchen acd792515d
[skip e2e]Fix comment typo (#14177) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:16:41 +08:00
godchen 8a2c3ecefd
[skip e2e]Fix comment typo (#14176) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:14:52 +08:00
godchen 85094a99fa
[skip e2e]Fix comment typo (#14175) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:13:10 +08:00
godchen af7517d80b
[skip e2e]Fix comment typo (#14174) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:10:54 +08:00
godchen 7bead10aa8
[skip e2e]sort imports (#14183) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:08:42 +08:00
godchen c52fa2412d
[skip e2e]Fix comment typo (#14185) 
Signed-off-by: godchen0212 <qingxiang.chen@zilliz.com>
 2021-12-24 17:02:50 +08:00
congqixia c62457a6ea
Add channel name check before querynode seek (#14169) 
Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>
 2021-12-24 15:38:17 +08:00
edward.zeng 4660edaaf4
[skip e2e] Refine querynode impl comment (#14137) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-24 13:26:30 +08:00
edward.zeng 1af317e3a3
[skip e2e] Refine querynode collection comment (#14136) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-24 13:23:51 +08:00
edward.zeng 43da0a80d4
[skip e2e] Refine querynode cgo helper comment (#14133) 
Signed-off-by: Edward Zeng <jie.zeng@zilliz.com>
 2021-12-24 13:14:50 +08:00
XuanYang-cn 3d24007efa
Add more logs for load sealed segment (#14079) 
Add logs before&after flowgraph_ddnode forwarding delete msg

See also: #13879

Signed-off-by: yangxuan <xuan.yang@zilliz.com>
 2021-12-24 10:50:07 +08:00
bigsheeper 7cc79d66b1
[skip e2e] Add comment for serviceTimeNode Name (#14102) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-23 21:54:15 +08:00
bigsheeper d050b804f4
[skip e2e] Add comment for serviceTimeNode (#14101) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-23 21:52:32 +08:00
bigsheeper 8e969a2ae8
[skip e2e] Add comment for queryNodeFlowGraph close (#14100) 
Signed-off-by: bigsheeper <yihao.dai@zilliz.com>
 2021-12-23 21:50:15 +08:00
Cai Yudong 891c202b73
Use GlobalParamTable for all components (#13885) 
Signed-off-by: yudong.cai <yudong.cai@zilliz.com>
 2021-12-23 18:39:11 +08:00
cai.zhang 3418c17d90
[skip e2e]Add log for reducing results in querynode (#13972) 
Signed-off-by: Cai.Zhang <cai.zhang@zilliz.com>
 2021-12-22 20:47:08 +08:00