milvus/CHANGELOG.md

608 lines
25 KiB
Markdown
Raw Normal View History

# Changelog
Please mark all change in change log and use the issue from GitHub
# Milvus 0.7.0 (TBD)
## Bug
- \#715 Milvus crash when searching and building index simultaneously using SQ8H
- \#744 Don't return partition table for show_tables
- \#770 Server unittest run failed on low-end server
- \#805 IVFTest.gpu_seal_test unittest failed
- \#831 Judge branch error in CommonUtil.cpp
- \#977 Server crash when create tables concurrently
- \#990 check gpu resources setting when assign repeated value
- \#995 table count set to 0 if no tables found
- \#1010 improve error message when offset or page_size is equal 0
- \#1022 check if partition name is legal
- \#1028 check if table exists when show partitions
- \#1029 check if table exists when try to delete partition
- \#1066 optimize http insert and search speed
- \#1067 Add binary vectors support in http server
- \#1075 improve error message when page size or offset is illegal
- \#1082 check page_size or offset value to avoid float
- \#1115 http server support load table into memory
- \#1152 Error log output continuously after server start
- \#1211 Server down caused by searching with index_type: HNSW
- \#1240 Update license declaration
- \#1298 Unittest failed when on CPU2GPU case
- \#1359 Negative distance value returned when searching with HNSW index type
- \#1429 Server crashed when searching vectors using GPU
- \#1476 Fix vectors results bug when getting vectors from segments
- \#1484 Index type changed to IDMAP after compacted
- \#1499 Fix duplicated ID number issue
- \#1491 Server crashed during adding vectors
- \#1504 Avoid possible race condition between delete and search
- \#1510 Add set interfaces for WAL configurations
- \#1511 Fix big integer cannot pass to server correctly
- \#1518 Table count did not match after deleting vectors and compact
- \#1525 Add setter API for config preload_table
## Feature
- \#216 Add CLI to get server info
- \#343 Add Opentracing
- \#665 Support get/set config via CLI
- \#759 Put C++ sdk out of milvus/core
- \#766 If partition tag is similar, wrong partition is searched
- \#771 Add server build commit info interface
- \#788 Add web server into server module
- \#813 Add push mode for prometheus monitor
- \#815 Support MinIO storage
- \#823 Support binary vector tanimoto/jaccard/hamming metric
- \#830 Support WAL(write-ahead logging)
- \#853 Support HNSW
- \#861 Support DeleteById / SearchByID / GetVectorById / Flush
- \#910 Change Milvus c++ standard to c++17
- \#1122 Support AVX-512 in FAISS
- \#1204 Add api to get table data information
- \#1250 Support CPU profiling
- \#1302 Get all record IDs in a segment by given a segment id
- \#1461 Add crud APIs and segments APIs into http module
- \#1463 Update config version to 0.2
## Improvement
- \#738 Use Openblas / lapack from apt install
- \#758 Enhance config description
- \#791 Remove Arrow
- \#834 add cpu mode for built-in Faiss
- \#848 Add ready-to-use config files to the Milvus repo for enhanced user experince
- \#860 Remove redundant checks in CacheMgr's constructor
- \#908 Move "primary_path" and "secondary_path" to storage config
- \#931 Remove "collector" from config
- \#966 Update NOTICE.md
- \#1002 Rename minio to s3 in Storage Config section
- \#1078 Move 'insert_buffer_size' to Cache Config section
- \#1105 Error message is not clear when creating IVFSQ8H index without gpu resources
- \#740, #849, #878, #972, #1033, #1161, #1173, #1199, #1190, #1223, #1222, #1257, #1264, #1269, #1164, #1303, #1304, #1324, #1388, #1459 Various fixes and improvements for Milvus documentation.
- \#1297 Hide partition_name parameter, avid user directly access partition table
- \#1310 Add default partition tag for a table
- \#1234 Do S3 server validation check when Milvus startup
- \#1263 Allow system conf modifiable and some take effect directly
- \#1320 Remove debug logging from faiss
- \#1426 Support to configure whether to enabled autoflush and the autoflush interval
- \#1444 Improve delete
- \#1480 Add return code for AVX512 selection
Delete and WAL feature branch merge (#1436) * add read/write lock * change compact to ddl queue * add api to get vector data * add flush / merge / compact lock * add api to get vector data * add data size for table info * add db recovery test * add data_size check * change file name to uppercase Signed-off-by: jinhai <hai.jin@zilliz.com> * update wal flush_merge_compact_mutex_ * update wal flush_merge_compact_mutex_ * change requirement * change requirement * upd requirement * add logging * add logging * add logging * add logging * add logging * add logging * add logging * add logging * add logging * delete part * add all size checks * fix bug * update faiss get_vector_by_id * add get_vector case * update get vector by id * update server * fix DBImpl * attempting to fix #1268 * lint * update unit test * fix #1259 * issue 1271 fix wal config * update * fix cases Signed-off-by: del.zhenwu <zhenxiang.li@zilliz.com> * update read / write error message * update read / write error message * [skip ci] get vectors by id from raw files instead faiss * [skip ci] update FilesByType meta * update * fix ci error * update * lint * Hide partition_name parameter * Remove douban pip source Signed-off-by: zhenwu <zw@zilliz.com> * Update epsilon value in test cases Signed-off-by: zhenwu <zw@zilliz.com> * Add default partition * Caiyd crud (#1313) * fix clang format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix unittest build error Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add faiss_bitset_test Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * avoid user directly operate partition table * fix has table bug * Caiyd crud (#1323) * fix clang format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix unittest build error Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * use compile option -O3 Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update faiss_bitset_test.cpp Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * change open flags * change OngoingFileChecker to static instance * mark ongoing files when applying deletes * update clean up with ttl * fix centos ci * update * lint * update partition Signed-off-by: zhenwu <zw@zilliz.com> * update delete and flush to include partitions * update * Update cases Signed-off-by: zhenwu <zw@zilliz.com> * Fix test cases crud (#1350) * fix order * add wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix invalid operation issue Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix invalid operation issue Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix bug Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix bug Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * crud fix Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * crud fix Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * add table info test cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * merge cases Signed-off-by: zhenwu <zw@zilliz.com> * Shengjun (#1349) * Add GPU sharing solution on native Kubernetes (#1102) * run hadolint with reviewdog * add LINCENSE in Dockerfile * run hadolint with reviewdog * Reporter of reviewdog command is "github-pr-check" * format Dockerfile * ignore DL3007 in hadolint * clean up old docker images * Add GPU sharing solution on native Kubernetes * nightly test mailer * Fix http server bug (#1096) * refactoring(create_table done) * refactoring * refactor server delivery (insert done) * refactoring server module (count_table done) * server refactor done * cmake pass * refactor server module done. * set grpc response status correctly * format done. * fix redefine ErrorMap() * optimize insert reducing ids data copy * optimize grpc request with reducing data copy * clang format * [skip ci] Refactor server module done. update changlog. prepare for PR * remove explicit and change int32_t to int64_t * add web server * [skip ci] add license in web module * modify header include & comment oatpp environment config * add port configure & create table in handler * modify web url * simple url complation done & add swagger * make sure web url * web functionality done. debuging * add web unittest * web test pass * add web server port * add web server port in template * update unittest cmake file * change web server default port to 19121 * rename method in web module & unittest pass * add search case in unittest for web module * rename some variables * fix bug * unittest pass * web prepare * fix cmd bug(check server status) * update changlog * add web port validate & default set * clang-format pass * add web port test in unittest * add CORS & redirect root to swagger ui * add web status * web table method func cascade test pass * add config url in web module * modify thirdparty cmake to avoid building oatpp test * clang format * update changlog * add constants in web module * reserve Config.cpp * fix constants reference bug * replace web server with async module * modify component to support async * format * developing controller & add test clent into unittest * add web port into demo/server_config * modify thirdparty cmake to allow build test * remove unnecessary comment * add endpoint info in controller * finish web test(bug here) * clang format * add web test cpp to lint exclusions * check null field in GetConfig * add macro RETURN STATUS DTo * fix cmake conflict * fix crash when exit server * remove surplus comments & add http param check * add uri /docs to direct swagger * format * change cmd to system * add default value & unittest in web module * add macros to judge if GPU supported * add macros in unit & add default in index dto & print error message when bind http port fail * format (fix #788) * fix cors bug (not completed) * comment cors * change web framework to simple api * comments optimize * change to simple API * remove comments in controller.hpp * remove EP_COMMON_CMAKE_ARGS in oatpp and oatpp-swagger * add ep cmake args to sqlite * clang-format * change a format * test pass * change name to * fix compiler issue(oatpp-swagger depend on oatpp) * add & in start_server.h * specify lib location with oatpp and oatpp-swagger * add comments * add swagger definition * [skip ci] change http method options status code * remove oatpp swagger(fix #970) * remove comments * check Start web behavior * add default to cpu_cache_capacity * remove swagger component.hpp & /docs url * remove /docs info * remove /docs in unittest * remove space in test rpc * remove repeate info in CHANGLOG * change cache_insert_data default value as a constant * [skip ci] Fix some broken links (#960) * [skip ci] Fix broken link * [skip ci] Fix broken link * [skip ci] Fix broken link * [skip ci] Fix broken links * fix issue 373 (#964) * fix issue 373 * Adjustment format * Adjustment format * Adjustment format * change readme * #966 update NOTICE.md (#967) * remove comments * check Start web behavior * add default to cpu_cache_capacity * remove swagger component.hpp & /docs url * remove /docs info * remove /docs in unittest * remove space in test rpc * remove repeate info in CHANGLOG * change cache_insert_data default value as a constant * adjust web port cofig place * rename web_port variable * change gpu resources invoke way to cmd() * set advanced config name add DEFAULT * change config setting to cmd * modify .. * optimize code * assign TableDto' count default value 0 (fix #995) * check if table exists when show partitions (fix #1028) * check table exists when drop partition (fix #1029) * check if partition name is legal (fix #1022) * modify status code when partition tag is illegal * update changlog * add info to /system url * add binary index and add bin uri & handler method(not completed) * optimize http insert and search time(fix #1066) | add binary vectors support(fix #1067) * fix test partition bug * fix test bug when check insert records * add binary vectors test * add default for offset and page_size * fix uinttest bug * [skip ci] remove comments * optimize web code for PR comments * add new folder named utils * check offset and pagesize (fix #1082) * improve error message if offset or page_size is not legal (fix #1075) * add log into web module * update changlog * check gpu sources setting when assign repeated value (fix #990) * update changlog * clang-format pass * add default handler in http handler * [skip ci] improve error msg when check gpu resources * change check offset way * remove func IsIntStr * add case * change int32 to int64 when check number str * add log in we module(doing) * update test case * add log in web controller Co-authored-by: jielinxu <52057195+jielinxu@users.noreply.github.com> Co-authored-by: JackLCL <53512883+JackLCL@users.noreply.github.com> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> * Filtering for specific paths in Jenkins CI (#1107) * run hadolint with reviewdog * add LINCENSE in Dockerfile * run hadolint with reviewdog * Reporter of reviewdog command is "github-pr-check" * format Dockerfile * ignore DL3007 in hadolint * clean up old docker images * Add GPU sharing solution on native Kubernetes * nightly test mailer * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Fix Filtering for specific paths in Jenkins CI bug (#1109) * run hadolint with reviewdog * add LINCENSE in Dockerfile * run hadolint with reviewdog * Reporter of reviewdog command is "github-pr-check" * format Dockerfile * ignore DL3007 in hadolint * clean up old docker images * Add GPU sharing solution on native Kubernetes * nightly test mailer * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Fix Filtering for specific paths in Jenkins CI bug (#1110) * run hadolint with reviewdog * add LINCENSE in Dockerfile * run hadolint with reviewdog * Reporter of reviewdog command is "github-pr-check" * format Dockerfile * ignore DL3007 in hadolint * clean up old docker images * Add GPU sharing solution on native Kubernetes * nightly test mailer * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Don't skip ci when triggered by a time (#1113) * run hadolint with reviewdog * add LINCENSE in Dockerfile * run hadolint with reviewdog * Reporter of reviewdog command is "github-pr-check" * format Dockerfile * ignore DL3007 in hadolint * clean up old docker images * Add GPU sharing solution on native Kubernetes * nightly test mailer * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Don't skip ci when triggered by a time * Don't skip ci when triggered by a time * Set default sending to Milvus Dev mail group (#1121) * run hadolint with reviewdog * add LINCENSE in Dockerfile * run hadolint with reviewdog * Reporter of reviewdog command is "github-pr-check" * format Dockerfile * ignore DL3007 in hadolint * clean up old docker images * Add GPU sharing solution on native Kubernetes * nightly test mailer * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Test filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * Filtering for specific paths in Jenkins CI * No skip ci when triggered by a time * Don't skip ci when triggered by a time * Set default sending to Milvus Dev * Support hnsw (#1131) * add hnsw * add config * format... * format.. * Remove test.template (#1129) * Update framework * remove files * Remove files * Remove ann-acc cases && Update java-sdk cases * change cn to en * [skip ci] remove doc test * [skip ci] change cn to en * Case stability * Add mail notification when test failed * Add main notification * Add main notification * gen milvus instance from utils * Distable case with multiprocess * Add mail notification when nightly test failed * add milvus handler param * add http handler * Remove test.template Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com> * Add doc for the RESTful API / Update contributor number in Milvus readme (#1100) * [skip ci] Update contributor number. * [skip ci] Add RESTful API doc. * [skip ci] Some updates. * [skip ci] Change port to 19121. * [skip ci] Update README.md. Update the descriptions for OPTIONS. * Update README.md Fix a typo. * #1105 update error message when creating IVFSQ8H index without GPU resources (#1117) * [skip ci] Update README (#1104) * remove Nvidia owned files from faiss (#1136) * #1135 remove Nvidia owned files from faiss * Revert "#1135 remove Nvidia owned files from faiss" This reverts commit 3bc007c28c8df5861fdd0452fd64c0e2e719eda2. * #1135 remove Nvidia API implementation * #1135 remove Nvidia owned files from faiss * Update CODE_OF_CONDUCT.md (#1163) * Improve codecov (#1095) * Optimize config test. Dir src/config 99% lines covered * add unittest coverage * optimize cache&config unittest * code format * format * format code * fix merge conflict * cover src/utils unittest * '#831 fix exe_path judge error' * #831 fix exe_path judge error * add some unittest coverage * add some unittest coverage * improve coverage of src/wrapper * improve src/wrapper coverage * *test optimize db/meta unittest * fix bug * *test optimize mysqlMetaImpl unittest * *style: format code * import server& scheduler unittest coverage * handover next work * *test: add some test_meta test case * *format code * *fix: fix typo * feat(codecov): improve code coverage for src/db(#872) * feat(codecov): improve code coverage for src/db/engine(#872) * feat(codecov): improve code coverage(#872) * fix config unittest bug * feat(codecov): improve code coverage core/db/engine(#872) * feat(codecov): improve code coverage core/knowhere * feat(codecov): improve code coverage core/knowhere * feat(codecov): improve code coverage * feat(codecov): fix cpu test some error * feat(codecov): improve code coverage * feat(codecov): rename some fiu * fix(db/meta): fix switch/case default action * feat(codecov): improve code coverage(#872) * fix error caused by merge code * format code * feat(codecov): improve code coverage & format code(#872) * feat(codecov): fix test error(#872) * feat(codecov): fix unittest test_mem(#872) * feat(codecov): fix unittest(#872) * feat(codecov): fix unittest for resource manager(#872) * feat(codecov): code format (#872) * feat(codecov): trigger ci(#872) * fix(RequestScheduler): remove a wrong sleep statement * test(test_rpc): fix rpc test * Fix format issue * Remove unused comments * Fix unit test error Co-authored-by: ABNER-1 <ABNER-1@users.noreply.github.com> Co-authored-by: Jin Hai <hai.jin@zilliz.com> * Support run dev test with http handler in python SDK (#1116) * refactoring(create_table done) * refactoring * refactor server delivery (insert done) * refactoring server module (count_table done) * server refactor done * cmake pass * refactor server module done. * set grpc response status correctly * format done. * fix redefine ErrorMap() * optimize insert reducing ids data copy * optimize grpc request with reducing data copy * clang format * [skip ci] Refactor server module done. update changlog. prepare for PR * remove explicit and change int32_t to int64_t * add web server * [skip ci] add license in web module * modify header include & comment oatpp environment config * add port configure & create table in handler * modify web url * simple url complation done & add swagger * make sure web url * web functionality done. debuging * add web unittest * web test pass * add web server port * add web server port in template * update unittest cmake file * change web server default port to 19121 * rename method in web module & unittest pass * add search case in unittest for web module * rename some variables * fix bug * unittest pass * web prepare * fix cmd bug(check server status) * update changlog * add web port validate & default set * clang-format pass * add web port test in unittest * add CORS & redirect root to swagger ui * add web status * web table method func cascade test pass * add config url in web module * modify thirdparty cmake to avoid building oatpp test * clang format * update changlog * add constants in web module * reserve Config.cpp * fix constants reference bug * replace web server with async module * modify component to support async * format * developing controller & add test clent into unittest * add web port into demo/server_config * modify thirdparty cmake to allow build test * remove unnecessary comment * add endpoint info in controller * finish web test(bug here) * clang format * add web test cpp to lint exclusions * check null field in GetConfig * add macro RETURN STATUS DTo * fix cmake conflict * fix crash when exit server * remove surplus comments & add http param check * add uri /docs to direct swagger * format * change cmd to system * add default value & unittest in web module * add macros to judge if GPU supported * add macros in unit & add default in index dto & print error message when bind http port fail * format (fix #788) * fix cors bug (not completed) * comment cors * change web framework to simple api * comments optimize * change to simple API * remove comments in controller.hpp * remove EP_COMMON_CMAKE_ARGS in oatpp and oatpp-swagger * add ep cmake args to sqlite * clang-format * change a format * test pass * change name to * fix compiler issue(oatpp-swagger depend on oatpp) * add & in start_server.h * specify lib location with oatpp and oatpp-swagger * add comments * add swagger definition * [skip ci] change http method options status code * remove oatpp swagger(fix #970) * remove comments * check Start web behavior * add default to cpu_cache_capacity * remove swagger component.hpp & /docs url * remove /docs info * remove /docs in unittest * remove space in test rpc * remove repeate info in CHANGLOG * change cache_insert_data default value as a constant * [skip ci] Fix some broken links (#960) * [skip ci] Fix broken link * [skip ci] Fix broken link * [skip ci] Fix broken link * [skip ci] Fix broken links * fix issue 373 (#964) * fix issue 373 * Adjustment format * Adjustment format * Adjustment format * change readme * #966 update NOTICE.md (#967) * remove comments * check Start web behavior * add default to cpu_cache_capacity * remove swagger component.hpp & /docs url * remove /docs info * remove /docs in unittest * remove space in test rpc * remove repeate info in CHANGLOG * change cache_insert_data default value as a constant * adjust web port cofig place * rename web_port variable * change gpu resources invoke way to cmd() * set advanced config name add DEFAULT * change config setting to cmd * modify .. * optimize code * assign TableDto' count default value 0 (fix #995) * check if table exists when show partitions (fix #1028) * check table exists when drop partition (fix #1029) * check if partition name is legal (fix #1022) * modify status code when partition tag is illegal * update changlog * add info to /system url * add binary index and add bin uri & handler method(not completed) * optimize http insert and search time(fix #1066) | add binary vectors support(fix #1067) * fix test partition bug * fix test bug when check insert records * add binary vectors test * add default for offset and page_size * fix uinttest bug * [skip ci] remove comments * optimize web code for PR comments * add new folder named utils * check offset and pagesize (fix #1082) * improve error message if offset or page_size is not legal (fix #1075) * add log into web module * update changlog * check gpu sources setting when assign repeated value (fix #990) * update changlog * clang-format pass * add default handler in http handler * [skip ci] improve error msg when check gpu resources * change check offset way * remove func IsIntStr * add case * change int32 to int64 when check number str * add log in we module(doing) * update test case * add log in web controller * remove surplus dot * add preload into /system/ * change get_milvus() to get_milvus(args['handler']) * support load table into memory with http server (fix #1115) * [skip ci] comment surplus dto in VectorDto Co-authored-by: jielinxu <52057195+jielinxu@users.noreply.github.com> Co-authored-by: JackLCL <53512883+JackLCL@users.noreply.github.com> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> * Fix #1140 (#1162) * fix Signed-off-by: Nicky <nicky.xj.lin@gmail.com> * update... Signed-off-by: Nicky <nicky.xj.lin@gmail.com> * fix2 Signed-off-by: Nicky <nicky.xj.lin@gmail.com> * fix3 Signed-off-by: Nicky <nicky.xj.lin@gmail.com> * update changelog Signed-off-by: Nicky <nicky.xj.lin@gmail.com> * Update INSTALL.md (#1175) * Update INSTALL.md 1. Change image tag and Milvus source code to latest. 2. Fix a typo Signed-off-by: Lu Wang <yamasite@qq.com> * Update INSTALL.md Signed-off-by: lu.wang <yamasite@qq.com> * add Tanimoto ground truth (#1138) * add milvus ground truth * add milvus groundtruth * [skip ci] add milvus ground truth * [skip ci]add tanimoto ground truth * fix mix case bug (#1208) * fix mix case bug Signed-off-by: del.zhenwu <zhenxiang.li@zilliz.com> * Remove case.md Signed-off-by: del.zhenwu <zhenxiang.li@zilliz.com> * Update README.md (#1206) Add LFAI mailing lists. Signed-off-by: Lutkin Wang <yamasite@qq.com> * Add design.md to store links to design docs (#1219) * Update README.md Add link to Milvus design docs Signed-off-by: Lutkin Wang <yamasite@qq.com> * Create design.md Signed-off-by: Lutkin Wang <yamasite@qq.com> * Update design.md Signed-off-by: Lutkin Wang <yamasite@qq.com> * Add troubleshooting info about libmysqlpp.so.3 error (#1225) * Update INSTALL.md Signed-off-by: Lutkin Wang <yamasite@qq.com> * Update INSTALL.md Signed-off-by: Lutkin Wang <yamasite@qq.com> * Update README.md (#1233) Signed-off-by: Lutkin Wang <yamasite@qq.com> * #1240 Update license declaration of each file (#1241) * #1240 Update license declaration of each files Signed-off-by: jinhai <hai.jin@zilliz.com> * #1240 Update CHANGELOG Signed-off-by: jinhai <hai.jin@zilliz.com> * Update README.md (#1258) Add Jenkins master badge. Signed-off-by: Lutkin Wang <yamasite@qq.com> * Update INSTALL.md (#1265) Fix indentation. * support CPU profiling (#1251) * #1250 support CPU profiling Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * #1250 fix code coverage Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * Fix HNSW crash (#1262) * fix Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com> * update. Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com> * Add troubleshooting information for INSTALL.md and enhance readability (#1274) * Update INSTALL.md 1. Add new troubleshooting message; 2. Enhance readability. Signed-off-by: Lutkin Wang <yamasite@qq.com> * Update INSTALL.md Signed-off-by: Lutkin Wang <yamasite@qq.com> * Update INSTALL.md Signed-off-by: Lutkin Wang <yamasite@qq.com> * Update INSTALL.md Add CentOS link. Signed-off-by: Lutkin Wang <yamasite@qq.com> * Create COMMUNITY.md (#1292) Signed-off-by: Lutkin Wang <yamasite@qq.com> * fix gtest * add copyright * fix gtest * MERGE_NOT_YET * fix lint Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com> Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com> Co-authored-by: jielinxu <52057195+jielinxu@users.noreply.github.com> Co-authored-by: JackLCL <53512883+JackLCL@users.noreply.github.com> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> Co-authored-by: Tinkerrr <linxiaojun.cn@outlook.com> Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com> Co-authored-by: Lutkin Wang <yamasite@qq.com> Co-authored-by: shengjh <46514371+shengjh@users.noreply.github.com> Co-authored-by: ABNER-1 <ABNER-1@users.noreply.github.com> Co-authored-by: Jin Hai <hai.jin@zilliz.com> Co-authored-by: shiyu22 <cshiyu22@gmail.com> * #1302 Get all record IDs in a segment by given a segment id * Remove query time ranges Signed-off-by: zhenwu <zw@zilliz.com> * #1295 let wal enable by default * fix cases Signed-off-by: zhenwu <zw@zilliz.com> * fix partition cases Signed-off-by: zhenwu <zw@zilliz.com> * [skip ci] update test_db * update * fix case bug Signed-off-by: zhenwu <zw@zilliz.com> * lint * fix test case failures * remove some code * Caiyd crud 1 (#1377) * fix clang format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix unittest build error Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix build issue when enable profiling Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix hastable bug * update bloom filter * update * benchmark * update benchmark * update * update * remove wal record size Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * remove wal record size config Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * update apply deletes: switch to binary search * update sdk_simple Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update apply deletes: switch to binary search * add test_search_by_id Signed-off-by: zhenwu <zw@zilliz.com> * add more log * flush error with multi same ids Signed-off-by: zhenwu <zw@zilliz.com> * modify wal config Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * update * add binary search_by_id * fix case bug Signed-off-by: zhenwu <zw@zilliz.com> * update cases Signed-off-by: zhenwu <zw@zilliz.com> * fix unit test #1395 * improve merge performance * add uids_ for VectorIndex to improve search performance Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix error Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update * fix search * fix record num Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * refine code * refine code * Add get_vector_ids test cases (#1407) * fix order * add wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix wal case Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix invalid operation issue Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix invalid operation issue Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix bug Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * fix bug Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * crud fix Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * crud fix Signed-off-by: sahuang <xiaohaix@student.unimelb.edu.au> * add table info test cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> Signed-off-by: JinHai-CN <hai.jin@zilliz.com> * add to compact case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * add to compact case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * add to compact case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * add case and debug compact Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * test pdb Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * test pdb Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * test pdb Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix cases Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update table_info case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update table_info case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update table_info case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update get vector ids case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update get vector ids case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update get vector ids case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update get vector ids case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * update case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * pdb test Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * pdb test Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * add tests for get_vector_ids Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix case Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * add binary and ip Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix binary index Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * fix pdb Signed-off-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> * #1408 fix search result in-correct after DeleteById Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * add one case * delete failed segment * update serialize * update serialize * fix case Signed-off-by: zhenwu <zw@zilliz.com> * update * update case assertion Signed-off-by: zhenwu <zw@zilliz.com> * [skip ci] update config * change bloom filter msync flag to async * #1319 add more timing debug info Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * update * update * add normalize Signed-off-by: zhenwu <zw@zilliz.com> * add normalize Signed-off-by: zhenwu <zw@zilliz.com> * add normalize Signed-off-by: zhenwu <zw@zilliz.com> * Fix compiling error Signed-off-by: jinhai <hai.jin@zilliz.com> * support ip (#1383) * support ip Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com> * IP result distance sort by descend Signed-off-by: Nicky <nicky.xj.lin@gmail.com> * update Signed-off-by: Nicky <nicky.xj.lin@gmail.com> * format Signed-off-by: xiaojun.lin <xiaojun.lin@zilliz.com> * get table lsn * Remove unused third party Signed-off-by: jinhai <hai.jin@zilliz.com> * Refine code Signed-off-by: jinhai <hai.jin@zilliz.com> * #1319 fix clang format Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * fix wal applied lsn Signed-off-by: shengjun.li <shengjun.li@zilliz.com> * validate partition tag * #1319 improve search performance Signed-off-by: yudong.cai <yudong.cai@zilliz.com> * build error Co-authored-by: Zhiru Zhu <youny626@hotmail.com> Co-authored-by: groot <yihua.mo@zilliz.com> Co-authored-by: Xiaohai Xu <xiaohaix@student.unimelb.edu.au> Co-authored-by: shengjh <46514371+shengjh@users.noreply.github.com> Co-authored-by: del-zhenwu <56623710+del-zhenwu@users.noreply.github.com> Co-authored-by: shengjun.li <49774184+shengjun1985@users.noreply.github.com> Co-authored-by: Cai Yudong <yudong.cai@zilliz.com> Co-authored-by: quicksilver <zhifeng.zhang@zilliz.com> Co-authored-by: BossZou <40255591+BossZou@users.noreply.github.com> Co-authored-by: jielinxu <52057195+jielinxu@users.noreply.github.com> Co-authored-by: JackLCL <53512883+JackLCL@users.noreply.github.com> Co-authored-by: Tinkerrr <linxiaojun.cn@outlook.com> Co-authored-by: Lutkin Wang <yamasite@qq.com> Co-authored-by: ABNER-1 <ABNER-1@users.noreply.github.com> Co-authored-by: shiyu22 <cshiyu22@gmail.com>
2020-02-29 08:11:31 +00:00
## Task
- \#1327 Exclude third-party code from codebeat
- \#1331 Exclude third-party code from codacy
2019-12-07 08:25:32 +00:00
# Milvus 0.6.0 (2019-12-07)
2019-11-04 17:48:11 +00:00
## Bug
- \#228 memory usage increased slowly during searching vectors
- \#246 Exclude src/external folder from code coverage for jenkin ci
- \#248 Reside src/external in thirdparty
- \#316 Some files not merged after vectors added
- \#327 Search does not use GPU when index type is FLAT
- \#331 Add exception handle when search fail
- \#340 Test cases run failed on 0.6.0
- \#353 Rename config.h.in to version.h.in
- \#374 sdk_simple return empty result
- \#377 Create partition success if tag name only contains spaces
- \#397 sdk_simple return incorrect result
- \#399 Create partition should be failed if partition tag existed
- \#412 Message returned is confused when partition created with null partition name
- \#416 Drop the same partition success repeatally
- \#440 Query API in customization still uses old version
- \#440 Server cannot startup with gpu_resource_config.enable=false in GPU version
- \#458 Index data is not compatible between 0.5 and 0.6
- \#465 Server hang caused by searching with nsg index
- \#485 Increase code coverage rate
- \#486 gpu no usage during index building
- \#497 CPU-version search performance decreased
- \#504 The code coverage rate of core/src/scheduler/optimizer is too low
- \#509 IVF_PQ index build trapped into dead loop caused by invalid params
- \#513 Unittest DELETE_BY_RANGE sometimes failed
- \#523 Erase file data from cache once the file is marked as deleted
- \#527 faiss benchmark not compatible with faiss 1.6.0
- \#530 BuildIndex stop when do build index and search simultaneously
- \#532 assigin value to `table_name` from confest shell
- \#533 NSG build failed with MetricType Inner Product
- \#543 client raise exception in shards when search results is empty
- \#545 Avoid dead circle of build index thread when error occurs
- \#547 NSG build failed using GPU-edition if set gpu_enable false
- \#548 NSG search accuracy is too low
- \#552 Server down during building index_type: IVF_PQ using GPU-edition
- \#561 Milvus server should report exception/error message or terminate on mysql metadata backend error
- \#579 Build index hang in GPU version when gpu_resources disabled
- \#596 Frequently insert operation cost too much disk space
- \#599 Build index log is incorrect
- \#602 Optimizer specify wrong gpu_id
- \#606 No log generated during building index with CPU
- \#616 IP search metric_type is not supported by IVF_PQ index
- \#631 FAISS isn't compiled with O3 option
- \#636 (CPU) Create index PQ should be failed if table metric type set Inner Product
- \#649 Typo "partiton" should be "partition"
- \#654 Random crash when frequently insert vector one by one
- \#658 Milvus error out when building SQ8H index without GPU resources
- \#668 Update badge of README
- \#670 Random failure of unittest db_test::SEARCH_TEST
- \#674 Server down in stability test
- \#696 Metric_type changed from IP to L2
- \#705 Fix search SQ8H crash without GPU resource
2019-11-04 17:48:11 +00:00
## Feature
- \#12 Pure CPU version for Milvus
- \#77 Support table partition
- \#127 Support new Index type IVFPQ
- \#226 Experimental shards middleware for Milvus
- \#227 Support new index types SPTAG-KDT and SPTAG-BKT
- \#346 Support build index with multiple gpu
- \#420 Update shards merge part to match v0.5.3
- \#488 Add log in scheduler/optimizer
- \#502 C++ SDK support IVFPQ and SPTAG
- \#560 Add version in server config file
- \#605 Print more messages when server start
- \#644 Add a new rpc command to get milvus build version whether cpu or gpu
- \#709 Show last commit id when server start
2019-11-04 17:48:11 +00:00
## Improvement
- \#255 Add ivfsq8 test report detailed version
- \#260 C++ SDK README
- \#266 Rpc request source code refactor
- \#274 Logger the time cost during preloading data
- \#275 Rename C++ SDK IndexType
- \#284 Change C++ SDK to shared library
- \#306 Use int64 for all config integer
- \#310 Add Q&A for 'protocol https not supported or disable in libcurl' issue
- \#314 add Find FAISS in CMake
- \#322 Add option to enable / disable prometheus
- \#354 Build migration scripts into milvus docker image
- \#358 Add more information in build.sh and install.md
- \#404 Add virtual method Init() in Pass abstract class
- \#409 Add a Fallback pass in optimizer
- \#433 C++ SDK query result is not easy to use
- \#449 Add ShowPartitions example for C++ SDK
- \#470 Small raw files should not be build index
- \#584 Intergrate internal FAISS
- \#611 Remove MILVUS_CPU_VERSION
- \#634 FAISS GPU version is compiled with O0
- \#737 Refactor server module to separate Grpc from server handler and scheduler
2019-11-04 17:48:11 +00:00
## Task
2019-11-13 10:41:49 +00:00
# Milvus 0.5.3 (2019-11-13)
2019-11-06 04:04:28 +00:00
## Bug
- \#258 Bytes type in proto cause big-endian/little-endian problem
2019-11-06 04:04:28 +00:00
## Feature
## Improvement
- \#204 improve grpc performance in search
- \#207 Add more unittest for config set/get
- \#208 optimize unittest to support run single test more easily
- \#284 Change C++ SDK to shared library
- \#260 C++ SDK README
2019-11-06 04:04:28 +00:00
## Task
2019-11-04 17:48:11 +00:00
2019-11-07 07:45:14 +00:00
# Milvus 0.5.2 (2019-11-07)
2019-11-04 17:47:05 +00:00
## Bug
- \#194 Search faild: message="Table file doesn't exist"
2019-11-04 17:47:05 +00:00
## Feature
## Improvement
- \#190 Update default config:use_blas_threshold to 1100 and server version printout to 0.5.2
2019-11-04 17:47:05 +00:00
## Task
# Milvus 0.5.1 (2019-11-04)
## Bug
- \#134 JFrog cache error
- \#161 Search IVFSQHybrid crash on gpu
- \#169 IVF_FLAT search out of memory
## Feature
- \#90 The server start error messages could be improved to enhance user experience
- \#104 test_scheduler core dump
- \#115 Using new structure for tasktable
- \#139 New config option use_gpu_threshold
- \#146 Add only GPU and only CPU version for IVF_SQ8 and IVF_FLAT
- \#164 Add CPU version for building index
## Improvement
- \#64 Improvement dump function in scheduler
- \#80 Print version information into log during server start
- \#82 Move easyloggingpp into "external" directory
- \#92 Speed up CMake build process
- \#96 Remove .a file in milvus/lib for docker-version
- \#118 Using shared_ptr instead of weak_ptr to avoid performance loss
- \#122 Add unique id for Job
- \#130 Set task state MOVED after resource copy it completed
- \#149 Improve large query optimizer pass
- \#156 Not return error when search_resources and index_build_device set cpu
- \#159 Change the configuration name from 'use_gpu_threshold' to 'gpu_search_threshold'
- \#168 Improve result reduce
- \#175 add invalid config unittest
## Task
# Milvus 0.5.0 (2019-10-21)
## Bug
- MS-568 Fix gpuresource free error
- MS-572 Milvus crash when get SIGINT
- MS-577 Unittest Query randomly hung
- MS-587 Count get wrong result after adding vectors and index built immediately
- MS-599 Search wrong result when table created with metric_type: IP
- MS-601 Docker logs error caused by get CPUTemperature error
- MS-605 Server going down during searching vectors
- MS-620 Get table row counts display wrong error code
- MS-622 Delete vectors should be failed if date range is invalid
- MS-624 Search vectors failed if time ranges long enough
- MS-637 Out of memory when load too many tasks
- MS-639 SQ8H index created failed and server hang
- MS-640 Cache object size calculate incorrect
- MS-641 Segment fault(signal 11) in PickToLoad
- MS-644 Search crashed with index-type: flat
- MS-647 grafana display average cpu-temp
- MS-652 IVFSQH quantization double free
- MS-650 SQ8H index create issue
- MS-653 When config check fail, Milvus close without message
- MS-654 Describe index timeout when building index
- MS-658 Fix SQ8 Hybrid can't search
- MS-665 IVF_SQ8H search crash when no GPU resource in search_resources
- \#9 Change default gpu_cache_capacity to 4
- \#20 C++ sdk example get grpc error
- \#23 Add unittest to improve code coverage
- \#31 make clang-format failed after run build.sh -l
- \#39 Create SQ8H index hang if using github server version
- \#30 Some troubleshoot messages in Milvus do not provide enough information
- \#48 Config unittest failed
- \#59 Topk result is incorrect for small dataset
## Improvement
- MS-552 Add and change the easylogging library
- MS-553 Refine cache code
- MS-555 Remove old scheduler
- MS-556 Add Job Definition in Scheduler
- MS-557 Merge Log.h
- MS-558 Refine status code
- MS-562 Add JobMgr and TaskCreator in Scheduler
- MS-566 Refactor cmake
- MS-574 Milvus configuration refactor
- MS-578 Make sure milvus5.0 don't crack 0.3.1 data
- MS-585 Update namespace in scheduler
- MS-606 Speed up result reduce
- MS-608 Update TODO names
- MS-609 Update task construct function
- MS-611 Add resources validity check in ResourceMgr
- MS-619 Add optimizer class in scheduler
- MS-626 Refactor DataObj to support cache any type data
- MS-648 Improve unittest
- MS-655 Upgrade SPTAG
- \#42 Put union of index_build_device and search resources to gpu_pool
- \#67 Avoid linking targets multiple times in cmake
## Feature
- MS-614 Preload table at startup
- MS-627 Integrate new index: IVFSQHybrid
- MS-631 IVFSQ8H Index support
- MS-636 Add optimizer in scheduler for FAISS_IVFSQ8H
## Task
- MS-554 Change license to Apache 2.0
- MS-561 Add contributing guidelines, code of conduct and README docs
- MS-567 Add NOTICE.md
- MS-569 Complete the NOTICE.md
- MS-575 Add Clang-format & Clang-tidy & Cpplint
- MS-586 Remove BUILD_FAISS_WITH_MKL option
- MS-590 Refine cmake code to support cpplint
- MS-600 Reconstruct unittest code
- MS-602 Remove zilliz namespace
- MS-610 Change error code base value from hex to decimal
- MS-624 Re-organize project directory for open-source
- MS-635 Add compile option to support customized faiss
- MS-660 add ubuntu_build_deps.sh
- \#18 Add all test cases
2019-11-07 05:54:07 +00:00
# Milvus 0.4.0 (2019-09-12)
## Bug
- MS-119 The problem of combining the log files
- MS-121 The problem that user can't change the time zone
- MS-411 Fix metric unittest linking error
- MS-412 Fix gpu cache logical error
- MS-416 ExecutionEngineImpl::GpuCache has not return value cause crash
- MS-417 YAML sequence load disable cause scheduler startup failed
- MS-413 Create index failed and server exited
- MS-427 Describe index error after drop index
- MS-432 Search vectors params nprobe need to check max number
- MS-431 Search vectors params nprobe: 0/-1, expected result: raise exception
- MS-331 Crate Table : when table exists, error code is META_FAILED(code=15) rather than ILLEGAL TABLE NAME(code=9))
- MS-430 Search no result if index created with FLAT
- MS-443 Create index hang again
- MS-436 Delete vectors failed if index created with index_type: IVF_FLAT/IVF_SQ8
- MS-449 Add vectors twice success, once with ids, the other no ids
- MS-450 server hang after run stop_server.sh
- MS-458 Keep building index for one file when no gpu resource
- MS-461 Mysql meta unittest failed
- MS-462 Run milvus server twices, should display error
- MS-463 Search timeout
- MS-467 mysql db test failed
- MS-470 Drop index success, which table not created
- MS-471 code coverage run failed
- MS-492 Drop index failed if index have been created with index_type: FLAT
- MS-493 Knowhere unittest crash
- MS-453 GPU search error when nprobe set more than 1024
- MS-474 Create index hang if use branch-0.3.1 server config
- MS-510 unittest out of memory and crashed
- MS-507 Dataset 10m-512, index type sq8performance in-normal when set CPU_CACHE to 16 or 64
- MS-543 SearchTask fail without exception
- MS-582 grafana displays changes frequently
## Improvement
- MS-327 Clean code for milvus
- MS-336 Scheduler interface
- MS-344 Add TaskTable Test
- MS-345 Add Node Test
- MS-346 Add some implementation of scheduler to solve compile error
- MS-348 Add ResourceFactory Test
- MS-350 Remove knowhere submodule
- MS-354 Add task class and interface in scheduler
- MS-355 Add copy interface in ExcutionEngine
- MS-357 Add minimum schedule function
- MS-359 Add cost test in new scheduler
- MS-361 Add event in resource
- MS-364 Modify tasktableitem in tasktable
- MS-365 Use tasktableitemptr instead in event
- MS-366 Implement TaskTable
- MS-368 Implement cost.cpp
- MS-371 Add TaskTableUpdatedEvent
- MS-373 Add resource test
- MS-374 Add action definition
- MS-375 Add Dump implementation for Event
- MS-376 Add loader and executor enable flag in Resource avoid diskresource execute task
- MS-377 Improve process thread trigger in ResourceMgr, Scheduler and TaskTable
- MS-378 Debug and Update normal_test in scheduler unittest
- MS-379 Add Dump implementation in Resource
- MS-380 Update resource loader and executor, work util all finished
- MS-383 Modify condition variable usage in scheduler
- MS-384 Add global instance of ResourceMgr and Scheduler
- MS-389 Add clone interface in Task
- MS-390 Update resource construct function
- MS-391 Add PushTaskToNeighbourHasExecutor action
- MS-394 Update scheduler unittest
- MS-400 Add timestamp record in task state change function
- MS-402 Add dump implementation for TaskTableItem
- MS-406 Add table flag for meta
- MS-403 Add GpuCacheMgr
- MS-404 Release index after search task done avoid memory increment continues
- MS-405 Add delete task support
- MS-407 Reconstruct MetricsCollector
- MS-408 Add device_id in resource construct function
- MS-409 Using new scheduler
- MS-413 Remove thrift dependency
- MS-410 Add resource config comment
- MS-414 Add TaskType in Scheduler::Task
- MS-415 Add command tasktable to dump all tasktables
- MS-418 Update server_config.template file, set CPU compute only default
- MS-419 Move index_file_size from IndexParam to TableSchema
- MS-421 Add TaskLabel in scheduler
- MS-422 Support DeleteTask in Multi-GpuResource case
- MS-428 Add PushTaskByDataLocality in scheduler
- MS-440 Add DumpTaskTables in sdk
- MS-442 Merge Knowhere
- MS-445 Rename CopyCompleted to LoadCompleted
- MS-451 Update server_config.template file, set GPU compute default
- MS-455 Distribute tasks by minimal cost in scheduler
- MS-460 Put transport speed as weight when choosing neighbour to execute task
- MS-459 Add cache for pick function in tasktable
- MS-476 Improve search performance
- MS-482 Change search stream transport to unary in grpc
- MS-487 Define metric type in CreateTable
- MS-488 Improve code format in scheduler
- MS-495 cmake: integrated knowhere
- MS-496 Change the top_k limitation from 1024 to 2048
- MS-502 Update tasktable_test in scheduler
- MS-504 Update node_test in scheduler
- MS-505 Install core unit test and add to coverage
- MS-508 Update normal_test in scheduler
- MS-532 Add grpc server unittest
- MS-511 Update resource_test in scheduler
- MS-517 Update resource_mgr_test in scheduler
- MS-518 Add schedinst_test in scheduler
- MS-519 Add event_test in scheduler
- MS-520 Update resource_test in scheduler
- MS-524 Add some unittest in event_test and resource_test
- MS-525 Disable parallel reduce in SearchTask
- MS-527 Update scheduler_test and enable it
- MS-528 Hide some config used future
- MS-530 Add unittest for SearchTask->Load
- MS-531 Disable next version code
- MS-533 Update resource_test to cover dump function
- MS-523 Config file validation
- MS-539 Remove old task code
- MS-546 Add simple mode resource_config
- MS-570 Add prometheus docker-compose file
- MS-576 Scheduler refactor
- MS-592 Change showtables stream transport to unary
## Feature
- MS-343 Implement ResourceMgr
- MS-338 NewAPI: refine code to support CreateIndex
- MS-339 NewAPI: refine code to support DropIndex
- MS-340 NewAPI: implement DescribeIndex
## Task
- MS-297 disable mysql unit test
# Milvus 0.3.1 (2019-07-10)
## Bug
- MS-148 Disable cleanup if mode is read only
- MS-149 Fixed searching only one index file issue in distributed mode
- MS-153 Fix c_str error when connecting to MySQL
- MS-157 Fix changelog
- MS-190 Use env variable to switch mem manager and fix cmake
- MS-217 Fix SQ8 row count bug
- MS-224 Return AlreadyExist status in MySQLMetaImpl::CreateTable if table already exists
- MS-232 Add MySQLMetaImpl::UpdateTableFilesToIndex and set maximum_memory to default if config value = 0
- MS-233 Remove mem manager log
- MS-230 Change parameter name: Maximum_memory to insert_buffer_size
- MS-234 Some case cause background merge thread stop
- MS-235 Some test cases random fail
- MS-236 Add MySQLMetaImpl::HasNonIndexFiles
- MS-257 Update bzip2 download url
- MS-288 Update compile scripts
- MS-330 Stability test failed caused by server core dumped
- MS-347 Build index hangs again
- MS-382 fix MySQLMetaImpl::CleanUpFilesWithTTL unknown column bug
## Improvement
- MS-156 Add unittest for merge result functions
- MS-152 Delete assert in MySQLMetaImpl and change MySQLConnectionPool impl
- MS-204 Support multi db_path
- MS-206 Support SQ8 index type
- MS-208 Add buildinde interface for C++ SDK
- MS-212 Support Inner product metric type
- MS-241 Build Faiss with MKL if using Intel CPU; else build with OpenBlas
- MS-242 Clean up cmake and change MAKE_BUILD_ARGS to be user defined variable
- MS-245 Improve search result transfer performance
- MS-248 Support AddVector/SearchVector profiling
- MS-256 Add more cache config
- MS-260 Refine log
- MS-249 Check machine hardware during initialize
- MS-261 Update faiss version to 1.5.3 and add BUILD_FAISS_WITH_MKL as an option
- MS-266 Improve topk reduce time by using multi-threads
- MS-275 Avoid sqlite logic error excetion
- MS-278 add IndexStatsHelper
- MS-313 add GRPC
- MS-325 add grpc status return for C++ sdk and modify some format
- MS-278 Add IndexStatsHelper
- MS-312 Set openmp thread number by config
- MS-305 Add CPU core percent metric
- MS-310 Add milvus CPU utilization ratio and CPU/GPU temperature metrics
- MS-324 Show error when there is not enough gpu memory to build index
- MS-328 Check metric type on server start
- MS-332 Set grpc and thrift server run concurrently
- MS-352 Add hybrid index
## Feature
- MS-180 Add new mem manager
- MS-195 Add nlist and use_blas_threshold conf
- MS-137 Integrate knowhere
## Task
- MS-125 Create 0.3.1 release branch
- MS-306 Optimize build efficiency
# Milvus 0.3.0 (2019-06-30)
## Bug
- MS-104 Fix unittest lcov execution error
- MS-102 Fix build script file condition error
- MS-80 Fix server hang issue
- MS-89 Fix compile failed, libgpufaiss.a link missing
- MS-90 Fix arch match incorrect on ARM
- MS-99 Fix compilation bug
- MS-110 Avoid huge file size
## Improvement
- MS-82 Update server startup welcome message
- MS-83 Update vecwise to Milvus
- MS-77 Performance issue of post-search action
- MS-22 Enhancement for MemVector size control
- MS-92 Unify behavior of debug and release build
- MS-98 Install all unit test to installation directory
- MS-115 Change is_startup of metric_config switch from true to on
- MS-122 Archive criteria config
- MS-124 HasTable interface
- MS-126 Add more error code
- MS-128 Change default db path
## Feature
- MS-57 Implement index load/search pipeline
- MS-56 Add version information when server is started
- MS-64 Different table can have different index type
- MS-52 Return search score
- MS-66 Support time range query
- MS-68 Remove rocksdb from third-party
- MS-70 cmake: remove redundant libs in src
- MS-71 cmake: fix faiss dependency
- MS-72 cmake: change prometheus source to git
- MS-73 cmake: delete civetweb
- MS-65 Implement GetTableRowCount interface
- MS-45 Implement DeleteTable interface
- MS-75 cmake: change faiss version to 1.5.2; add CUDA gencode
- MS-81 fix faiss ptx issue; change cuda gencode
- MS-84 cmake: add arrow, jemalloc and jsoncons third party; default build option OFF
- MS-85 add NetIO metric
- MS-96 add new query interface for specified files
- MS-97 Add S3 SDK for MinIO Storage
- MS-105 Add MySQL
- MS-130 Add prometheus_test
- MS-144 Add nprobe config
- MS-147 Enable IVF
- MS-130 Add prometheus_test
## Task
- MS-74 Change README.md in cpp
- MS-88 Add support for arm architecture
# Milvus 0.2.0 (2019-05-31)
## Bug
- MS-32 Fix thrift error
- MS-34 Fix prometheus-cpp thirdparty
- MS-67 Fix license check bug
- MS-76 Fix pipeline crash bug
- MS-100 cmake: fix AWS build issue
- MS-101 change AWS build type to Release
## Improvement
- MS-20 Clean Code Part 1
## Feature
- MS-5 Implement Auto Archive Feature
- MS-6 Implement SDK interface part 1
- MS-16 Implement metrics without prometheus
- MS-21 Implement SDK interface part 2
- MS-26 cmake. Add thirdparty packages
- MS-31 cmake: add prometheus
- MS-33 cmake: add -j4 to make third party packages build faster
- MS-27 support gpu config and disable license build config in cmake
- MS-47 Add query vps metrics
- MS-37 Add query, cache usage, disk write speed and file data size metrics
- MS-30 Use faiss v1.5.2
- MS-54 cmake: Change Thrift third party URL to github.com
- MS-69 prometheus: add all proposed metrics
## Task
- MS-1 Add CHANGELOG.md
- MS-4 Refactor the vecwise_engine code structure
- MS-62 Search range to all if no date specified