hnswlib

Highlights:

• Multi-vector document search and epsilon search (for now, only in C++)
• By default, there is no statistic aggregation, which speeds up the multi-threaded search (it does not seem like people are using it anyway: Issue #495).
• Various bugfixes and improvements
• get_items now have return_type parameter, which can be either 'numpy' or 'list'

Huge thanks to the contributors:

1. Dmitry Yashunin (dyashuni) - Led the addition of a custom stop condition, multivector and epsilon search functionalities, and various code refinements and bug fixes.
2. Alexander Vieth (alxvth) - Contributed to resolving global linkage issues and improving Mac setup in the CI environment.
3. Taras Tsugrii (ttsugriy) - Focused on fixing build warnings, enhancing bruteforce removePoint functionality, and improving memory management.
4. aurora327 - Implemented more efficient AVX512 instruction sets in functions.
5. Étienne Mollier (emollier) - added a cap on the 'M' parameter and a typo.
6. Johan Rade (jrade) - Addressed a linking error when compiling with Visual Studio.
7. James Melville (jlmelville) - Addressed reordering warning, sign mismatch in loops, and standardized error handling using macros (#508 has the details)
8. stephematician - Resolved initialization order warning in GNU compilers.
9. moritz-h - Added CMake install targets and set CMake version range.
10. drons - Implemented functions for precise memory footprint control and fixed memory leak issues.
11. Atsushi Tatsuma (yoshoku) - Added a missing virtual destructor to BaseFilterFunctor.

version 0.7.0

• Added support to filtering (#402, #430) by @kishorenc
• Added python interface for filtering (though note its performance is limited by GIL) (#417) by @gtsoukas
• Added support for replacing the elements that were marked as delete with newly inserted elements (to control the size of the index, #418) by @dyashuni
• Fixed data races/deadlocks in updates/insertion, added stress test for multithreaded operation (#418) by @dyashuni
• Documentation, tests, exception handling, refactoring (#375, #379, #380, #395, #396, #401, #406, #404, #409, #410, #416, #415, #431, #432, #433) by @jlmelville, @dyashuni, @kishorenc, @korzhenevski, @yoshoku, @jianshu93, @PLNech
• global linkages (#383) by @MasterAler, USE_SSE usage in MSVC (#408) by @alxvth

• Fixed a bug in saving of large pickles. The pickles with > 4GB could have been corrupted. Thanks Kai Wohlfahrt for reporting.
• Thanks to (@GuyAv46) hnswlib inner product now is more consitent accross architectures (SSE, AVX, etc).

Thanks to (@tony-kuo) hnswlib AVX512 and AVX builds are not backwards-compatible with older SSE and non-AVX512 architectures.
Thanks to (@psobot) there is now a sensible message instead of segfault when passing a scalar to get_items.
Thanks to (@urigoren) hnswlib has a lazy index creation python wrapper.

Thanks to (@dyashuni) hnswlib now uses github actions for CI, there is a search speedup in some scenarios with deletions. unmark_deleted(label) is now also a part of the python interface (note now it throws an exception for double deletions).
Thanks to (@slice4e) we now support AVX512; thanks to (@LTLA) the cmake interface for the lib is now updated.
Thanks to (@alonre24) we now have a python bindings for brute-force (and examples for recall tuning: TESTING_RECALL.md.
Thanks to (@dorosy-yeong) there is a bug fixed in the handling large quantities of deleted elements and large K.

Bugfixes and improvements. Many thanks to:
@marekhanus for fixing the missing arguments, adding support for python 3.8, 3.9 in Travis, improving python wrapper and fixing typos/code style;
@apoorv-sharma for fixing the bug int the insertion/deletion logic;
@shengjun1985 for simplifying the memory reallocation logic;
@TakaakiFuruse for improved description of add_items;
@psobot for improving error handling;
@ShuAiii for reporting the bug in the python interface.

Added support for pickling indices, support for PEP-517 and PEP-518 building, small speedups, bugfixes and documentation improvements.
The setup.py file now resides in the root.

Many thanks to @dbespalov, @dyashuni, @groodt, @uestc-lfs, @vinnitu, @fabiencastan, @JinHai-CN, @js1010!

Thanks to Apoorv Sharma @apoorv-sharma, hnswlib now supports true element updates (the interface remained the same, but when you the performance/memory should not degrade as you update the element embeddings).

Thanks to Dmitry @2ooom, hnswlib got a boost in performance for vector dimensions that are not multiple of 4.

Bugfixes and other updates (@xiejianqiao, @Shujian2015, @mohamed-ali, @hussamaa).

Many thanks to all contributors!

Fixed many bugs and error messages.
Added support to marking elements as deleted.
Stopped support for old indices.

A huge thanks to all of the contributors!