bleve

Bug Fixes

• Fixes to spanish accents normalization (https://github.com/blevesearch/bleve/pull/1957)
• Critical fix around facets merging which could've caused inconsistent results (https://github.com/blevesearch/bleve/pull/1946)

Improvements

• Support for vectors indexing and search
  • Please read: vectors.md for more details
  • With this we will now default to a new zap format: v16, which co-locates text and vector indexes within scorch segments
• An empty query attribute within a SearchRequest will now be interpreted as a match_none (https://github.com/blevesearch/bleve/pull/2000)

Milestone

• v2.4.0

Bug Fixes

• Fix format of date times displayed in search result, to not drop precision (https://github.com/blevesearch/bleve/pull/1868)

Improvements

• Upgraded to golang.org/x/[email protected] to overcome a vulnerability (https://github.com/blevesearch/bleve/pull/1845)
• Support for polish language analyzer - pl (https://github.com/blevesearch/bleve/pull/1825)
• Additional attribute in search response to determine partial/complete hits for match queries (https://github.com/blevesearch/bleve/pull/1848)
• Performance optimization to speed up documents processing per mapping definition (https://github.com/blevesearch/bleve/pull/1865)
• Support fuzziness factor for matchphrase, multiphrase & phrase queries (https://github.com/blevesearch/bleve/pull/1847)
• Geo shape polygon decode optimization w/ buffer re-use (https://github.com/blevesearch/bleve/pull/1864)
• Override-able Marshal/Unmarshal methods (https://github.com/blevesearch/bleve/pull/1880)
• Enhanced data time format support:
  • sanitizedgo that validates layout against supported golang constants (https://github.com/blevesearch/bleve/pull/1877)
  • Ability to apply custom datetime parsers to query criteria and facets (https://github.com/blevesearch/bleve/pull/1860, https://github.com/blevesearch/bleve/pull/1878)
  • Ability to parse date times in these formats - unix_micro, unix_sec, unix_milli, unix_nano (https://github.com/blevesearch/bleve/pull/1870)
  • Additional layouts support with percentstyle and isostyle (https://github.com/blevesearch/bleve/pull/1866)

Milestone

• v2.3.10

Bug Fixes

• Fixes along recording IO bytes read for different query types (https://github.com/blevesearch/bleve/pull/1829)
  • Also changes the attribute name in the SearchResult construct from bytesRead to cost
• Fix incorrect facet counts when facetted queries are coupled with sorting over the same field (https://github.com/blevesearch/bleve/pull/1836)

Improvements

• Upgrade to RoaringBitmap/roaring@v1.2.3 (https://github.com/blevesearch/bleve/pull/1822)
• Upgrade to etcd-io/bblot@v1.3.7 (https://github.com/blevesearch/bleve/pull/1824)
• Consolidated some redundant work in a hot index path with https://github.com/blevesearch/bleve/pull/1832 which showed index time speed up by about 30% in certain situations especially while working with large documents.
• Upgrade to vellum@v1.0.10 which contains a minor optimization (https://github.com/blevesearch/bleve/pull/1838)

Milestone

• v2.3.9

Bug Fixes

• Addresses panics that could occur within zap (with [email protected])
  • https://github.com/blevesearch/zapx/pull/105
  • https://github.com/blevesearch/zapx/pull/150
  • https://github.com/blevesearch/zapx/pull/151
• Fixes analyzer inheritance for default mapping with https://github.com/blevesearch/bleve/pull/1807
• Addresses https://github.com/blevesearch/bleve/issues/1816 by fixing CLI's completion generation with https://github.com/blevesearch/bleve/pull/1817

Enhancements

• Introduces a new not-so-aggressive plural stemmer (stemmer_en_plural) for english with https://github.com/blevesearch/bleve/pull/1808

Milestone

• v2.3.8

Bug Fixes

• Replaced deprecated io/ioutil functions with https://github.com/blevesearch/bleve/pull/1781
• Fixed situation where bytesRead will not appear in search response if query gets everything it needs from mmaped content with https://github.com/blevesearch/bleve/pull/1788
• Fixed a panic due to misalignment within chunkedContentCoder on 32-bit platforms with https://github.com/blevesearch/zapx/pull/148 and https://github.com/blevesearch/bleve/pull/1791 (only applicable to zap@v15)
• A bot upgraded the golang.org/x/text version we use from 0.3.7 to 0.3.8 with https://github.com/blevesearch/bleve/pull/1792 🥇
• Include bytesWritten into bbolt (metadata for scorch) with https://github.com/blevesearch/bleve/pull/1774

Enhancements

• Support for improved rollback behavior by allowing sampling of snapshots based on a time interval with https://github.com/blevesearch/bleve/pull/1776 . Default behavior remains the same
• Changed the signature of scorch's AsyncErrorCallback with https://github.com/blevesearch/bleve/pull/1800 to provide more context. This is a breaking change.

Milestone

• v2.3.7

Bug Fixes

• More efficient alignment for chunkedContentCoder (https://github.com/blevesearch/zapx/pull/147)
• Addressed issue in indexing unigrams before punctuations in Japanese (cjk) (https://github.com/blevesearch/bleve/pull/1724)
• Addressed facets string output limitation for date ranges and numeric ranges reported by https://github.com/blevesearch/bleve/issues/1763 with https://github.com/blevesearch/bleve/pull/1764
• Fixed inaccurate IO stats accounting while loading segment metadata (https://github.com/blevesearch/bleve/pull/1769)

Enhancements

• New bytesRead attribute in search response (https://github.com/blevesearch/bleve/pull/1752, https://github.com/blevesearch/zapx/pull/142)
  • Applicable to zapx v15 only
• New tests showcasing performance benchmarks for popular queries (https://github.com/blevesearch/bleve/pull/1755)

Milestone

• v2.3.6

Bug Fixes

• Fixed throughput regression reported with https://github.com/blevesearch/bleve/issues/1731
  • https://github.com/blevesearch/bleve/pull/1738
  • https://github.com/blevesearch/bleve_index_api/pull/30
  • https://github.com/blevesearch/scorch_segment_api/pull/20
• Fix intersection logic for geo JSON searches, more unit tests
  • https://github.com/blevesearch/geo/pull/10
  • https://github.com/blevesearch/bleve/pull/1720
• Fix to support IP Range query
  • https://github.com/blevesearch/bleve/pull/1735

Other

• Breaking change in defining custom analyzers
  • https://github.com/blevesearch/bleve/pull/1729 (this is needed to allow custom analyzer definition without needing to always define CharFilters, Tokenizer, TokenFilters)
• Update spf13/cobra version v1.4.0 (to drop a number of indirect dependencies)
  • https://github.com/blevesearch/bleve/commit/aba7d6c0aa1bf36b2c30426e4c4f871f1a073dd1

Milestone

• v2.3.5

Enhancements

• Support store option for the geoshape field (https://github.com/blevesearch/bleve/pull/1714)
• Publishing read/write stats based on usage (https://github.com/blevesearch/bleve/pull/1702 , https://github.com/blevesearch/bleve/pull/1710)

Enhancements

• Introduce geo JSON spatial capability (https://s2geometry.io/)
  • More on this: https://github.com/blevesearch/bleve/blob/v2.3.3/geo/README.md

Bug Fixes

• Fix for https://github.com/blevesearch/bleve/issues/1681

Minor changes

• Adding configurable default threshold for field TFR cache (https://github.com/blevesearch/bleve/pull/1666)
• Forked certain third party dependencies (https://github.com/blevesearch/bleve/pull/1667)

Bug Fixes

• Fix for potential file handle leaks in merger. (https://github.com/blevesearch/bleve/pull/1652)
• Place a nil guard within TermFacets' Terms() API (https://github.com/blevesearch/bleve/pull/1654)
• Upgrade zapx versions (https://github.com/blevesearch/bleve/pull/1655)
  • Place bounds check within memUvarintReader's ReadUvarint (https://github.com/blevesearch/zapx/pull/107)

Enhancements

• Upgrade to golang.org/x/text to v0.3.7 (https://github.com/blevesearch/bleve/pull/1645)
• Optimize FacetsBuilder's UpdateVisitor - for some significant performance gains (https://github.com/blevesearch/bleve/pull/1405) (changes API)
• Optimize TermFacets - for significant performance gains (https://github.com/blevesearch/bleve/pull/1404) (changes API)
• Introduce support for a new document field type: IP that supports range queries (https://github.com/blevesearch/bleve/pull/1546)

Bug Fixes

• Fix breakage in highlighting when using the HTML character filter (https://github.com/blevesearch/bleve/pull/1641)
• Fix issue in parsing query strings over numeric data with boost settings (https://github.com/blevesearch/bleve/pull/1639)
• Address seg faults seen within zap; zap versions upgrade (https://github.com/blevesearch/zapx/pull/95, https://github.com/blevesearch/zapx/pull/96)
• Fix out-of-bounds issue while highlighting without term locations (https://github.com/blevesearch/bleve/pull/1590)

Enhancements

• Adding support for croatian (hr) (https://github.com/blevesearch/bleve/pull/1517)
  • Fixes to import paths (https://github.com/blevesearch/bleve/pull/1634)

Bug Fixes

• Fix issue in previous release in the tag used for blevesearch/vellum (https://github.com/blevesearch/bleve/pull/1630)

Enhancements

• Upgrade to RoaringBitMap/[email protected] (https://github.com/blevesearch/bleve/pull/1626)
  • Involves upgrading zapx, scorch_segment_api, vellum versions

Bug Fixes

• Fix issue in read-only mode of scorch index (https://github.com/blevesearch/bleve/pull/1624)

Enhancements

• Introducing a new hierarchy token filter (https://github.com/blevesearch/bleve/pull/1570)

Bug Fixes

• Update version of roaring used by dependency: scorch_segment_api to match bleve (https://github.com/blevesearch/bleve/pull/1622)
• Fix to firing error on panic in scorch async routines (https://github.com/blevesearch/bleve/pull/1566)

Enhancements

• Supporting backup (copy) of an online scorch index (https://github.com/blevesearch/bleve/pull/1605)

Bug Fixes

• Address a panic in the merger loop (https://github.com/blevesearch/bleve/pull/1613)

Bug Fixes

• Fix to HighTerm while sorting "missing" values (https://github.com/blevesearch/bleve/pull/1608)
• Fix to handling empty field name (https://github.com/blevesearch/bleve/pull/1594)

This release hopefully completes the cleanup of v2.x for issues caused by one of our dependencies moving a repository.

• Roaring Bitmaps updated to v0.7.3
• Update to vellum v1.0.5
• Update to zapx v15.2.1 v14.2.1 v13.2.1 v12.2.1 v11.2.1

Things brings all the repositories to the same versions of Roaring bitmaps and bitset libraries.

Emergency Release

Update to latest Roaring bitmaps (v0.7.1). This release is being made in an attempt to fix a completely broken repository. This has NOT had the usual testing, and users with backwards compatibility concerns should NOT upgrade at this time.

We will update this message as we learn more.