Bot releases are hidden (Show)

TileDB - TileDB 2.15.3

Published by ihnorton over 1 year ago

TileDB v2.15.3 Release Notes

Improvements

Enable support for estimating result size on nullable, remote arrays. #4079

Defects removed

Adds serialization for webp filter options. #4085
Serialize coalesce ranges for Subarray. #4043

TileDB - TileDB v2.15.2

Published by ihnorton over 1 year ago

TileDB v2.15.2 Release Notes

Defects removed

Allow empty AWS credentials for anonymous access. #4064
Fix Query Conditions for Boolean Attributes. #4046

Build

Set Curl CA path on Darwin #4059
Fix invalid CMake syntax due to empty variable #4026

TileDB - TileDB v2.15.1

Published by ihnorton over 1 year ago

TileDB v2.15.1 Release Notes

New features

Introduce "vfs.s3.no_sign_request" to allow unsigned s3 API calls, useful for anonymous s3 bucket access. #3953

Improvements

Deprecations

Defects removed

Fix support for empty strings for Dictionary and RLE encodings #3938
Sparse global order reader: fix read progress update for duplicates. #3937
Sparse unordered w/ dups: fix error on double var size overflow. #3963
Dense reader: fix copies for schema evolution. #3970
Sparse unordered w/ dups: overflow fix shouldn't include empty tile. #3985

API changes

C++ API

Add a Group constructor that accepts a Config in the C++ API. #4011

TileDB - TileDB 2.11.4

Published by ihnorton over 1 year ago

TileDB v2.11.4 Release Notes

Packaging Notes

The linux release packages are now build with manylinux2014 (previously manylinux2010).

Defects removed

Sparse global order reader: fixing incomplete reason for rest queries. #3620
Sparse global order reader: fix tile cleanup when ending an iteration. #3674
Sparse unordered w/ dups: fix error on double var size overflow. #3963
Sparse unordered w/ dups: overflow fix shouldn't include empty tile. #3985
Add missing query_type in array_open capnp #3616

TileDB - TileDB 2.14.2

Published by Shelnutt2 over 1 year ago

TileDB v2.14.2 Release Notes

Improvements

RLE and dictionary filter only enabled for UTF8 since format version 17. #3868
Fragment consolidation: using correct buffer weights. #3877
Sparse global order reader: fix read progress update for duplicates. #3937
Sparse unordered w/ dups: fix error on double var size overflow. #3963

Bug fixes

Deregister a remote array from the Consistency multimap before reopening. #3859
Fix buffer size error in deserializing query #3851
Fix support for empty strings for Dictionary and RLE encodings #3938

TileDB - TileDB v2.15.0

Published by ihnorton over 1 year ago

TileDB v2.15.0 Release Notes

Packaging Notes

The TileDB homebrew package will be archived after TileDB 2.15. Please use the release binaries instead.

Disk Format

Add FloatScaleFilter to format spec (documentation only; filter released in 2.11.0) #3494

Breaking C API changes

Remove deprecated Query methods. #3841
Remove deprecated API functions about buffers #3733
Remove deprecated C API tiledb_fragment_info_load_with_key #3740
Remove extra experimental set buffer methods #3761
Remove C API functions tiledb_array_open_at and variations. #3755
Remove C API functions tiledb_array_consolidate_metadata and tiledb_array_consolidate_metadata_with_key #3742
Remove deprecated C API function tiledb_coords. #3743
Remove bitsort filter (feature-flagged filter). #3852

Breaking behavior

An error is now raised when setting a subarray on a query that is already initialized. #3668
An error is now raised when setting a subarray on a write to a sparse array. #3668

New features

Partial attribute writes into a single fragment. #3714
Query condition support for TIME types #3784
Enable dimension labels for the C API (provisional) #3824
Add dimension label C++ API (provisional) #3839

Improvements

Add creation of ArrayDirectory on open to stats #3769
Return underlying errors in Win::remove_dir #3866
Removes tile-aligned restriction for remote global order writes by caching tile overflow data from submissions. #3762
Implement S3 buffering support for remote global order writes #3609
Add support for dimension labels on an encrypted array #3774
Refactor relevant fragments into a separate class. #3738
Implement duration instrument for measuring times in stats. #3746
Move attribute order check to writer base #3748
Fragment consolidator: use average cell size for buffer allocation. #3756
Show URL in logger trace before results are returned #3745
Refactor tiledb_set_subarray call in test/support/src/helpers.cc #3776
Move array open methods from StorageManager to Array #3790
Split Tile class into different classes for read and write. #3796
Update ArrayDirectory to use ContextResources #3800
Use EncryptionType::NO_ENCRYPTION instead of casting the C enum. #3545
Tile metadata generator: fix buffer overflow on string comparison. #3821
Ordered writer: process next tile batch while waiting on write. #3797
Query v3: Reduce array open operations on Cloud #3626
Read tiles: get rid of extra VFS allocation. #3848
Clean-up QueryBuffer and move it away from statuses. #3840
Storage manager: remove passthrough read/write functions. #3853
Fix partial attribute write test. #3862
Query v2: Add array directory and fragment meta ser/deser behind config flag #3845
Ordered dimension label reader: handle empty array. #3869
Fix comments in nullable attributes example. #3870
RLE and dictionary filter only enabled for UTF8 since format version 17. #3868
Move more resources into ContextResources #3807
Better errors for failed dimension label queries. #3872
Dense reader: process smaller units of work. #3856
Use different config variable for enabling open v2 and query v3 #3879
Fragment consolidation: using correct buffer weights. #3877
Show expected and actual version numbers in the error message #3855
Fix shallow test-only bug in unit_rtree.cc #3830
Fix stack buffer overflow bug in unit-curl.cc #3832
Fix stack use after free bug in unit_thread_pool #3831

Defects removed

Support in place update of group members. #3928
Fail if fragment info objects are accessed before loading them. #3846
Let ConsolidationPlan::dump() produce valid JSON #3751
Fix exceptions thrown during array schema validation for WebP filter. #3752
Modify GlobalStats::reset() to support compensation for registered stats being effectively leaked. #3723
Avoid pwrite bug on macOS Ventura 13.0 on Apple M1 #3799
Fix ASAN-detected UAF in sparse global order reader #3822
Improve WebP validation, fixes SC-24766, SC-24759 #3819
Deregister a remote array from the Consistency multimap before reopening. #3859
Fix buffer size error in deserializing query #3851
Fix segfault in unit_consistency #3880
Avoid segfault in unit_array #3883
VFS CAPIHandle class #3523

API changes

C API

Deprecate tiledb_query_submit_async #3802
Deprecate tiledb_fragment_info_get_name. #3791
Deprecate tiledb_query_submit_async. #3802
Add tiledb_array_delete_fragments_list API #3798
Add tiledb_string_handle_t so that C API functions can output strings with independent lifespans. #3792
Add tiledb_dimension_label_handle_t and tiledb_dimension_label_t handles to C-API #3820
Add tiledb_subarray_has_label_ranges and tiledb_subarray_get_label_name. #3858
Add tiledb_fragment_info_get_fragment_name_v2. #3842
Add function to access the dimension label attribute name. #3867
Change query set buffer methods to set label buffers in experimental builds #3761

C++ API

Move deprecated constructors into new array_deprecated.h file, manipulating them to use the new TemporalPolicy and EncryptionAlgorithm classes. #3854
Add function to access the dimension label attribute name. #3867
Add experimental set_data_buffer API that handles dimension labels #3882

Build system changes

Set ASAN options for TileDB #3843
Allow building TileDB with GNU GCC on MacOS #3779

Experimental features

Set of three exploratory but fully consistent schedulers for TileDB task graph library. #3683
Version 0.1 of specification task graph. #3754

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.14.0...2.15.0

TileDB - TileDB 2.15.0-rc2

Published by ihnorton over 1 year ago

TileDB v2.15.0 Release Notes

Disk Format

Add FloatScaleFilter to format spec (documentation only; filter released in 2.11.0) #3494

Breaking C API changes

Remove deprecated Query methods. #3841
Remove deprecated API functions about buffers #3733
Remove deprecated C API tiledb_fragment_info_load_with_key #3740
Remove extra experimental set buffer methods #3761
Remove C API functions tiledb_array_open_at and variations. #3755
Remove C API functions tiledb_array_consolidate_metadata and tiledb_array_consolidate_metadata_with_key #3742
Remove deprecated C API function tiledb_coords. #3743
Remove bitsort filter (feature-flagged filter). #3852

Breaking behavior

An error is now raised when setting a subarray on a query that is already initialized. #3668
An error is now raised when setting a subarray on a write to a sparse array. #3668

New features

Partial attribute writes into a single fragment. #3714
Query condition support for TIME types #3784
Enable dimension labels for the C API (provisional) #3824
Add dimension label C++ API (provisional) #3839

Improvements

Add creation of ArrayDirectory on open to stats #3769
Return underlying errors in Win::remove_dir #3866
Removes tile-aligned restriction for remote global order writes by caching tile overflow data from submissions. #3762
Implement S3 buffering support for remote global order writes #3609
Add support for dimension labels on an encrypted array #3774
Refactor relevant fragments into a separate class. #3738
Implement duration instrument for measuring times in stats. #3746
Move attribute order check to writer base #3748
Fragment consolidator: use average cell size for buffer allocation. #3756
Show URL in logger trace before results are returned #3745
Refactor tiledb_set_subarray call in test/support/src/helpers.cc #3776
Move array open methods from StorageManager to Array #3790
Split Tile class into different classes for read and write. #3796
Update ArrayDirectory to use ContextResources #3800
Use EncryptionType::NO_ENCRYPTION instead of casting the C enum. #3545
Tile metadata generator: fix buffer overflow on string comparison. #3821
Ordered writer: process next tile batch while waiting on write. #3797
Query v3: Reduce array open operations on Cloud #3626
Read tiles: get rid of extra VFS allocation. #3848
Clean-up QueryBuffer and move it away from statuses. #3840
Storage manager: remove passthrough read/write functions. #3853
Fix partial attribute write test. #3862
Query v2: Add array directory and fragment meta ser/deser behind config flag #3845
Ordered dimension label reader: handle empty array. #3869
Fix comments in nullable attributes example. #3870
RLE and dictionary filter only enabled for UTF8 since format version 17. #3868
Move more resources into ContextResources #3807
Better errors for failed dimension label queries. #3872
Dense reader: process smaller units of work. #3856
Use different config variable for enabling open v2 and query v3 #3879
Fragment consolidation: using correct buffer weights. #3877
Show expected and actual version numbers in the error message #3855
Fix shallow test-only bug in unit_rtree.cc #3830
Fix stack buffer overflow bug in unit-curl.cc #3832
Fix stack use after free bug in unit_thread_pool #3831

Defects removed

Fail if fragment info objects are accessed before loading them. #3846
Let ConsolidationPlan::dump() produce valid JSON #3751
Fix exceptions thrown during array schema validation for WebP filter. #3752
Modify GlobalStats::reset() to support compensation for registered stats being effectively leaked. #3723
Avoid pwrite bug on macOS Ventura 13.0 on Apple M1 #3799
Fix ASAN-detected UAF in sparse global order reader #3822
Improve WebP validation, fixes SC-24766, SC-24759 #3819
Deregister a remote array from the Consistency multimap before reopening. #3859
Fix buffer size error in deserializing query #3851
Fix segfault in unit_consistency #3880
Avoid segfault in unit_array #3883
VFS CAPIHandle class #3523

API changes

C API

Deprecate tiledb_query_submit_async #3802
Deprecate tiledb_fragment_info_get_name. #3791
Deprecate tiledb_query_submit_async. #3802
Add tiledb_array_delete_fragments_list API #3798
Add tiledb_string_handle_t so that C API functions can output strings with independent lifespans. #3792
Add tiledb_dimension_label_handle_t and tiledb_dimension_label_t handles to C-API #3820
Add tiledb_subarray_has_label_ranges and tiledb_subarray_get_label_name. #3858
Add tiledb_fragment_info_get_fragment_name_v2. #3842
Add function to access the dimension label attribute name. #3867
Change query set buffer methods to set label buffers in experimental builds #3761

C++ API

Move deprecated constructors into new array_deprecated.h file, manipulating them to use the new TemporalPolicy and EncryptionAlgorithm classes. #3854
Add function to access the dimension label attribute name. #3867
Add experimental set_data_buffer API that handles dimension labels #3882

Build system changes

Set ASAN options for TileDB #3843
Allow building TileDB with GNU GCC on MacOS #3779

Experimental features

Set of three exploratory but fully consistent schedulers for TileDB task graph library. #3683
Version 0.1 of specification task graph. #3754

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.14.0...2.15.0

TileDB - TileDB 2.15.0-rc1

Published by ihnorton over 1 year ago

TileDB v2.15.0 Release Notes

Disk Format

Add FloatScaleFilter to format spec (documentation only; filter released in 2.11.0) #3494

Breaking C API changes

Remove deprecated Query methods. #3841
Remove deprecated API functions about buffers #3733
Remove deprecated C API tiledb_fragment_info_load_with_key #3740
Remove extra experimental set buffer methods #3761
Remove C API functions tiledb_array_open_at and variations. #3755
Remove C API functions tiledb_array_consolidate_metadata and tiledb_array_consolidate_metadata_with_key #3742
Remove deprecated C API function tiledb_coords. #3743
Remove bitsort filter (feature-flagged filter). #3852

Breaking behavior

An error is now raised when setting a subarray on a query that is already initialized. #3668
An error is now raised when setting a subarray on a write to a sparse array. #3668

New features

Partial attribute writes into a single fragment. #3714
Query condition support for TIME types #3784
Enable dimension labels for the C API (provisional) #3824
Add dimension label C++ API (provisional) #3839

Improvements

Add creation of ArrayDirectory on open to stats #3769
Return underlying errors in Win::remove_dir #3866
Removes tile-aligned restriction for remote global order writes by caching tile overflow data from submissions. #3762
Implement S3 buffering support for remote global order writes #3609
Add support for dimension labels on an encrypted array #3774
Refactor relevant fragments into a separate class. #3738
Implement duration instrument for measuring times in stats. #3746
Move attribute order check to writer base #3748
Fragment consolidator: use average cell size for buffer allocation. #3756
Show URL in logger trace before results are returned #3745
Refactor tiledb_set_subarray call in test/support/src/helpers.cc #3776
Move array open methods from StorageManager to Array #3790
Split Tile class into different classes for read and write. #3796
Update ArrayDirectory to use ContextResources #3800
Use EncryptionType::NO_ENCRYPTION instead of casting the C enum. #3545
Tile metadata generator: fix buffer overflow on string comparison. #3821
Ordered writer: process next tile batch while waiting on write. #3797
Query v3: Reduce array open operations on Cloud #3626
Read tiles: get rid of extra VFS allocation. #3848
Clean-up QueryBuffer and move it away from statuses. #3840
Storage manager: remove passthrough read/write functions. #3853
Fix partial attribute write test. #3862
Query v2: Add array directory and fragment meta ser/deser behind config flag #3845
Ordered dimension label reader: handle empty array. #3869
Fix comments in nullable attributes example. #3870
RLE and dictionary filter only enabled for UTF8 since format version 17. #3868
Move more resources into ContextResources #3807
Better errors for failed dimension label queries. #3872
Dense reader: process smaller units of work. #3856
Use different config variable for enabling open v2 and query v3 #3879
Fragment consolidation: using correct buffer weights. #3877
Show expected and actual version numbers in the error message #3855
Fix shallow test-only bug in unit_rtree.cc #3830
Fix stack buffer overflow bug in unit-curl.cc #3832
Fix stack use after free bug in unit_thread_pool #3831

Defects removed

Fail if fragment info objects are accessed before loading them. #3846
Let ConsolidationPlan::dump() produce valid JSON #3751
Fix exceptions thrown during array schema validation for WebP filter. #3752
Modify GlobalStats::reset() to support compensation for registered stats being effectively leaked. #3723
Avoid pwrite bug on macOS Ventura 13.0 on Apple M1 #3799
Fix ASAN-detected UAF in sparse global order reader #3822
Improve WebP validation, fixes SC-24766, SC-24759 #3819
Deregister a remote array from the Consistency multimap before reopening. #3859
Fix buffer size error in deserializing query #3851
Fix segfault in unit_consistency #3880
Avoid segfault in unit_array #3883
VFS CAPIHandle class #3523

API changes

C API

Deprecate tiledb_query_submit_async #3802
Deprecate tiledb_fragment_info_get_name. #3791
Deprecate tiledb_query_submit_async. #3802
Add tiledb_array_delete_fragments_list API #3798
Add tiledb_string_handle_t so that C API functions can output strings with independent lifespans. #3792
Add tiledb_dimension_label_handle_t and tiledb_dimension_label_t handles to C-API #3820
Add tiledb_subarray_has_label_ranges and tiledb_subarray_get_label_name. #3858
Add tiledb_fragment_info_get_fragment_name_v2. #3842
Add function to access the dimension label attribute name. #3867
Change query set buffer methods to set label buffers in experimental builds #3761

C++ API

Move deprecated constructors into new array_deprecated.h file, manipulating them to use the new TemporalPolicy and EncryptionAlgorithm classes. #3854
Add function to access the dimension label attribute name. #3867
Add experimental set_data_buffer API that handles dimension labels #3882

Build system changes

Set ASAN options for TileDB #3843
Allow building TileDB with GNU GCC on MacOS #3779

Experimental features

Set of three exploratory but fully consistent schedulers for TileDB task graph library. #3683
Version 0.1 of specification task graph. #3754

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.14.0...2.15.0

TileDB - TileDB 2.15.0-rc0

Published by ihnorton over 1 year ago

TileDB v2.15.0 Release Notes

Disk Format

Add FloatScaleFilter to format spec (documentation only; filter released in 2.11.0) #3494

Breaking C API changes

Remove deprecated Query methods. #3841
Remove deprecated API functions about buffers #3733
Remove deprecated C API tiledb_fragment_info_load_with_key #3740
Remove extra experimental set buffer methods #3761
Remove C API functions tiledb_array_open_at and variations. #3755
Remove C API functions tiledb_array_consolidate_metadata and tiledb_array_consolidate_metadata_with_key #3742
Remove deprecated C API function tiledb_coords. #3743
Remove bitsort filter (feature-flagged filter). #3852

Breaking behavior

An error is now raised when setting a subarray on a query that is already initialized. #3668
An error is now raised when setting a subarray on a write to a sparse array. #3668

New features

Partial attribute writes into a single fragment. #3714
Query condition support for TIME types #3784
Enable dimension labels for the C API (provisional) #3824
Add dimension label C++ API (provisional) #3839

Improvements

Add creation of ArrayDirectory on open to stats #3769
Return underlying errors in Win::remove_dir #3866
Removes tile-aligned restriction for remote global order writes by caching tile overflow data from submissions. #3762
Implement S3 buffering support for remote global order writes #3609
Add support for dimension labels on an encrypted array #3774
Refactor relevant fragments into a separate class. #3738
Implement duration instrument for measuring times in stats. #3746
Move attribute order check to writer base #3748
Fragment consolidator: use average cell size for buffer allocation. #3756
Show URL in logger trace before results are returned #3745
Refactor tiledb_set_subarray call in test/support/src/helpers.cc #3776
Move array open methods from StorageManager to Array #3790
Split Tile class into different classes for read and write. #3796
Update ArrayDirectory to use ContextResources #3800
Use EncryptionType::NO_ENCRYPTION instead of casting the C enum. #3545
Tile metadata generator: fix buffer overflow on string comparison. #3821
Ordered writer: process next tile batch while waiting on write. #3797
Query v3: Reduce array open operations on Cloud #3626
Read tiles: get rid of extra VFS allocation. #3848
Clean-up QueryBuffer and move it away from statuses. #3840
Storage manager: remove passthrough read/write functions. #3853
Fix partial attribute write test. #3862
Query v2: Add array directory and fragment meta ser/deser behind config flag #3845
Ordered dimension label reader: handle empty array. #3869
Fix comments in nullable attributes example. #3870
RLE and dictionary filter only enabled for UTF8 since format version 17. #3868
Move more resources into ContextResources #3807
Better errors for failed dimension label queries. #3872
Dense reader: process smaller units of work. #3856
Use different config variable for enabling open v2 and query v3 #3879
Fragment consolidation: using correct buffer weights. #3877
Show expected and actual version numbers in the error message #3855
Fix shallow test-only bug in unit_rtree.cc #3830
Fix stack buffer overflow bug in unit-curl.cc #3832
Fix stack use after free bug in unit_thread_pool #3831

Defects removed

Fail if fragment info objects are accessed before loading them. #3846
Let ConsolidationPlan::dump() produce valid JSON #3751
Fix exceptions thrown during array schema validation for WebP filter. #3752
Modify GlobalStats::reset() to support compensation for registered stats being effectively leaked. #3723
Avoid pwrite bug on macOS Ventura 13.0 on Apple M1 #3799
Fix ASAN-detected UAF in sparse global order reader #3822
Improve WebP validation, fixes SC-24766, SC-24759 #3819
Deregister a remote array from the Consistency multimap before reopening. #3859
Fix buffer size error in deserializing query #3851
Fix segfault in unit_consistency #3880
Avoid segfault in unit_array #3883
VFS CAPIHandle class #3523

API changes

C API

Deprecate tiledb_query_submit_async #3802
Deprecate tiledb_fragment_info_get_name. #3791
Deprecate tiledb_query_submit_async. #3802
Add tiledb_array_delete_fragments_list API #3798
Add tiledb_string_handle_t so that C API functions can output strings with independent lifespans. #3792
Add tiledb_dimension_label_handle_t and tiledb_dimension_label_t handles to C-API #3820
Add tiledb_subarray_has_label_ranges and tiledb_subarray_get_label_name. #3858
Add tiledb_fragment_info_get_fragment_name_v2. #3842
Add function to access the dimension label attribute name. #3867
Change query set buffer methods to set label buffers in experimental builds #3761

C++ API

Move deprecated constructors into new array_deprecated.h file, manipulating them to use the new TemporalPolicy and EncryptionAlgorithm classes. #3854
Add function to access the dimension label attribute name. #3867
Add experimental set_data_buffer API that handles dimension labels #3882

Build system changes

Set ASAN options for TileDB #3843
Allow building TileDB with GNU GCC on MacOS #3779

Experimental features

Set of three exploratory but fully consistent schedulers for TileDB task graph library. #3683
Version 0.1 of specification task graph. #3754

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.14.0...2.15.0

TileDB - TileDB 2.14.1

Published by Shelnutt2 over 1 year ago

TileDB v2.14.1 Release Notes

Defects removed

Tile metadata generator: fix buffer overflow on string comparison. #3821

TileDB - TileDB 2.14.0

Published by ihnorton almost 2 years ago

TileDB v2.14.0 Release Notes

Announcements

TileDB 2.17, targeted for release in April 2023, will require C++20 support for full library compilation.
C++17 compatibility will be maintained in the public C++ API for several releases beyond 2.17.

Disk Format

Format version updated to 17. #3611

Breaking API changes

In TileDB 2.15, we will begin removing C and C++ API functions deprecated through TileDB 2.12. See list of deprecated functions.

Breaking behavior

For global order reads, an error is now raised if multiple ranges are set when the query is submitted. Previously, an error was raised immediately upon adding another range during subarray creation. #3664
For dense writes, an error is now raised if multiple ranges are set when the query is submitted. Previously, an error was raised immediately upon adding another range during subarray creation. #3664
For sparse writes, an error is now raised if the subarray is not default when the query is submitted. Previously, an error was raised when adding ranges to a subarray for a sparse write. #3664
TileDB will not longer override the AWS_DEFAULT_REGION or AWS_REGION environment variables, if set in the process environment. Previously, the library would unconditionally default to setting the region to "us-east-1" unless the vfs.s3.region config variable was specified; now the library will check for the mentioned environment variables, and avoid setting a default if present. The vfs.s3.region config variable will still take precedence if provided. #3788

New features

Add UTF-8 attribute binary comparison support for QueryConditions. #3766
Enable RLE and Dictionary encoding on UTF-8 attrs. #3808
Add Boolean Support For Sparse Query Conditions. #3651
Consolidation plan API. #3647

Improvements

Defects Removed

Fix initialization of include_coords for multipart, sparse, unordered remote queries. #3795
Sparse global order reader: fix tile cleanup when ending an iteration. #3674
Dense array: Tile var size metadata not loaded on read. #3645
Fix parsing of multiple enable= arguments to bootstrap script. #3650
Fix bad write by label error. #3664
Fix build issue with consolidation plan. #3704
Fix macOS versions before 10.15 do not support <filesystem>. #3729
Segfault reading array with query condition after schema evolution. #3732
Query condition: fix when attribute condition is not in user buffers. #3713
Fragment consolidator: stop consolidation if no progress can be made. #3671
Throw error when memory budget leads to splitting to unary range. #3715
Do not resubmit http requests for curl errors for REST requests. #3712

Internal

Validation for scale and offsets input to the float scale filter. #3726
Adding attribute ordering checks to ordered dimension label reader. #3643
Sparse globalorder reader: prevent unused tiles from being loaded again. #3710
Array directory serialization. #3543
Global order writes serialization tests: fixing incorrect section usage. #3670
Update dimension labels to use an array instead of group. #3667
Consolidation plan: implementation for creating a plan using MBRs. #3696
Sparse unordered w/dups reader: allow partial tile offsets loading. #3716
Implement static thread pool scheduler using "throw-catch" state transitions, as well as segmented nodes. #3638
Remove open_arrays_ from StorageManager. #3649
Replaces domain_str with range_str. #3640
Replace use of Buffer with Serializer/Deserializer FragmentMetadata footer methods. #3551
Remove non owning tile constructor. #3661
Add CMake modules for compactly specifying object libraries. #3548
CMake environment unit_test. #3703
Add Attribute data order constructor and checks. #3662
Adding size computation for tile offsets to sparse readers. #3669
Extract CA certificate discovery from GlobalState. #3717
Extract libcurl initialization from GlobalState. #3718
Remove StorageManager::array_close_for_$type methods. #3658
Fix build on upcoming gcc-13. #3722
Adds nightly C++20 builds alongside C++17 nightlies. #3688
Adding average cell size API to array. #3700

API Changes

C API

Deprecate tiledb_array_delete_array add tiledb_array_delete. #3744
Deprecate tiledb_query_submit_async. #3802
Added tiledb_group_delete_group API. #3560
Document tiledb_mime_type_t. #3625
Fix missing argument in set dimension label tile C-API. #3739

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.13.0...2.14.0

TileDB - TileDB 2.13.2

Published by Shelnutt2 almost 2 years ago

TileDB v2.13.2 Release Notes

Bug fixes

Ensures coordinate tiles are initialized for multipart remote queries #3795

TileDB - TileDB 2.14.0-rc0

Published by ihnorton almost 2 years ago

TileDB v2.14.0 Release Notes

Announcements

TileDB 2.17, targeted for release in April 2023, will require C++20 support for full library compilation.
C++17 compatibility will be maintained in the public C++ API for several releases beyond 2.17.

Disk Format

Format version updated to 17. #3611

Breaking API changes

In TileDB 2.15, we will begin removing C and C++ API functions deprecated through TileDB 2.12. See list of deprecated functions.

Breaking behavior

For global order reads, an error is now raised if multiple ranges are set when the query is submitted. Previously, an error was raised immediately upon adding another range during subarray creation. #3664
For dense writes, an error is now raised if multiple ranges are set when the query is submitted. Previously, an error was raised immediately upon adding another range during subarray creation. #3664
For sparse writes, an error is now raised if the subarray is not default when the query is submitted. Previously, an error was raised when adding ranges to a subarray for a sparse write. #3664
TileDB will not longer override the AWS_DEFAULT_REGION or AWS_REGION environment variables, if set in the process environment. Previously, the library would unconditionally default to setting the region to "us-east-1" unless the vfs.s3.region config variable was specified; now the library will check for the mentioned environment variables, and avoid setting a default if present. The vfs.s3.region config variable will still take precedence if provided #3788

New features

Add UTF-8 attribute binary comparison support for QueryConditions. #3766
Add Boolean Support For Sparse Query Conditions. #3651
Consolidation plan API. #3647

Improvements

Defects Removed

Sparse global order reader: fix tile cleanup when ending an iteration. #3674
Dense array: Tile var size metadata not loaded on read. #3645
Fix parsing of multiple enable= arguments to bootstrap script. #3650
Fix bad write by label error. #3664
Fix build issue with consolidation plan. #3704
Fix macOS versions before 10.15 do not support <filesystem>. #3729
Segfault reading array with query condition after schema evolution. #3732
Query condition: fix when attribute condition is not in user buffers. #3713
Fragment consolidator: stop consolidation if no progress can be made. #3671
Throw error when memory budget leads to splitting to unary range. #3715
Do not resubmit http requests for curl errors for REST requests #3712

Internal

Validation for scale and offsets input to the float scale filter. #3726
Adding attribute ordering checks to ordered dimension label reader. #3643
Sparse globalorder reader: prevent unused tiles from being loaded again. #3710
Array directory serialization. #3543
Global order writes serialization tests: fixing incorrect section usage. #3670
Update dimension labels to use an array instead of group. #3667
Consolidation plan: implementation for creating a plan using MBRs. #3696
Sparse unordered w/dups reader: allow partial tile offsets loading. #3716
Implement static thread pool scheduler using "throw-catch" state transitions, as well as segmented nodes. #3638
Remove open_arrays_ from StorageManager. #3649
Replaces domain_str with range_str. #3640
Replace use of Buffer with Serializer/Deserializer FragmentMetadata footer methods. #3551
Remove non owning tile constructor. #3661
Add CMake modules for compactly specifying object libraries. #3548
CMake environment unit_test. #3703
Add Attribute data order constructor and checks. #3662
Adding size computation for tile offsets to sparse readers. #3669
Extract CA certificate discovery from GlobalState. #3717
Extract libcurl initialization from GlobalState. #3718
Remove StorageManager::array_close_for_$type methods. #3658
Fix build on upcoming gcc-13. #3722
Adds nightly C++20 builds alongside C++17 nightlies. #3688
Adding average cell size API to array. #3700

API Changes

C API

Deprecate tiledb_array_delete_array add tiledb_array_delete. #3744
Added tiledb_group_delete_group API. #3560
Document tiledb_mime_type_t. #3625
Fix missing argument in set dimension label tile C-API. #3739

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.13.0...2.14.0

TileDB - TileDB 2.13.1

Published by ihnorton almost 2 years ago

TileDB v2.13.1 Release Notes

Improvements

Sparse global order reader: prevent unused tiles from being loaded again. #3710
Do not resubmit http requests for curl errors for REST requests #3712
Enables WebP by default #3724

Bug fixes

Query condition: fix when attribute condition is not in user buffers. #3713

TileDB - TileDB 2.13.0

Published by ihnorton almost 2 years ago

TileDB v2.13.0 Release Notes

Announcements

We have created a deprecation and removal policy for the C and C++ API. Deprecated functions will be in place for at least two releases, with a removal notification one version prior to the removal.
All functions deprecated through TileDB 2.12 will be removed in TileDB 2.15. See list of deprecated functions here.

Disk Format

Documentation

Document dictionary encoding format #3566
Improve documentation of consolidated commits files and ignore files #3606

New features

Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

API Changes

C API

Added tiledb_status_code #3580
Added tiledb_group_get_is_relative_uri_by_name #3550
Added tiledb_array_delete_array API #3539

Documentation

Document missing fragment info C APIs. #3577
Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

Improvements

Defects removed

Sparse global order reader: fix tile cleanup when ending an iteration. 3674
https://github.com/TileDB-Inc/TileDB/pull/3674) (https://github.com/TileDB-Inc/TileDB/pull/3678)
Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
Dense consolidation: set correct non-empty domain #3635
Sparse global order reader: fixing incomplete reason for rest queries #3620
Fixes dim label reader range order to always take valid ranges #3561
Global order writes should send relative uris over serialization #3557
Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
Fix duplicate logger instantiations in global state #3591
Add missing query_type in array_open capnp #3616
Fix use-after-free on a capnp::FlatArrayMessageReader #3631
Bitsort filter: preallocate filtered buffer for dim tiles. #3632
Add Boolean Support For Sparse Query Conditions #3651

Internal

Read tiles: refactor tile creation code. #3492
Use snprintf to avoid deprecation warnings as errors #3608
Removes non-C.41 range setting (part 1) #3598
Add [[nodiscard]] attribute to Status class and fix failures #3559
Remove WhiteboxTile. #3612
Global order writer: allow splitting fragments. #3605
Move unit uuid to unit tests. #3614
Consolidation: add fragment max size. #3607
Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
Adding rtree object library. #3613
Global order writes serialization: no tests when serialization disabled. #3603
Fragment info serialization support #3530
Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
Class VFS C.41 compliance, Part 1 #3477
Updates: adding strategy. #3513
Remove stale declarations from query #3565
Improvements in C API implementation #3524
Sparse readers: adding relevant cells stats. #3593
Fix find_heap_api_violations.py #3634

Build system changes

Update to catch2 ver3 #3504
Linux release artifacts now target manylinux2014 #3656

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.12.0...2.13.0

TileDB - TileDB 2.13.0-rc3

Published by ihnorton almost 2 years ago

TileDB v2.13.0 Release Notes

Announcements

We have created a deprecation and removal policy for the C and C++ API.. Deprecated functions will be in place for at least two releases, with a removal notification one version prior to the removal.
All functions deprecated through TileDB 2.12 will be removed in TileDB 2.15. See list of deprecated functions here.

Disk Format

Documentation

Document dictionary encoding format #3566
Improve documentation of consolidated commits files and ignore files #3606

New features

Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

API Changes

C API

Added tiledb_status_code #3580
Added tiledb_group_get_is_relative_uri_by_name #3550
Added tiledb_array_delete_array API #3539

Documentation

Document missing fragment info C APIs. #3577
Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

Improvements

Defects removed

Sparse global order reader: fix tile cleanup when ending an iteration. 3674
https://github.com/TileDB-Inc/TileDB/pull/3674) (https://github.com/TileDB-Inc/TileDB/pull/3678)
Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
Dense consolidation: set correct non-empty domain #3635
Sparse global order reader: fixing incomplete reason for rest queries #3620
Fixes dim label reader range order to always take valid ranges #3561
Global order writes should send relative uris over serialization #3557
Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
Fix duplicate logger instantiations in global state #3591
Add missing query_type in array_open capnp #3616
Fix use-after-free on a capnp::FlatArrayMessageReader #3631
Bitsort filter: preallocate filtered buffer for dim tiles. #3632
Add Boolean Support For Sparse Query Conditions #3651

Internal

Read tiles: refactor tile creation code. #3492
Use snprintf to avoid deprecation warnings as errors #3608
Removes non-C.41 range setting (part 1) #3598
Add [[nodiscard]] attribute to Status class and fix failures #3559
Remove WhiteboxTile. #3612
Global order writer: allow splitting fragments. #3605
Move unit uuid to unit tests. #3614
Consolidation: add fragment max size. #3607
Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
Adding rtree object library. #3613
Global order writes serialization: no tests when serialization disabled. #3603
Fragment info serialization support #3530
Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
Class VFS C.41 compliance, Part 1 #3477
Updates: adding strategy. #3513
Remove stale declarations from query #3565
Improvements in C API implementation #3524
Sparse readers: adding relevant cells stats. #3593
Fix find_heap_api_violations.py #3634

Build system changes

Update to catch2 ver3 #3504
Linux release artifacts now target manylinux2014 #3656

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.12.0...2.13.0

TileDB - TileDB 2.12.3

Published by Shelnutt2 almost 2 years ago

TileDB v2.12.3 Release Notes

Improvements

Dense consolidation: set correct non-empty domain. #3635

Bug fixes

Dense array: Tile var size metadata not loaded on read #3645
Sparse global order reader: fix tile cleanup when ending an iteration. #3674

TileDB - TileDB 2.13.0-rc2

Published by ihnorton almost 2 years ago

TileDB v2.13.0 Release Notes

Disk Format

Documentation

Document dictionary encoding format #3566
Improve documentation of consolidated commits files and ignore files #3606

New features

Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

API Changes

C API

Added tiledb_status_code #3580
Added tiledb_group_get_is_relative_uri_by_name #3550
Added tiledb_array_delete_array API #3539

Documentation

Document missing fragment info C APIs. #3577
Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

Improvements

Defects removed

Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
Dense consolidation: set correct non-empty domain #3635
Sparse global order reader: fixing incomplete reason for rest queries #3620
Fixes dim label reader range order to always take valid ranges #3561
Global order writes should send relative uris over serialization #3557
Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
Fix duplicate logger instantiations in global state #3591
Add missing query_type in array_open capnp #3616
Fix use-after-free on a capnp::FlatArrayMessageReader #3631
Bitsort filter: preallocate filtered buffer for dim tiles. #3632
Add Boolean Support For Sparse Query Conditions #3651

Internal

Read tiles: refactor tile creation code. #3492
Use snprintf to avoid deprecation warnings as errors #3608
Removes non-C.41 range setting (part 1) #3598
Add [[nodiscard]] attribute to Status class and fix failures #3559
Remove WhiteboxTile. #3612
Global order writer: allow splitting fragments. #3605
Move unit uuid to unit tests. #3614
Consolidation: add fragment max size. #3607
Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
Adding rtree object library. #3613
Global order writes serialization: no tests when serialization disabled. #3603
Fragment info serialization support #3530
Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
Class VFS C.41 compliance, Part 1 #3477
Updates: adding strategy. #3513
Remove stale declarations from query #3565
Improvements in C API implementation #3524
Sparse readers: adding relevant cells stats. #3593
Fix find_heap_api_violations.py #3634

Build system changes

Update to catch2 ver3 #3504
Linux release artifacts now target manylinux2014 #3656

Full Changelog:

https://github.com/TileDB-Inc/TileDB/compare/2.12.0...2.13.0

TileDB - TileDB 2.13.0-rc1

Published by ihnorton almost 2 years ago

TileDB v2.13.0 Release Notes

Disk Format

Documentation

Document dictionary encoding format #3566
Improve documentation of consolidated commits files and ignore files #3606

New features

Bitsort filter, to write attribute and associated coordinates in the attribute's data order, and re-sort into global order on read #3483, #3570
Support for lossless and lossy RGB(A) and BGR(A) image compression using WebP #3549

API Changes

C API

Added tiledb_status_code #3580
Added tiledb_group_get_is_relative_uri_by_name #3550
Added tiledb_array_delete_array API #3539

Documentation

Document missing fragment info C APIs. #3577
Document that the callback of tiledb_query_submit_async is executed in an internal thread pool thread. #3558

Improvements

Defects removed

Fix for dense arrays: var size metadata not loaded on read [#3645]https://github.com/TileDB-Inc/TileDB/pull/3645)
Dense consolidation: set correct non-empty domain #3635
Sparse global order reader: fixing incomplete reason for rest queries #3620
Fixes dim label reader range order to always take valid ranges #3561
Global order writes should send relative uris over serialization #3557
Fix unit-cppapi-update-queries test failure when throwing UpdateValue exception. #3578
Fix duplicate logger instantiations in global state #3591
Add missing query_type in array_open capnp #3616
Fix use-after-free on a capnp::FlatArrayMessageReader #3631
Bitsort filter: preallocate filtered buffer for dim tiles. #3632

Internal

Read tiles: refactor tile creation code. #3492
Use snprintf to avoid deprecation warnings as errors #3608
Removes non-C.41 range setting (part 1) #3598
Add [[nodiscard]] attribute to Status class and fix failures #3559
Remove WhiteboxTile. #3612
Global order writer: allow splitting fragments. #3605
Move unit uuid to unit tests. #3614
Consolidation: add fragment max size. #3607
Move unit-bytevecvalue, unit-TileDomain and unit-domain to unit test. #3615
Adding rtree object library. #3613
Global order writes serialization: no tests when serialization disabled. #3603
Fragment info serialization support #3530
Array Metadata related refactoring, use Serializer/Deserializer instead of Buffer #3544
Class VFS C.41 compliance, Part 1 #3477
Updates: adding strategy. #3513
Remove stale declarations from query #3565
Improvements in C API implementation #3524
Sparse readers: adding relevant cells stats. #3593
Fix find_heap_api_violations.py #3634