delta-rs

What's Changed

• Unique delta object store url by @gruuya in https://github.com/delta-io/delta-rs/pull/1212
• fix: make sure we handle data checking correctly by @wjones127 in https://github.com/delta-io/delta-rs/pull/1222

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.8.0...python-v0.8.1

What's Changed

• Selectively overwrite data with python by @ismoshkov in https://github.com/delta-io/delta-rs/pull/1101
• Python write_deltalake fails if pyarrow table contains binary columns by @rbushri in https://github.com/delta-io/delta-rs/pull/1167
• minor: optimize partition lookup for vacuum loop by @houqp in https://github.com/delta-io/delta-rs/pull/1120
• improve debuggability of json ser/de errors by @houqp in https://github.com/delta-io/delta-rs/pull/1119
• docs(python): update docs by @wjones127 in https://github.com/delta-io/delta-rs/pull/1155
• Set AddAction timestamps to milliseconds. Fixes #1124 by @guyrt in https://github.com/delta-io/delta-rs/pull/1133
• fix: change unexpected field logging level to debug by @houqp in https://github.com/delta-io/delta-rs/pull/1112
• chore: update datafusion by @roeap in https://github.com/delta-io/delta-rs/pull/1114
• build(deps): bump tokio from 1.23.1 to 1.24.2 in /delta-inspect by @dependabot in https://github.com/delta-io/delta-rs/pull/1118
• Make rustls default across all packages by @wjones127 in https://github.com/delta-io/delta-rs/pull/1097
• build(deps): bump openssl-src from 111.22.0+1.1.1q to 111.25.0+1.1.1t in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/1134
• chore: remove star dependencies by @wjones127 in https://github.com/delta-io/delta-rs/pull/1139
• add function & test for parsing table_or_uri by @marijncv in https://github.com/delta-io/delta-rs/pull/1138
• build(deps): update errno requirement from 0.2 to 0.3 by @dependabot in https://github.com/delta-io/delta-rs/pull/1142
• use Path object in writer tests by @marijncv in https://github.com/delta-io/delta-rs/pull/1147
• fix typo by @spebern in https://github.com/delta-io/delta-rs/pull/1166
• add test for extract_partition_values by @marijncv in https://github.com/delta-io/delta-rs/pull/1159
• fix: avoid some allocations in DeltaStorageHandler by @roeap in https://github.com/delta-io/delta-rs/pull/1115
• first setup of ruff for python linting by @marijncv in https://github.com/delta-io/delta-rs/pull/1158
• feat: move and update Optimize operation by @roeap in https://github.com/delta-io/delta-rs/pull/1154
• chore: increment dynamodb_lock version by @wjones127 in https://github.com/delta-io/delta-rs/pull/1202
• feat: extend configuration handling by @roeap in https://github.com/delta-io/delta-rs/pull/1206
• feat: typed commit info by @roeap in https://github.com/delta-io/delta-rs/pull/1207
• build(deps): bump datafusion by @roeap in https://github.com/delta-io/delta-rs/pull/1217

New Contributors

• @guyrt made their first contribution in https://github.com/delta-io/delta-rs/pull/1133
• @rbushri made their first contribution in https://github.com/delta-io/delta-rs/pull/1167

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.7.0...python-v0.8.0

Full Changelog

Implemented enhancements:

• feat(rust): support additional types for partition values #1170

Fixed bugs:

• File pruning does not occur on partition columns #1175
• Bug: Error loading Delta table locally #1157
• Deltalake 0.7.0 with s3 feature compliation error due to rusoto_dynamodb version conflict #1191
• Writing from a Delta table scan using WriteBuilder fails due to missing object store #1186

Merged pull requests:

• build(deps): bump datafusion #1217 (roeap)
• Implement pruning on partition columns #1179 (Blajda)
• feat: enable passing storage options to Delta table builder via Datafusion's CREATE EXTERNAL TABLE #1043 (gruuya)
• feat: typed commit info #1207 (roeap)
• add boolean, date, timestamp & binary partition types #1180 (marijncv)
• feat: extend configuration handling #1206 (marijncv)
• fix: load command for local tables #1205 (roeap)
• Enable passing Datafusion session state to WriteBuilder #1187 (gruuya)
• chore: increment dynamodb_lock version #1202 (wjones127)
• fix: update out-of-date doc about datafusion #1183 (xudong963)
• feat: move and update Optimize operation #1154 (roeap)
• add test for extract_partition_values #1159 (marijncv)
• fix typo #1166 (spebern)
• chore: remove star dependencies #1139 (wjones127)

Full Changelog

Implemented enhancements:

• Support FSCK REPAIR TABLE Operation #1092
• Expose the Delta Log in a DataFrame that's easy for analysis #1031
• Provide case-insensitive storage options in backend #999
• Support local file path in CreateBuilder::with_location() #998
• Save operational params in the same way with delta io #1054 (ismoshkov)

Fixed bugs:

• DeltaTable DataFusion TableProvider does not support filter pushdown #1064
• DeltaTable DataFusion scan does not prune files properly #1063
• deltalake.DeltaTable constructor hangs in Jupyter #1093
• Transaction log JSON formatting issue when writing data via Python bindings #1017
• crates.io entry is missing link to rustdoc documentation #1076
• URL Registered with ObjectStore registry is different from url in DeltaScan #1018
• Not able to connect to Azure Storage with client id/secret #977
• Deltalake 0.5 crate s3 feature dynamodb version mismatch #973
• Overwrite mode does not work with Azure #939
• Use Chrono without default features #914
• cargo test does not run due to tls conflict #985
• Azure SAS authorization fails with <AuthenticationErrorDetail>Signature fields not well formed. #910

Merged pull requests:

• Make rustls default across all packages #1097 (wjones127)
• Implement filesystem check #1103 (Blajda)
• refactor: move vacuum command to operations module #1045 (roeap)
• feat: enable passing storage options to Delta table builder via DataFusion's CREATE EXTERNAL TABLE #1043 (gruuya)
• feat: improve storage location handling #1065 (roeap)
• Fix to support UTC timezone #1022 (andrei-ionescu)
• feat: harmonize and simplify storage configuration #1052 (roeap)
• feat: expose function to get table of add actions #1033 (wjones127)
• fix: change unexpected field logging level to debug #1112 (houqp)
• fix: datafusion predicate pushdown and dependencies #1071 (roeap)
• fix: azure sas key url encoding #1036 (roeap)
• Add provisional workaround to support CDC #1039 #1042 (Fazzani)
• improve debuggability of json ser/de errors #1119 (houqp)
• Add an example of writing to a delta table with a RecordBatch #1085 (rtyler)
• minor: optimize partition lookup for vacuum loop #1120 (houqp)
• Add missing documentation metadata to Cargo.toml #1077 (johnbatty)
• add test for null_count_schema_for_fields #1135 (marijncv)
• add test for min_max_schema_for_fields #1122 (marijncv)
• add test for get_boolean_from_metadata #1121 (marijncv)
• add test for left_larger_than_right #1110 (marijncv)
• Add test for: to_scalar_value #1086 (marijncv)
• Fix typo in delta-inspect #1072 (byteink)
• chore: update datafusion #1114 (roeap)

* This Changelog was automatically generated by github_changelog_generator

What's Changed

• Add the support of the AWS_PROFILE environment variable for S3 by @fvaleye in https://github.com/delta-io/delta-rs/pull/986
• Handle pandas timestamps by @hayesgb in https://github.com/delta-io/delta-rs/pull/958
• fix: get azure client secret from config by @roeap in https://github.com/delta-io/delta-rs/pull/981
• fix truncating signature on SAS by @damiondoesthings in https://github.com/delta-io/delta-rs/pull/1007
• fix: azure sas key url encoding by @roeap in https://github.com/delta-io/delta-rs/pull/1036
• test: add Data Acceptance Tests by @wjones127 in https://github.com/delta-io/delta-rs/pull/909
• test(python): add read / write benchmarks by @wjones127 in https://github.com/delta-io/delta-rs/pull/933
• Add a new release github action for Python binding: macos with universal2 wheel by @fvaleye in https://github.com/delta-io/delta-rs/pull/976
• test(python): add azure integration tests by @wjones127 in https://github.com/delta-io/delta-rs/pull/912
• Loosen version requirement for maturin by @gyscos in https://github.com/delta-io/delta-rs/pull/1005
• Support DataFusion 15 by @andrei-ionescu in https://github.com/delta-io/delta-rs/pull/1021
• Add provisional workaround to support CDC #1039 by @Fazzani in https://github.com/delta-io/delta-rs/pull/1042
• refactor(api!): refactor Python APIs for getting file list by @wjones127 in https://github.com/delta-io/delta-rs/pull/1032
• feat: make DeltaStorageHandler pickle serializable by @roeap in https://github.com/delta-io/delta-rs/pull/1016
• Expose checkpoint creation for current table state in python by @ismoshkov in https://github.com/delta-io/delta-rs/pull/1058
• feat: expose function to get table of add actions by @wjones127 in https://github.com/delta-io/delta-rs/pull/1033
• Save operational params in the same way with delta io by @ismoshkov in https://github.com/delta-io/delta-rs/pull/1054

New Contributors

• @hayesgb made their first contribution in https://github.com/delta-io/delta-rs/pull/958
• @iajoiner made their first contribution in https://github.com/delta-io/delta-rs/pull/1001
• @gyscos made their first contribution in https://github.com/delta-io/delta-rs/pull/1005
• @damiondoesthings made their first contribution in https://github.com/delta-io/delta-rs/pull/1007
• @Fazzani made their first contribution in https://github.com/delta-io/delta-rs/pull/1042
• @gruuya made their first contribution in https://github.com/delta-io/delta-rs/pull/1043
• @ismoshkov made their first contribution in https://github.com/delta-io/delta-rs/pull/1058
• @byteink made their first contribution in https://github.com/delta-io/delta-rs/pull/1072
• @johnbatty made their first contribution in https://github.com/delta-io/delta-rs/pull/1077
• @marijncv made their first contribution in https://github.com/delta-io/delta-rs/pull/1086

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.6.4...python-v0.7.0

What's Changed

• Add a new release github action for Python binding: macos with universal2 wheel by @fvaleye in https://github.com/delta-io/delta-rs/pull/976
• feat: check invariants in write command by @roeap in https://github.com/delta-io/delta-rs/pull/980
• fix: get azure client secret from config by @roeap in https://github.com/delta-io/delta-rs/pull/981
• Update .gitignore and add/remove Cargo.lock when appropriate by @iajoiner in https://github.com/delta-io/delta-rs/pull/1001
• Add the support of the AWS_PROFILE environment variable for S3 by @fvaleye in https://github.com/delta-io/delta-rs/pull/986
• fix truncating signature on SAS by @damiondoesthings in https://github.com/delta-io/delta-rs/pull/1007
• Support DataFusion 15 by @andrei-ionescu in https://github.com/delta-io/delta-rs/pull/1021
• Update versions to 0.6.0 and update changelog by @iajoiner in https://github.com/delta-io/delta-rs/pull/1023
• Add PR autolabeling by @iajoiner in https://github.com/delta-io/delta-rs/pull/1030
• build(deps): bump env_logger from 0.9.3 to 0.10.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/962
• build(deps): bump serde_json from 1.0.88 to 1.0.89 by @dependabot in https://github.com/delta-io/delta-rs/pull/963
• build(deps): bump bytes from 1.2.1 to 1.3.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/964
• build(deps): bump tokio from 1.21.2 to 1.22.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/965
• build(deps): bump openssl from 0.10.42 to 0.10.43 by @dependabot in https://github.com/delta-io/delta-rs/pull/966
• build(deps): bump async-trait from 0.1.58 to 0.1.59 by @dependabot in https://github.com/delta-io/delta-rs/pull/990
• build(deps): bump parquet2 from 0.16.3 to 0.17.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/991
• build(deps): bump libc from 0.2.137 to 0.2.138 by @dependabot in https://github.com/delta-io/delta-rs/pull/992
• build(deps): bump serde from 1.0.147 to 1.0.149 by @dependabot in https://github.com/delta-io/delta-rs/pull/995
• build(deps): bump flatbuffers from 2.1.2 to 22.9.29 in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/952
• build(deps): bump flatbuffers from 2.1.2 to 22.9.29 in /delta-inspect by @dependabot in https://github.com/delta-io/delta-rs/pull/951

New Contributors

• @iajoiner made their first contribution in https://github.com/delta-io/delta-rs/pull/1001
• @damiondoesthings made their first contribution in https://github.com/delta-io/delta-rs/pull/1007

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.6.4...rust-v0.6.0

What's Changed

• fix(python): enforce reader protocol version by @wjones127 in https://github.com/delta-io/delta-rs/pull/932
• feat: rewrite operations by @roeap in https://github.com/delta-io/delta-rs/pull/852
• chore: bump datafusion and arrow by @roeap in https://github.com/delta-io/delta-rs/pull/940
• Allow for reading columns as dictionaries using to_pyarrow_dataset by @Kuhlwein in https://github.com/delta-io/delta-rs/pull/941
• Add TableProviderFactory and test for SQL to register tables dynamically at runtime by @avantgardnerio in https://github.com/delta-io/delta-rs/pull/892
• feat: improve write perfromance of DeltaFileSystemHandler by @roeap in https://github.com/delta-io/delta-rs/pull/943
• Update CONTRIBUTING.md by @wjones127 in https://github.com/delta-io/delta-rs/pull/944
• build(deps): bump serde_json from 1.0.87 to 1.0.88 by @dependabot in https://github.com/delta-io/delta-rs/pull/947
• build(deps): bump reqwest from 0.11.12 to 0.11.13 by @dependabot in https://github.com/delta-io/delta-rs/pull/946
• docs: add simple operations example by @roeap in https://github.com/delta-io/delta-rs/pull/953
• bump rust core version to 0.5.0 by @houqp in https://github.com/delta-io/delta-rs/pull/961
• pin glic_version version in dep to unblock creates.io release by @houqp in https://github.com/delta-io/delta-rs/pull/967
• remove all wildcard version pin in Cargo.toml by @houqp in https://github.com/delta-io/delta-rs/pull/968
• Bump version of the Python binding to 0.6.4 by @fvaleye in https://github.com/delta-io/delta-rs/pull/970

New Contributors

• @Kuhlwein made their first contribution in https://github.com/delta-io/delta-rs/pull/941
• @avantgardnerio made their first contribution in https://github.com/delta-io/delta-rs/pull/892

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.6.3...python-v0.6.4

What's Changed

• Add max and min values to Statistics by @viirya in https://github.com/delta-io/delta-rs/pull/327
• Use WebIdentityProvider for DynamoDb client in k8s by @rusty-jules in https://github.com/delta-io/delta-rs/pull/328
• bump rust version in preparation for the next release by @houqp in https://github.com/delta-io/delta-rs/pull/329
• fix automated rust release CD job by @houqp in https://github.com/delta-io/delta-rs/pull/326
• add pandas keyword to python package metadata by @houqp in https://github.com/delta-io/delta-rs/pull/325
• expose update_incremental API to python binding by @houqp in https://github.com/delta-io/delta-rs/pull/332
• update python related docs by @houqp in https://github.com/delta-io/delta-rs/pull/331
• Upgrade arrow, parquet and datafusion by @Dandandan in https://github.com/delta-io/delta-rs/pull/335
• added warning message if the detected glibc version is < 2.28 by @Smurphy000 in https://github.com/delta-io/delta-rs/pull/334
• Convert scalar value to correct type based on arrow data type. by @viirya in https://github.com/delta-io/delta-rs/pull/336
• Fix consecutive checkpoints by @mosyp in https://github.com/delta-io/delta-rs/pull/333
• Fix new clippy warnings coming up in CI by @xianwill in https://github.com/delta-io/delta-rs/pull/341
• perform incremental update after transaction commit by @houqp in https://github.com/delta-io/delta-rs/pull/343
• Add timestamp handling to checkpoint writer by @xianwill in https://github.com/delta-io/delta-rs/pull/340
• Add clear table state in load_version when no checkpoint found. by @zijie0 in https://github.com/delta-io/delta-rs/pull/347
• Low level create table by @Smurphy000 in https://github.com/delta-io/delta-rs/pull/342
• pub DeltaTable method to retrieve table configurations by @Smurphy000 in https://github.com/delta-io/delta-rs/pull/356
• Modify partition_values field type in Add/Remove actions. by @zijie0 in https://github.com/delta-io/delta-rs/pull/354
• fix sleep workaround in checkpoint test by @houqp in https://github.com/delta-io/delta-rs/pull/360
• Modify get_files_by_partitions to use partition values by @zijie0 in https://github.com/delta-io/delta-rs/pull/362
• Fix get_latest_version returning version < 0. by @zijie0 in https://github.com/delta-io/delta-rs/pull/364
• fix typo in python release CI config by @houqp in https://github.com/delta-io/delta-rs/pull/365
• cache cargo builds by @houqp in https://github.com/delta-io/delta-rs/pull/359
• Add '.tmp' suffix to temporary file of prepared commit by @mosyp in https://github.com/delta-io/delta-rs/pull/366
• support partition value string deserialization for float/double/date by @houqp in https://github.com/delta-io/delta-rs/pull/363
• Implement atomic put_obj. by @zijie0 in https://github.com/delta-io/delta-rs/pull/367
• Make Format.options to be required field by @mosyp in https://github.com/delta-io/delta-rs/pull/370
• Allow filesystem backend put_obj to overwrite existing by @mosyp in https://github.com/delta-io/delta-rs/pull/376
• Wrap DeltaTransactionError with DeltaTableError. by @zijie0 in https://github.com/delta-io/delta-rs/pull/374
• Refactoring of black, isort, mypy tools usages into pyproject.toml by @fvaleye in https://github.com/delta-io/delta-rs/pull/378
• Implement consistent behavior in Windows with regard to swap parameter. by @zijie0 in https://github.com/delta-io/delta-rs/pull/379
• Merge Cargo.toml into pyproject.toml by @fvaleye in https://github.com/delta-io/delta-rs/pull/381
• Update datafusion and ballista links in README by @ei-grad in https://github.com/delta-io/delta-rs/pull/382
• Add sts assume role credentials provider for S3 by @mosyp in https://github.com/delta-io/delta-rs/pull/383
• Reuse table/storage instances in checkpoints by @mosyp in https://github.com/delta-io/delta-rs/pull/384
• additional error handling to atomic_rename by @Marnixvdb in https://github.com/delta-io/delta-rs/pull/386
• Upgrade to DataFusion 5.0 by @Dandandan in https://github.com/delta-io/delta-rs/pull/389
• added initial commit info on create method for a DeltaTable by @Smurphy000 in https://github.com/delta-io/delta-rs/pull/387
• Google cloud by @blogle in https://github.com/delta-io/delta-rs/pull/355
• Remove version param from create_checkpoint_from_table by @mosyp in https://github.com/delta-io/delta-rs/pull/399
• Implement delete_objs in fs and s3 storage backends. by @zijie0 in https://github.com/delta-io/delta-rs/pull/395
• Add examples for reading delta table with Rust API. by @zijie0 in https://github.com/delta-io/delta-rs/pull/400
• Update pyproject definition in pyproject.toml by @fvaleye in https://github.com/delta-io/delta-rs/pull/405
• Use tokio::fs::rename in put_obj. by @zijie0 in https://github.com/delta-io/delta-rs/pull/403
• Fix duplicates on update call by @mosyp in https://github.com/delta-io/delta-rs/pull/398
• Add a Makefile build task in the Python binding by @fvaleye in https://github.com/delta-io/delta-rs/pull/410
• Add implementation for load_with_datetime in Python package. by @zijie0 in https://github.com/delta-io/delta-rs/pull/411
• Add filesystem argument for reading DeltaTable in Python binding by @fvaleye in https://github.com/delta-io/delta-rs/pull/414
• Fix reading nullable action fields from parquet by @mosyp in https://github.com/delta-io/delta-rs/pull/417
• Ensures that all table schemas are of StructType by @blogle in https://github.com/delta-io/delta-rs/pull/415
• Gcs writer bugs by @blogle in https://github.com/delta-io/delta-rs/pull/412
• Add S3StorageOptions to allow configuring S3 backend explicitly by @xianwill in https://github.com/delta-io/delta-rs/pull/418
• Read a DeltaTable using a Data Catalog by @fvaleye in https://github.com/delta-io/delta-rs/pull/419
• Change checkpoint creation logs from info to debug by @mosyp in https://github.com/delta-io/delta-rs/pull/423
• Add LICENSE file in the Python binding and refer it in the pyproject by @fvaleye in https://github.com/delta-io/delta-rs/pull/422
• Audit action field optionality by @fvaleye in https://github.com/delta-io/delta-rs/pull/380
• Introduce DeltaConfig and tombstones retention policy by @mosyp in https://github.com/delta-io/delta-rs/pull/420
• [README] Replace the inactive rust-dataframe with polars by @sa- in https://github.com/delta-io/delta-rs/pull/426
• Bump arrow to 6.0.0-SNAPSHOT and bring map support to schema by @mosyp in https://github.com/delta-io/delta-rs/pull/375
• Support partition value string deserialization for timestamp/binary by @zijie0 in https://github.com/delta-io/delta-rs/pull/371
• Document the valid primitive types by @Ekleog in https://github.com/delta-io/delta-rs/pull/430
• Add is_non_acquirable field to the dynamodb lock by @mosyp in https://github.com/delta-io/delta-rs/pull/429
• Clean up DeltaTransactionError by @mosyp in https://github.com/delta-io/delta-rs/pull/432
• Optimize remove action apply with early iteration exit #424 by @akshay26031996 in https://github.com/delta-io/delta-rs/pull/431
• Decode path in Add and Remove actions. by @zijie0 in https://github.com/delta-io/delta-rs/pull/434
• reenable datafusion integration with temporary fork by @houqp in https://github.com/delta-io/delta-rs/pull/436
• Add history command in delta-rs by @fvaleye in https://github.com/delta-io/delta-rs/pull/428
• Release Python binding version 0.5.3 by @fvaleye in https://github.com/delta-io/delta-rs/pull/439
• Add delete_lock and fix release_lock by @mosyp in https://github.com/delta-io/delta-rs/pull/440
• Fixing test to compare sorted vec by @akshay26031996 in https://github.com/delta-io/delta-rs/pull/443
• Batch-apply remove actions in tombstone handling by @dispanser in https://github.com/delta-io/delta-rs/pull/444
• Update datafusion links by @bbigras in https://github.com/delta-io/delta-rs/pull/446
• Run all tests under s3 feature flag by @mosyp in https://github.com/delta-io/delta-rs/pull/447
• Add maturin develop command with extras in Python binding by @fvaleye in https://github.com/delta-io/delta-rs/pull/448
• README: mark Checkpoint creation as done for Rust by @bbigras in https://github.com/delta-io/delta-rs/pull/449
• Fix broken tombstones metadata when extended_file_metadata is different between tomstones in state by @mosyp in https://github.com/delta-io/delta-rs/pull/450
• No tombstone loading by @dispanser in https://github.com/delta-io/delta-rs/pull/445
• return lazy iterator in get tombstone methods by @houqp in https://github.com/delta-io/delta-rs/pull/452
• Generate new session name on assume role credentials provider refresh by @mosyp in https://github.com/delta-io/delta-rs/pull/451
• Add pool_idle_timeout options for s3 and sts clients by @mosyp in https://github.com/delta-io/delta-rs/pull/458
• Do action reconciliation by @viirya in https://github.com/delta-io/delta-rs/pull/456
• Use action default stats by @viirya in https://github.com/delta-io/delta-rs/pull/459
• Add new module for DeltaTableState by @viirya in https://github.com/delta-io/delta-rs/pull/464
• Support hash lookup by path string for Remove action by @viirya in https://github.com/delta-io/delta-rs/pull/462
• Fix coverage of the Python tests by @fvaleye in https://github.com/delta-io/delta-rs/pull/467
• materialize tables in pyhton via native storage backend by @roeap in https://github.com/delta-io/delta-rs/pull/463
• Make file storage backend's atomic rename async by @viirya in https://github.com/delta-io/delta-rs/pull/471
• Add GCS feature to the Python Cargo.toml file by @kelvins in https://github.com/delta-io/delta-rs/pull/476
• Throw an error when filter key is not in partitioned columns. by @zijie0 in https://github.com/delta-io/delta-rs/pull/475
• Fix documentation for the DeltaStorageHandler by @fvaleye in https://github.com/delta-io/delta-rs/pull/483
• Update README.adoc by @dennyglee in https://github.com/delta-io/delta-rs/pull/482
• Update azure crates by @roeap in https://github.com/delta-io/delta-rs/pull/474
• Try to fix flaky rename under Windows by @viirya in https://github.com/delta-io/delta-rs/pull/485
• Fix python build by @roeap in https://github.com/delta-io/delta-rs/pull/487
• Add compare operators for partition filter. by @zijie0 in https://github.com/delta-io/delta-rs/pull/478
• Add docs for ADLS Gen2. by @thovoll in https://github.com/delta-io/delta-rs/pull/492
• Change Rust edition from 2018 to 2021 by @fvaleye in https://github.com/delta-io/delta-rs/pull/490
• Add DeltaTableError in Python binding by @fvaleye in https://github.com/delta-io/delta-rs/pull/496
• Bump arrow to 6.1.0 by @mosyp in https://github.com/delta-io/delta-rs/pull/494
• Add authorization options for azure storage backend by @roeap in https://github.com/delta-io/delta-rs/pull/486
• Clean up expired delta table commit logs after checkpoint by @viirya in https://github.com/delta-io/delta-rs/pull/484
• pin arrow to major version by @houqp in https://github.com/delta-io/delta-rs/pull/501
• Add bool config enableExpiredLogCleanup by @viirya in https://github.com/delta-io/delta-rs/pull/500
• Fix cleanup_expired_logs timestamp by @mosyp in https://github.com/delta-io/delta-rs/pull/503
• Fix nullable partition values by @mosyp in https://github.com/delta-io/delta-rs/pull/498
• Fix memory overhead when creating checkpoint by @mosyp in https://github.com/delta-io/delta-rs/pull/502
• Retry S3 get request on 500 Internal Server Error by @mosyp in https://github.com/delta-io/delta-rs/pull/510
• Update datafusion version to 6 by @matthewmturner in https://github.com/delta-io/delta-rs/pull/516
• Add support for creating Array and Map schema types by @andrei-ionescu in https://github.com/delta-io/delta-rs/pull/517
• Fix S3 list_objs and cleanup_metadata by @mosyp in https://github.com/delta-io/delta-rs/pull/518
• Add datafusion examples to docs by @matthewmturner in https://github.com/delta-io/delta-rs/pull/519
• Move dynamodb lock into its own crate by @viirya in https://github.com/delta-io/delta-rs/pull/508
• Return complete history by @viirya in https://github.com/delta-io/delta-rs/pull/526
• Add gcs test and improve credential error by @viirya in https://github.com/delta-io/delta-rs/pull/533
• Fix Delta metadata transaction schema by @langelgjm in https://github.com/delta-io/delta-rs/pull/531
• Refresh Python usage documentation by @wjones127 in https://github.com/delta-io/delta-rs/pull/539
• add macOS 11 by @sa- in https://github.com/delta-io/delta-rs/pull/541
• Remove coupling of DynamoDbLockClient from S3 storage by @ahmedriza in https://github.com/delta-io/delta-rs/pull/535
• Add storage options for backends by @fvaleye in https://github.com/delta-io/delta-rs/pull/544
• Release of the Python binding version 0.5.5 by @fvaleye in https://github.com/delta-io/delta-rs/pull/547
• Fix the Python release Github action workflow for macos11 by @fvaleye in https://github.com/delta-io/delta-rs/pull/548
• Implement polling for table updates. (#413) by @Blajda in https://github.com/delta-io/delta-rs/pull/550
• Implement missing Azure storage backend methods by @thovoll in https://github.com/delta-io/delta-rs/pull/499
• formally verify S3 atomic rename by @houqp in https://github.com/delta-io/delta-rs/pull/540
• Fix VACUUM by using the URI of the DeltaTable when filtering the files to delete by @fvaleye in https://github.com/delta-io/delta-rs/pull/551
• move delta-inspect to its own crate by @houqp in https://github.com/delta-io/delta-rs/pull/557
• Upgrade DataFusion, Arrow, Parquet dependencies by @Dandandan in https://github.com/delta-io/delta-rs/pull/562
• fix clippy warnings by @houqp in https://github.com/delta-io/delta-rs/pull/567
• Azure improvements by @thovoll in https://github.com/delta-io/delta-rs/pull/556
• Update ADLSGen2-HOWTO.md by @dgcaron in https://github.com/delta-io/delta-rs/pull/560
• Parse partition values before handing to PyArrow by @wjones127 in https://github.com/delta-io/delta-rs/pull/565
• Upgrade arrow, parquet, datafusion version by @zemelLeong in https://github.com/delta-io/delta-rs/pull/583
• Record Batch Writer by @roeap in https://github.com/delta-io/delta-rs/pull/573
• Replace table location prefix from s3a to s3 by @novakov-alexey-zz in https://github.com/delta-io/delta-rs/pull/585
• Allow metadata for write_deltalake by @PadenZach in https://github.com/delta-io/delta-rs/pull/587
• Change private time_utils module to public. by @xianwill in https://github.com/delta-io/delta-rs/pull/586
• Release of the Python binding version 0.5.7 by @fvaleye in https://github.com/delta-io/delta-rs/pull/589
• Expose read and write options in public API by @george-zubrienko in https://github.com/delta-io/delta-rs/pull/581
• [proof] make sure lock at least expires once by @houqp in https://github.com/delta-io/delta-rs/pull/591
• Avoid building pandas and numpy from source by @wjones127 in https://github.com/delta-io/delta-rs/pull/595
• Introduce require_files for tracking the add files in table state by @mosyp in https://github.com/delta-io/delta-rs/pull/594
• Make sure pandas is optional by @wjones127 in https://github.com/delta-io/delta-rs/pull/597
• High level Delta Operations with Datafusion by @roeap in https://github.com/delta-io/delta-rs/pull/584
• Re-enable datafusion tests and improve supported types. by @roeap in https://github.com/delta-io/delta-rs/pull/601
• default to root for empty path in azure store by @roeap in https://github.com/delta-io/delta-rs/pull/603
• publish dynamodb_lock to crates.io by @houqp in https://github.com/delta-io/delta-rs/pull/605
• Configure Azure storage using a map (#555) by @Blajda in https://github.com/delta-io/delta-rs/pull/598
• Azure options by @roeap in https://github.com/delta-io/delta-rs/pull/606
• Update rusoto dependencies to 0.48 by @ahmedriza in https://github.com/delta-io/delta-rs/pull/611
• upgrade to datafusion 8 by @houqp in https://github.com/delta-io/delta-rs/pull/612
• fix: Use relative paths for add paths by @wjones127 in https://github.com/delta-io/delta-rs/pull/618
• Bin packing optimization by @Blajda in https://github.com/delta-io/delta-rs/pull/607
• feat: impl rename_noreplace with std::fs::hard_link by default by @wjones127 in https://github.com/delta-io/delta-rs/pull/621
• Fix the AWS_REGION environment variable configuration in S3 backend by @fvaleye in https://github.com/delta-io/delta-rs/pull/633
• Refactor azure storage with crate updates by @roeap in https://github.com/delta-io/delta-rs/pull/644
• Defer creation of storage backend in DeltaTableBuilder by @Blajda in https://github.com/delta-io/delta-rs/pull/639
• fix: Add correct size and null paritition values to add actions by @wjones127 in https://github.com/delta-io/delta-rs/pull/625
• fix: Prevent warning spam when reading tables generated by delta 1.2.1 by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/651
• refactor: move version field to DeltaTableState by @roeap in https://github.com/delta-io/delta-rs/pull/649
• feat: add enforce_retention_duration param to vacuum method by @houqp in https://github.com/delta-io/delta-rs/pull/648
• fix: read vacuumed delta log without _last_checkpoint by @roeap in https://github.com/delta-io/delta-rs/pull/643
• feat: Upgrade to arrow/parquet 15 and datafusion 9 by @xianwill in https://github.com/delta-io/delta-rs/pull/652
• Release of the Python binding version 0.5.8 by @fvaleye in https://github.com/delta-io/delta-rs/pull/640
• JSON Writer writing partitions values fix by @Blajda in https://github.com/delta-io/delta-rs/pull/658
• Support date32 and decimal stats in write_deltalake by @wjones127 in https://github.com/delta-io/delta-rs/pull/659
• bugfix: Make sure vacuum works on relative paths by @wjones127 in https://github.com/delta-io/delta-rs/pull/664
• Fix linting / build on main by @mrk-its in https://github.com/delta-io/delta-rs/pull/670
• feat: add support for HTTPS_PROXY env var by @xfrancois in https://github.com/delta-io/delta-rs/pull/665
• Utilise struct stats when available by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/656
• fix: inconsistent path in azure list by @roeap in https://github.com/delta-io/delta-rs/pull/673
• Factor vacuum and implement a builder by @Blajda in https://github.com/delta-io/delta-rs/pull/672
• fix: get docs building and add to CI checks by @wjones127 in https://github.com/delta-io/delta-rs/pull/679
• fix: omit common prefixes in azure list_objs by @roeap in https://github.com/delta-io/delta-rs/pull/683
• fix: traverse directories in local list_objs by @roeap in https://github.com/delta-io/delta-rs/pull/681
• Implement vacuum tests and general test setup utils by @Blajda in https://github.com/delta-io/delta-rs/pull/682
• Setup Github dependabot for Rust by @fvaleye in https://github.com/delta-io/delta-rs/pull/687
• feat: sharable reference to storage backend by @roeap in https://github.com/delta-io/delta-rs/pull/697
• feat: integrate with object_store / datafusion APIs by @roeap in https://github.com/delta-io/delta-rs/pull/703
• Prune scanned files on column stats by @roeap in https://github.com/delta-io/delta-rs/pull/724
• Fix parsing null counts for struct type columns in the struct stats by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/714
• Implement Python Schema in Rust by @wjones127 in https://github.com/delta-io/delta-rs/pull/684
• fix: Address clippy lint warnings. by @tsh56 in https://github.com/delta-io/delta-rs/pull/742
• Cast min and max too when parsing stats by @wjones127 in https://github.com/delta-io/delta-rs/pull/753
• turn table state version into a private field by @houqp in https://github.com/delta-io/delta-rs/pull/772
• parquet2 implementation backed by parquet2 feature gate by @houqp in https://github.com/delta-io/delta-rs/pull/465
• Adopt ObjectStore by @roeap in https://github.com/delta-io/delta-rs/pull/761
• chore: cleanup by @roeap in https://github.com/delta-io/delta-rs/pull/774
• feat: add gcs integration tests by @roeap in https://github.com/delta-io/delta-rs/pull/779
• feat: integrate object_store for read/write with pyarrow by @roeap in https://github.com/delta-io/delta-rs/pull/799
• cleanup errors and make DeltaWriterError internal by @roeap in https://github.com/delta-io/delta-rs/pull/784
• Refactoring of the Github Action python release by @fvaleye in https://github.com/delta-io/delta-rs/pull/810
• Re-allow writing to non-existant local paths by @wjones127 in https://github.com/delta-io/delta-rs/pull/811
• chore: bump arrow and friends by @roeap in https://github.com/delta-io/delta-rs/pull/814
• feat: improve storage configuration by @roeap in https://github.com/delta-io/delta-rs/pull/822
• Datafusion-imports by @roeap in https://github.com/delta-io/delta-rs/pull/823
• Fix warnings by @wjones127 in https://github.com/delta-io/delta-rs/pull/839
• comment cleanup by @houqp in https://github.com/delta-io/delta-rs/pull/840
• add codeowners by @houqp in https://github.com/delta-io/delta-rs/pull/841
• use rustls for delta checkpoint lambda by @houqp in https://github.com/delta-io/delta-rs/pull/842
• Add invariant enforcement support by @wjones127 in https://github.com/delta-io/delta-rs/pull/834
• Add contributing page with roadmap and good first issues link by @wjones127 in https://github.com/delta-io/delta-rs/pull/853
• Added without_files flag to DeltaTable constructor by @MykhailoHevak in https://github.com/delta-io/delta-rs/pull/866
• chore: update object-store dependency by @wjones127 in https://github.com/delta-io/delta-rs/pull/884
• Fix parsing struct stats after schema evolution by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/901
• fix: pass storage options down when getting delta table by @wjones127 in https://github.com/delta-io/delta-rs/pull/893
• Fix cargo clippy issues 0.1.65 in Rust by @fvaleye in https://github.com/delta-io/delta-rs/pull/923
• feat: rewrite operations by @roeap in https://github.com/delta-io/delta-rs/pull/852
• chore: bump datafusion and arrow by @roeap in https://github.com/delta-io/delta-rs/pull/940
• Allow for reading columns as dictionaries using to_pyarrow_dataset by @Kuhlwein in https://github.com/delta-io/delta-rs/pull/941
• Add TableProviderFactory and test for SQL to register tables dynamically at runtime by @avantgardnerio in https://github.com/delta-io/delta-rs/pull/892
• feat: improve write perfromance of DeltaFileSystemHandler by @roeap in https://github.com/delta-io/delta-rs/pull/943
• Update CONTRIBUTING.md by @wjones127 in https://github.com/delta-io/delta-rs/pull/944
• docs: add simple operations example by @roeap in https://github.com/delta-io/delta-rs/pull/953
• bump rust core version to 0.5.0 by @houqp in https://github.com/delta-io/delta-rs/pull/961

New Contributors

• @rusty-jules made their first contribution in https://github.com/delta-io/delta-rs/pull/328
• @Smurphy000 made their first contribution in https://github.com/delta-io/delta-rs/pull/334
• @ei-grad made their first contribution in https://github.com/delta-io/delta-rs/pull/382
• @Marnixvdb made their first contribution in https://github.com/delta-io/delta-rs/pull/386
• @blogle made their first contribution in https://github.com/delta-io/delta-rs/pull/355
• @sa- made their first contribution in https://github.com/delta-io/delta-rs/pull/426
• @Ekleog made their first contribution in https://github.com/delta-io/delta-rs/pull/430
• @akshay26031996 made their first contribution in https://github.com/delta-io/delta-rs/pull/431
• @bbigras made their first contribution in https://github.com/delta-io/delta-rs/pull/446
• @roeap made their first contribution in https://github.com/delta-io/delta-rs/pull/463
• @kelvins made their first contribution in https://github.com/delta-io/delta-rs/pull/476
• @dennyglee made their first contribution in https://github.com/delta-io/delta-rs/pull/482
• @thovoll made their first contribution in https://github.com/delta-io/delta-rs/pull/492
• @matthewmturner made their first contribution in https://github.com/delta-io/delta-rs/pull/516
• @andrei-ionescu made their first contribution in https://github.com/delta-io/delta-rs/pull/517
• @langelgjm made their first contribution in https://github.com/delta-io/delta-rs/pull/531
• @wjones127 made their first contribution in https://github.com/delta-io/delta-rs/pull/525
• @ahmedriza made their first contribution in https://github.com/delta-io/delta-rs/pull/535
• @Blajda made their first contribution in https://github.com/delta-io/delta-rs/pull/550
• @dgcaron made their first contribution in https://github.com/delta-io/delta-rs/pull/560
• @zemelLeong made their first contribution in https://github.com/delta-io/delta-rs/pull/583
• @novakov-alexey-zz made their first contribution in https://github.com/delta-io/delta-rs/pull/585
• @PadenZach made their first contribution in https://github.com/delta-io/delta-rs/pull/587
• @george-zubrienko made their first contribution in https://github.com/delta-io/delta-rs/pull/581
• @WarSame made their first contribution in https://github.com/delta-io/delta-rs/pull/590
• @dependabot made their first contribution in https://github.com/delta-io/delta-rs/pull/626
• @Tom-Newton made their first contribution in https://github.com/delta-io/delta-rs/pull/651
• @mrk-its made their first contribution in https://github.com/delta-io/delta-rs/pull/670
• @xfrancois made their first contribution in https://github.com/delta-io/delta-rs/pull/665
• @Bernolt made their first contribution in https://github.com/delta-io/delta-rs/pull/704
• @tsh56 made their first contribution in https://github.com/delta-io/delta-rs/pull/742
• @MykhailoHevak made their first contribution in https://github.com/delta-io/delta-rs/pull/866
• @Kuhlwein made their first contribution in https://github.com/delta-io/delta-rs/pull/941
• @avantgardnerio made their first contribution in https://github.com/delta-io/delta-rs/pull/892

Full Changelog: https://github.com/delta-io/delta-rs/compare/rust-v0.4.0...rust-v0.5.0

What's Changed

• Added without_files flag to DeltaTable constructor by @MykhailoHevak in https://github.com/delta-io/delta-rs/pull/866
• chore: update object-store dependency by @wjones127 in https://github.com/delta-io/delta-rs/pull/884
• build(deps): bump serde from 1.0.145 to 1.0.147 by @dependabot in https://github.com/delta-io/delta-rs/pull/895
• build(deps): bump anyhow from 1.0.65 to 1.0.66 by @dependabot in https://github.com/delta-io/delta-rs/pull/897
• build(deps): bump serde_json from 1.0.86 to 1.0.87 by @dependabot in https://github.com/delta-io/delta-rs/pull/896
• build(deps): bump futures from 0.3.24 to 0.3.25 by @dependabot in https://github.com/delta-io/delta-rs/pull/899
• build(deps): bump async-trait from 0.1.57 to 0.1.58 by @dependabot in https://github.com/delta-io/delta-rs/pull/898
• build(deps): bump once_cell from 1.15.0 to 1.16.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/907
• build(deps): bump libc from 0.2.135 to 0.2.137 by @dependabot in https://github.com/delta-io/delta-rs/pull/905
• build(deps): bump lambda_runtime from 0.6.1 to 0.7.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/903
• Fix parsing struct stats after schema evolution by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/901
• fix: pass storage options down when getting delta table by @wjones127 in https://github.com/delta-io/delta-rs/pull/893
• Fix cargo clippy issues 0.1.65 in Rust by @fvaleye in https://github.com/delta-io/delta-rs/pull/923

New Contributors

• @MykhailoHevak made their first contribution in https://github.com/delta-io/delta-rs/pull/866

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.6.2...python-v0.6.3

What's Changed

• Added without_files flag to DeltaTable constructor by @MykhailoHevak in https://github.com/delta-io/delta-rs/pull/866
• chore: update object-store dependency by @wjones127 in https://github.com/delta-io/delta-rs/pull/884
• build(deps): bump serde from 1.0.145 to 1.0.147 by @dependabot in https://github.com/delta-io/delta-rs/pull/895
• build(deps): bump anyhow from 1.0.65 to 1.0.66 by @dependabot in https://github.com/delta-io/delta-rs/pull/897
• build(deps): bump serde_json from 1.0.86 to 1.0.87 by @dependabot in https://github.com/delta-io/delta-rs/pull/896
• build(deps): bump futures from 0.3.24 to 0.3.25 by @dependabot in https://github.com/delta-io/delta-rs/pull/899
• build(deps): bump async-trait from 0.1.57 to 0.1.58 by @dependabot in https://github.com/delta-io/delta-rs/pull/898
• build(deps): bump once_cell from 1.15.0 to 1.16.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/907
• build(deps): bump libc from 0.2.135 to 0.2.137 by @dependabot in https://github.com/delta-io/delta-rs/pull/905
• build(deps): bump lambda_runtime from 0.6.1 to 0.7.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/903
• Fix parsing struct stats after schema evolution by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/901
• fix: pass storage options down when getting delta table by @wjones127 in https://github.com/delta-io/delta-rs/pull/893
• Fix cargo clippy issues 0.1.65 in Rust by @fvaleye in https://github.com/delta-io/delta-rs/pull/923
• Bump version of the Python binding to 0.6.3 by @fvaleye in https://github.com/delta-io/delta-rs/pull/924

New Contributors

• @MykhailoHevak made their first contribution in https://github.com/delta-io/delta-rs/pull/866

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.6.2...python-v0.6.3

What's Changed

• build(deps): bump url from 2.2.2 to 2.3.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/800
• build(deps): bump lambda_runtime from 0.6.0 to 0.6.1 by @dependabot in https://github.com/delta-io/delta-rs/pull/801
• build(deps): bump parquet2 from 0.16.2 to 0.16.3 by @dependabot in https://github.com/delta-io/delta-rs/pull/802
• build(deps): bump criterion from 0.3.6 to 0.4.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/803
• build(deps): bump percent-encoding from 2.1.0 to 2.2.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/804
• build(deps): bump url from 2.3.0 to 2.3.1 by @dependabot in https://github.com/delta-io/delta-rs/pull/805
• feat: integrate object_store for read/write with pyarrow by @roeap in https://github.com/delta-io/delta-rs/pull/799
• cleanup errors and make DeltaWriterError internal by @roeap in https://github.com/delta-io/delta-rs/pull/784
• Refactoring of the Github Action python release by @fvaleye in https://github.com/delta-io/delta-rs/pull/810
• Re-allow writing to non-existant local paths by @wjones127 in https://github.com/delta-io/delta-rs/pull/811
• [Python][Docs] Add description to landing page by @wjones127 in https://github.com/delta-io/delta-rs/pull/817
• [Python] Fix handling of null stats in write_deltalake by @wjones127 in https://github.com/delta-io/delta-rs/pull/815
• feat(python): Set smaller defaults on row group and file size by @wjones127 in https://github.com/delta-io/delta-rs/pull/818
• chore: bump arrow and friends by @roeap in https://github.com/delta-io/delta-rs/pull/814
• feat: improve storage configuration by @roeap in https://github.com/delta-io/delta-rs/pull/822
• Datafusion-imports by @roeap in https://github.com/delta-io/delta-rs/pull/823
• build(deps): bump tokio-stream from 0.1.9 to 0.1.10 by @dependabot in https://github.com/delta-io/delta-rs/pull/824
• build(deps): bump anyhow from 1.0.64 to 1.0.65 by @dependabot in https://github.com/delta-io/delta-rs/pull/825
• build(deps): bump once_cell from 1.13.1 to 1.14.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/826
• build(deps): bump thiserror from 1.0.34 to 1.0.35 by @dependabot in https://github.com/delta-io/delta-rs/pull/827
• build(deps): bump env_logger from 0.7.1 to 0.9.1 by @dependabot in https://github.com/delta-io/delta-rs/pull/829
• build(deps): bump tokio from 1.21.0 to 1.21.1 by @dependabot in https://github.com/delta-io/delta-rs/pull/828
• Fix warnings by @wjones127 in https://github.com/delta-io/delta-rs/pull/839
• comment cleanup by @houqp in https://github.com/delta-io/delta-rs/pull/840
• add codeowners by @houqp in https://github.com/delta-io/delta-rs/pull/841
• build(deps): bump thiserror from 1.0.35 to 1.0.36 by @dependabot in https://github.com/delta-io/delta-rs/pull/843
• build(deps): bump libc from 0.2.132 to 0.2.133 by @dependabot in https://github.com/delta-io/delta-rs/pull/844
• build(deps): bump serde from 1.0.144 to 1.0.145 by @dependabot in https://github.com/delta-io/delta-rs/pull/846
• build(deps): bump reqwest from 0.11.11 to 0.11.12 by @dependabot in https://github.com/delta-io/delta-rs/pull/847
• build(deps): bump once_cell from 1.14.0 to 1.15.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/849
• use rustls for delta checkpoint lambda by @houqp in https://github.com/delta-io/delta-rs/pull/842
• Add invariant enforcement support by @wjones127 in https://github.com/delta-io/delta-rs/pull/834
• Add contributing page with roadmap and good first issues link by @wjones127 in https://github.com/delta-io/delta-rs/pull/853
• build(deps): bump tokio from 1.21.1 to 1.21.2 by @dependabot in https://github.com/delta-io/delta-rs/pull/856
• build(deps): bump libc from 0.2.133 to 0.2.134 by @dependabot in https://github.com/delta-io/delta-rs/pull/857
• build(deps): bump openssl from 0.10.41 to 0.10.42 by @dependabot in https://github.com/delta-io/delta-rs/pull/858
• build(deps): bump thiserror from 1.0.36 to 1.0.37 by @dependabot in https://github.com/delta-io/delta-rs/pull/859
• build(deps): bump uuid from 1.1.2 to 1.2.1 by @dependabot in https://github.com/delta-io/delta-rs/pull/872
• build(deps): bump serde_json from 1.0.85 to 1.0.86 by @dependabot in https://github.com/delta-io/delta-rs/pull/874
• build(deps): bump pyo3 from 0.17.1 to 0.17.2 by @dependabot in https://github.com/delta-io/delta-rs/pull/873
• Bump Python binding version to 0.6.2 by @fvaleye in https://github.com/delta-io/delta-rs/pull/876
• Fix the Python Release Github Action by @fvaleye in https://github.com/delta-io/delta-rs/pull/877

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.6.1...python-v0.6.2

What's Changed

• feat: add gcs integration tests by @roeap in https://github.com/delta-io/delta-rs/pull/779
• build(deps): bump lz4-sys from 1.9.2 to 1.9.4 in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/782
• build(deps): bump lz4-sys from 1.9.2 to 1.9.4 in /delta-inspect by @dependabot in https://github.com/delta-io/delta-rs/pull/783
• build(deps): bump tokio from 1.20.1 to 1.21.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/790
• build(deps): bump thiserror from 1.0.32 to 1.0.34 by @dependabot in https://github.com/delta-io/delta-rs/pull/792
• build(deps): bump pretty_assertions from 1.2.1 to 1.3.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/791
• build(deps): bump anyhow from 1.0.62 to 1.0.64 by @dependabot in https://github.com/delta-io/delta-rs/pull/793
• build(deps): bump env_logger from 0.7.1 to 0.9.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/794
• hotfix: python object store paths by @roeap in https://github.com/delta-io/delta-rs/pull/787
• prepare python release 0.6.1 by @roeap in https://github.com/delta-io/delta-rs/pull/795

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.6.0...python-v0.6.1

What's Changed

• JSON Writer writing partitions values fix by @Blajda in https://github.com/delta-io/delta-rs/pull/658
• Support date32 and decimal stats in write_deltalake by @wjones127 in https://github.com/delta-io/delta-rs/pull/659
• bugfix: Make sure vacuum works on relative paths by @wjones127 in https://github.com/delta-io/delta-rs/pull/664
• Fix linting / build on main by @mrk-its in https://github.com/delta-io/delta-rs/pull/670
• feat: add support for HTTPS_PROXY env var by @xfrancois in https://github.com/delta-io/delta-rs/pull/665
• Utilise struct stats when available by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/656
• fix: inconsistent path in azure list by @roeap in https://github.com/delta-io/delta-rs/pull/673
• Factor vacuum and implement a builder by @Blajda in https://github.com/delta-io/delta-rs/pull/672
• Bump openssl-src from 111.21.0+1.1.1p to 111.22.0+1.1.1q by @dependabot in https://github.com/delta-io/delta-rs/pull/674
• Bump openssl-src from 111.20.0+1.1.1o to 111.22.0+1.1.1q in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/675
• fix: get docs building and add to CI checks by @wjones127 in https://github.com/delta-io/delta-rs/pull/679
• fix: omit common prefixes in azure list_objs by @roeap in https://github.com/delta-io/delta-rs/pull/683
• fix: traverse directories in local list_objs by @roeap in https://github.com/delta-io/delta-rs/pull/681
• Implement vacuum tests and general test setup utils by @Blajda in https://github.com/delta-io/delta-rs/pull/682
• Setup Github dependabot for Rust by @fvaleye in https://github.com/delta-io/delta-rs/pull/687
• Bump regex from 1.5.6 to 1.6.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/694
• Bump serde_json from 1.0.81 to 1.0.82 by @dependabot in https://github.com/delta-io/delta-rs/pull/692
• Bump serial_test from 0.7.0 to 0.8.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/699
• Bump hyper from 0.14.19 to 0.14.20 by @dependabot in https://github.com/delta-io/delta-rs/pull/702
• feat: sharable reference to storage backend by @roeap in https://github.com/delta-io/delta-rs/pull/697
• Bump openssl from 0.10.40 to 0.10.41 by @dependabot in https://github.com/delta-io/delta-rs/pull/700
• Get file size from Pyarrow directly (>= 9.0.0) by @Bernolt in https://github.com/delta-io/delta-rs/pull/704
• Bump bytes from 1.1.0 to 1.2.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/707
• Bump serde from 1.0.137 to 1.0.140 by @dependabot in https://github.com/delta-io/delta-rs/pull/708
• Bump crossbeam from 0.8.1 to 0.8.2 by @dependabot in https://github.com/delta-io/delta-rs/pull/709
• Bump tokio from 1.19.2 to 1.20.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/710
• Fix usage documentation in Python binding by @fvaleye in https://github.com/delta-io/delta-rs/pull/716
• Bump bytes from 1.2.0 to 1.2.1 by @dependabot in https://github.com/delta-io/delta-rs/pull/719
• Bump tokio from 1.20.0 to 1.20.1 by @dependabot in https://github.com/delta-io/delta-rs/pull/720
• Bump lambda_runtime from 0.3.0 to 0.6.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/711
• feat: integrate with object_store / datafusion APIs by @roeap in https://github.com/delta-io/delta-rs/pull/703
• Bump async-trait from 0.1.56 to 0.1.57 by @dependabot in https://github.com/delta-io/delta-rs/pull/730
• Bump serde from 1.0.140 to 1.0.142 by @dependabot in https://github.com/delta-io/delta-rs/pull/726
• Bump anyhow from 1.0.58 to 1.0.60 by @dependabot in https://github.com/delta-io/delta-rs/pull/727
• Bump libc from 0.2.126 to 0.2.127 by @dependabot in https://github.com/delta-io/delta-rs/pull/728
• Bump serde_json from 1.0.82 to 1.0.83 by @dependabot in https://github.com/delta-io/delta-rs/pull/731
• Bump thiserror from 1.0.31 to 1.0.32 by @dependabot in https://github.com/delta-io/delta-rs/pull/732
• Prune scanned files on column stats by @roeap in https://github.com/delta-io/delta-rs/pull/724
• Fix parsing null counts for struct type columns in the struct stats by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/714
• Python: fix: fix minimal test and bump minimum pyarrow version by @wjones127 in https://github.com/delta-io/delta-rs/pull/733
• Implement Python Schema in Rust by @wjones127 in https://github.com/delta-io/delta-rs/pull/684
• fix: Address clippy lint warnings. by @tsh56 in https://github.com/delta-io/delta-rs/pull/742
• Bump serde from 1.0.142 to 1.0.143 by @dependabot in https://github.com/delta-io/delta-rs/pull/737
• Bump libc from 0.2.127 to 0.2.132 by @dependabot in https://github.com/delta-io/delta-rs/pull/743
• build(deps): bump anyhow from 1.0.60 to 1.0.62 by @dependabot in https://github.com/delta-io/delta-rs/pull/744
• build(deps): bump serial_test from 0.8.0 to 0.9.0 by @dependabot in https://github.com/delta-io/delta-rs/pull/745
• build(deps): bump chrono from 0.4.20 to 0.4.22 by @dependabot in https://github.com/delta-io/delta-rs/pull/748
• build(deps): bump futures from 0.3.21 to 0.3.23 by @dependabot in https://github.com/delta-io/delta-rs/pull/747
• Cast min and max too when parsing stats by @wjones127 in https://github.com/delta-io/delta-rs/pull/753
• build(deps): bump serde_json from 1.0.83 to 1.0.85 by @dependabot in https://github.com/delta-io/delta-rs/pull/759
• build(deps): bump serde from 1.0.143 to 1.0.144 by @dependabot in https://github.com/delta-io/delta-rs/pull/760
• turn table state version into a private field by @houqp in https://github.com/delta-io/delta-rs/pull/772
• build(deps): bump pyo3 from 0.16.5 to 0.16.6 by @dependabot in https://github.com/delta-io/delta-rs/pull/773
• Remove Python version 3.6 support And run multiple python versions by @fvaleye in https://github.com/delta-io/delta-rs/pull/770
• parquet2 implementation backed by parquet2 feature gate by @houqp in https://github.com/delta-io/delta-rs/pull/465
• Adopt ObjectStore by @roeap in https://github.com/delta-io/delta-rs/pull/761
• chore: cleanup by @roeap in https://github.com/delta-io/delta-rs/pull/774
• Bump version of the Python binding to 0.6.0 by @fvaleye in https://github.com/delta-io/delta-rs/pull/762

New Contributors

• @mrk-its made their first contribution in https://github.com/delta-io/delta-rs/pull/670
• @xfrancois made their first contribution in https://github.com/delta-io/delta-rs/pull/665
• @Bernolt made their first contribution in https://github.com/delta-io/delta-rs/pull/704
• @tsh56 made their first contribution in https://github.com/delta-io/delta-rs/pull/742

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.5.8...python-v0.6.0

What's Changed

• Expose read and write options in public API by @george-zubrienko in https://github.com/delta-io/delta-rs/pull/581
• [proof] make sure lock at least expires once by @houqp in https://github.com/delta-io/delta-rs/pull/591
• Python API - delta.appendOnly enforcement by @WarSame in https://github.com/delta-io/delta-rs/pull/590
• Avoid building pandas and numpy from source by @wjones127 in https://github.com/delta-io/delta-rs/pull/595
• Introduce require_files for tracking the add files in table state by @mosyp in https://github.com/delta-io/delta-rs/pull/594
• Make sure pandas is optional by @wjones127 in https://github.com/delta-io/delta-rs/pull/597
• High level Delta Operations with Datafusion by @roeap in https://github.com/delta-io/delta-rs/pull/584
• Re-enable datafusion tests and improve supported types. by @roeap in https://github.com/delta-io/delta-rs/pull/601
• default to root for empty path in azure store by @roeap in https://github.com/delta-io/delta-rs/pull/603
• publish dynamodb_lock to crates.io by @houqp in https://github.com/delta-io/delta-rs/pull/605
• Configure Azure storage using a map (#555) by @Blajda in https://github.com/delta-io/delta-rs/pull/598
• Azure options by @roeap in https://github.com/delta-io/delta-rs/pull/606
• Update rusoto dependencies to 0.48 by @ahmedriza in https://github.com/delta-io/delta-rs/pull/611
• upgrade to datafusion 8 by @houqp in https://github.com/delta-io/delta-rs/pull/612
• fix: cap sphinx version to avoid bug in 5.0 by @wjones127 in https://github.com/delta-io/delta-rs/pull/615
• Provide Python aarch64 wheels for Linux. by @fvaleye in https://github.com/delta-io/delta-rs/pull/613
• Refactoring of the Python release Github action by @fvaleye in https://github.com/delta-io/delta-rs/pull/616
• fix: Use relative paths for add paths by @wjones127 in https://github.com/delta-io/delta-rs/pull/618
• Bin packing optimization by @Blajda in https://github.com/delta-io/delta-rs/pull/607
• feat: impl rename_noreplace with std::fs::hard_link by default by @wjones127 in https://github.com/delta-io/delta-rs/pull/621
• feat(python): validate schema in write_deltalake by @wjones127 in https://github.com/delta-io/delta-rs/pull/624
• Fix the AWS_REGION environment variable configuration in S3 backend by @fvaleye in https://github.com/delta-io/delta-rs/pull/633
• Refactor azure storage with crate updates by @roeap in https://github.com/delta-io/delta-rs/pull/644
• Defer creation of storage backend in DeltaTableBuilder by @Blajda in https://github.com/delta-io/delta-rs/pull/639
• fix: Add correct size and null paritition values to add actions by @wjones127 in https://github.com/delta-io/delta-rs/pull/625
• Bump flatbuffers from 0.8.4 to 2.1.2 in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/626
• Bump hyper from 0.14.9 to 0.14.19 in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/628
• Bump regex from 1.5.4 to 1.5.5 in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/629
• Bump regex from 1.5.4 to 1.5.6 in /delta-inspect by @dependabot in https://github.com/delta-io/delta-rs/pull/630
• Bump thread_local from 1.1.3 to 1.1.4 in /aws/delta-checkpoint by @dependabot in https://github.com/delta-io/delta-rs/pull/646
• fix: Prevent warning spam when reading tables generated by delta 1.2.1 by @Tom-Newton in https://github.com/delta-io/delta-rs/pull/651
• refactor: move version field to DeltaTableState by @roeap in https://github.com/delta-io/delta-rs/pull/649
• feat: add enforce_retention_duration param to vacuum method by @houqp in https://github.com/delta-io/delta-rs/pull/648
• fix: read vacuumed delta log without _last_checkpoint by @roeap in https://github.com/delta-io/delta-rs/pull/643
• feat: Upgrade to arrow/parquet 15 and datafusion 9 by @xianwill in https://github.com/delta-io/delta-rs/pull/652
• Release of the Python binding version 0.5.8 by @fvaleye in https://github.com/delta-io/delta-rs/pull/640

New Contributors

• @george-zubrienko made their first contribution in https://github.com/delta-io/delta-rs/pull/581
• @WarSame made their first contribution in https://github.com/delta-io/delta-rs/pull/590
• @dependabot made their first contribution in https://github.com/delta-io/delta-rs/pull/626
• @Tom-Newton made their first contribution in https://github.com/delta-io/delta-rs/pull/651

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.5.7...python-v0.5.8

What's Changed

• Upgrade DataFusion, Arrow, Parquet dependencies by @Dandandan in https://github.com/delta-io/delta-rs/pull/562
• fix clippy warnings by @houqp in https://github.com/delta-io/delta-rs/pull/567
• Azure improvements by @thovoll in https://github.com/delta-io/delta-rs/pull/556
• Update ADLSGen2-HOWTO.md by @dgcaron in https://github.com/delta-io/delta-rs/pull/560
• Parse partition values before handing to PyArrow by @wjones127 in https://github.com/delta-io/delta-rs/pull/565
• [Python] Test in minimal and latest Python environments by @wjones127 in https://github.com/delta-io/delta-rs/pull/572
• [Python] Initial PyArrow writer by @wjones127 in https://github.com/delta-io/delta-rs/pull/566
• Upgrade arrow, parquet, datafusion version by @zemelLeong in https://github.com/delta-io/delta-rs/pull/583
• Record Batch Writer by @roeap in https://github.com/delta-io/delta-rs/pull/573
• Replace table location prefix from s3a to s3 by @novakov-alexey in https://github.com/delta-io/delta-rs/pull/585
• Allow metadata for write_deltalake by @PadenZach in https://github.com/delta-io/delta-rs/pull/587
• Change private time_utils module to public. by @xianwill in https://github.com/delta-io/delta-rs/pull/586
• Release of the Python binding version 0.5.7 by @fvaleye in https://github.com/delta-io/delta-rs/pull/589

New Contributors

• @dgcaron made their first contribution in https://github.com/delta-io/delta-rs/pull/560
• @zemelLeong made their first contribution in https://github.com/delta-io/delta-rs/pull/583
• @novakov-alexey made their first contribution in https://github.com/delta-io/delta-rs/pull/585
• @PadenZach made their first contribution in https://github.com/delta-io/delta-rs/pull/587

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.5.6...python-v0.5.7

• Bump version of Python binding to 0.5.6 (#558)
• Move delta-inspect to its own crate (#557)
• Fix VACUUM by using table_uri when filtering files to delete (#551)
• Formally verify S3 atomic rename (#540)
• Implement missing Azure storage backend methods (#499)
• Implement polling for table updates (#550)
• Add target in Python release Github action workflow. (#548)

Credits:
QP Hou, Thomas Vollmer, David Blajda, Florian Valeye

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.5.5...python-v0.5.6

• Add storage options for backends (#544)
• Remove coupling of DynamoDbLockClient from S3 storage (#535)
• add macOS 11 support in python binding release (#541)
• Refresh Python usage documentation (#539)
• [Python] Create PyArrow dataset fragments from delta log (#525)
• Fix Delta metadata transaction schema (#531)
• Add gcs test and improve credential error (#533)
• Return complete history (#526)
• Move dynamodb lock into its own crate (#508)
• Add datafusion examples to docs (#519)
• Fix S3 list_objs and cleanup_metadata (#518)
• Add support for creating List and Map schema types (#517)
• Update datafusion version to 6 (#516)
• Retry S3 get request on 500 Internal Server Error (#510)
• Fix memory overhead when creating checkpoint (#502)
• Fix nullable partition values (#498)
• Fix cleanup_expired_logs timestamp (#503)
• Add bool config enableExpiredLogCleanup. (#500)
• pin arrow to major version (#501)

Credits:
Florian Valeye, ahmedriza, Will Jones, Liang-Chi Hsieh, Gabriel J. Michael, Matthew Turner, Mykhailo Osypov, Andrei Ionescu, QP Hou

Full Changelog: https://github.com/delta-io/delta-rs/compare/python-v0.5.4...python-v0.5.5

• Clean up expired delta table commit logs after checkpoint (#484)
• Add authorization options for azure storage backend (#486)
• Bump arrow to 6.1.0 (#494)
• Add DeltaTableError in Python binding. Add markers for integration tests with pytest. (#496)
• Change Rust edition from 2018 to 2021 (#490)
• Add docs for ADLS Gen2. (#492)
• Add gt, gte, lt and lte partition filters. (#478)
• Fix python build (#487)
• Try to fix flaky rename under Windows (#485)
• Update azure crates (#474)
• Update README.adoc (#482)
• Fix documentation for the DeltaStorageHandler (#483)
• Throw an error when filter key is not in partitioned columns. (#475)
• Add GCS feature to the Python Cargo.toml file (#476)
• Make file storage backend's atomic rename async (#471)
• materialize tables in python via native storage backend (#463)
• Fix coverage of the Python tests (#467)
• Support hash lookup by path string for Remove action (#462)
• Add new module for DeltaTableState (#464)
• Avoid table stats override in datafusion extension. (#459)
• Fix action reconciliation for add after remove (#456)
• Add pool_idle_timeout options for s3 and sts clients (#458)
• Generate new session name on assume role credentials provider refresh (#451)
• return lazy iterator in get tombstone methods (#452)
• Support no tombstone loading & new table builder API (#445)
• Fix broken tombstones metadata when extended_file_metadata is different between tomstones in state (#450)
• README: mark Checkpoint creation as done for Rust (#449)
• Add maturin develop command with extra (#448)
• Run all tests under s3 feature flag (#447)
• Update datafusion links (#446)
• Batch-apply remove actions in tombstone handling (#444)
• Fixing test to compare sorted vec (#443)
• Add delete_lock and fix release_lock (#440)

Credits:
Liang-Chi Hsieh, Robert Pack, Mykhailo Osypov, Florian Valeye, Thomas Vollmer, Yuan Zhou, roeap, Denny Lee, Yuan Zhou, Kelvin S. do Prado, QP Hou, Thomas Peiselt, Bruno Bigras, Akshay Ghiya

• Add history command in delta-rs (#428)
• reenable datafusion integration with temporary fork (#436)
• Decode path in Add and Remove actions. (#434)
• Optimize remove action apply with early iteration exit #424 (#431)
• Clean up DeltaTransactionError (#432)
• Add is_non_acquirable field to the dynamodb lock (#429)
• Expose valid primitive type list to public doc (#430)
• Support partition value string deserialization for timestamp/binary (#371)
• Bump arrow to 6.0.0-SNAPSHOT and bring map support to schema (#375)
• Update README.adoc (#426)
• Introduce DeltaConfig and tombstones retention policy (#420)
• Sync Action attributes with delta (#380)
• Add LICENSE file in the Python binding and refer it in the pyproject.toml (#422)
• Change checkpoint creation logs from info to debug (#423)
• Add the Glue Data Catalog for reading the DeltaTable (#419)
• Add S3StorageOptions to allow configuring S3 backend explicitly (#418)
• BUGFIX: writes to gcs must include the content length header
• Ensures that all table schemas are of StructType (#415)
• Fix reading nullable action fields from parquet (#417)
• Add filesystem argument for reading DeltaTable in Python binding (#414)
• Add implementation for load_with_datetime in Python package. (#411)
• Add a Makefile build task in the Python binding (#410)
• Use update_incremental in update (#398)
• Use tokio::fs::rename in put_obj. (#403)
• Update python readme (#406)
• Update pyproject definition in pyproject.toml (#405)
• Add examples for reading delta table with Rust API. (#400)
• Implement delete_objs in fs and s3 storage backends. (#395)
• Remove version param from create_checkpoint_from_table (#399)
• Google cloud storage backend (#355)
• added initial commit info on create method for a DeltaTable (#387)
• Upgrade to DataFusion 5.0 (#389)
• additional error handling to atomic_rename (#386)
• Reuse table/storage instances in checkpoints (#384)
• Add sts assume role creds for S3 (#383)
• Update datafusion and ballista links in README (#382)
• Merge Cargo.toml into pyproject.toml (#381)
• Implement consistent behavior in Windows with regard to swap parameter. (#379)
• Refactoring of black, isort, mypy tools usages into pyproject.toml (#378)
• Wrap DeltaTransactionError with DeltaTableError. (#374)
• Allow filesystem backend put_obj to overwrite existing (#376)
• Make Format.options to be required field (#370)
• Implement atomic put_obj. (#367)
• support partition value string deserialization for float/double/date (#363)
• Add '.tmp' suffix to temporary file of prepared commit (#366)
• cache cargo builds in CI (#359)

• new update_incremental API for streaming table update
• fix a bug in load_version method causing duplicated data @zijie0
• fix crash on table load caused by null partition value @zijie0
• support filtering on null partition value in table load predicate @zijie0

• added columns argument to to_pyarrow_table method to support projections on PyArrow Table conversion @zijie0
• added to_pandas shortcut method to convert a DeltaTable directly to pandas dataframe @bramrodenburg