parquet-dotnet

What's Changed

• All compression methods are supported - Snappy, Gzip, LZO, Brotli, LZ4, ZSTD.
• ReadAsTableAsync: Return an empty table if there is no row by @dvasseur in https://github.com/aloneguid/parquet-dotnet/pull/186

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.0.2...4.1.0

What's Changed

• Fix ArgumentNullException during serialisation when schema data field… by @El-Gor-do in https://github.com/aloneguid/parquet-dotnet/pull/180
• Add RLE_DICTIONARY encoding support by @mukunku in https://github.com/aloneguid/parquet-dotnet/pull/181

New Contributors

• @El-Gor-do made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/180
• @mukunku made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/181

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.0.1...4.0.2

What's Changed

• Preserve ticks precision by @nilsgaeq in https://github.com/aloneguid/parquet-dotnet/pull/182

New Contributors

• @nilsgaeq made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/182

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.0.0...4.0.1

This release starts a new major version 4 for parquet-dotnet. Parquet interface becomes incompatible due to full switch to async API. You will see some performance improvements in high load scenarios, but performance in general will improve during v4.0 lifetime.

In addition to that, I have upgraded parquet.thrift to latest version and regenerated/fixed thrift contracts to enable progress and implementation of the newest Parquet features.

And:

What's Changed

• FIX: Check if Year is 0, set it to 1 then by @dvasseur in https://github.com/aloneguid/parquet-dotnet/pull/177

New Contributors

• @dvasseur made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/177

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/3.10.0...4.0.0

What's Changed

• fix typo in package description. by @MarkPflug in https://github.com/aloneguid/parquet-dotnet/pull/128
• Remove HasValueFlags array. by @MarkPflug in https://github.com/aloneguid/parquet-dotnet/pull/129
• Fix breaking change in v3.9 due to missing default fallback for Decimal Precision/Scale by @cajuncoding in https://github.com/aloneguid/parquet-dotnet/pull/132
• Fix issue #168 optionals without stats by @Teun in https://github.com/aloneguid/parquet-dotnet/pull/169
• 161 Reuse buffer for string encoding from ArrayPool by @stelmuhov in https://github.com/aloneguid/parquet-dotnet/pull/162
• Implement decimal validation according to spec by @ferenc-radius in https://github.com/aloneguid/parquet-dotnet/pull/157
• Enable support for nullable Boolean by @DBojsen in https://github.com/aloneguid/parquet-dotnet/pull/113
• Accept offset and count on array parameters by @aclemmensen in https://github.com/aloneguid/parquet-dotnet/pull/136
• Support LIsts for AWS Athena by @bdquig in https://github.com/aloneguid/parquet-dotnet/pull/150
• Corrected error message by @markpattison in https://github.com/aloneguid/parquet-dotnet/pull/155

New Contributors

• @MarkPflug made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/128
• @cajuncoding made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/132
• @Teun made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/169
• @stelmuhov made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/162
• @ferenc-radius made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/157
• @DBojsen made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/113
• @aclemmensen made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/136
• @markpattison made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/155

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/3.9.1...3.10.0

thread safety fix #125

Another release. It's 3.9 now, is it going to be 4.0 any time soon? I don't know!

Breaking Changes

"parq" tool removed as according to stats it's rarely used by anyone, and I don't have time to support it. If you want to view parquet files, try Parquet Viewer.

Community Contributions

• #116 Fix for NullReference-Exception in case of Int16, UnsignedShort and UnsignedByte. (thanks @nikolapeja6).
• #109 Add support for decimal scale and precision. (thanks @bdquig).
• #112 Preserve ticks during the conversion from NanoTime to DateTimeOffset (thanks @MaratFaskhiev).

Small Improvements

• csproj: removed NET14 directive - wasn't in use at all.
• explicit dependencies on System.Buffers and System.Memory were removed.
• dependency on System.Reflection.Emit.Lightweight is now only required for netstandard 2.0 and lower.
• added 2 new .NET Core explicit LTS targets - .NET Core 2.1 and .NET Core 3.1. For you it only means that if you are targeting one of those frameworks, you will have 2 less transitive nuget dependencies.
• using official parquet graphics.

Bugfix release. Note to users: parquet-dotnet is pretty much alive. It's actively used in quite a few projects and is pretty solid.

• Many thanks to @felipepessoto for fixing and issue in RunLengthBitPackingHybridValuesReader and adding support for dots in field names.
• Thanks to @DmitryBaranov1986 for improving native serialiser interface (#92).
• Thanks to @ishepherd for handling empty data pages correctly (#95) and also congrats on the first OSS contribution 💛
• @aloncatz is also a hero - fixed reading plain dictionary of zero length (#96).

various bugfixes

Changes:

• #58: Dictionary nulls multi page fix
• #57: Reading dictionary-encoded string columns with null values from multi-page parquet files yields misaligned data

This list of changes was auto generated.

Changes:

• #48: Parquet.Net Version 3.6.0 assembly not signed with a strong name

This list of changes was auto generated.

• Fixes Snappy encoding issues #53 #56.
• Snappy write speeds are up to x40 times faster comparing to v3.6.x

Huge thanks to @skyyearxp for amazing work on Snappy!

Changes:

• #21: New SNAPPY Implementation

This list of changes was auto generated.

Changes:

• e0fd30ca16e25f724104535c29d6f549dd538e93 Fix issue with reading and writing table with empty list field of structs (#50)
• 92a1d3c130f5c377cdd46e2064cbae8018ffe795 Update writing.md
• ee76e8babca9d0dce58e34b9842166cbbe614ee5 16 integer serialization (#45)
• b6ee39ef3d60b68818ac20c11140bc6e48c16bba Added public static IEnumerable<T[]> DeserializeGroups(Stream inpu… (#44)
• b125f1f423e16046ff55de9c02f30c2a185fccb8 Fix issue with repetition levels when having nested lists in a table (#43)
• b755350969217cd92305a5426a0ba9e7b14e6d99 Fix parsing of Decimal types

This list of changes was auto generated.