What's Changed

• "root" column by @aloneguid in https://github.com/aloneguid/parquet-dotnet/pull/218
• Support reading Parquet files with data page header version 2 by @dmishne in https://github.com/aloneguid/parquet-dotnet/pull/212
• support TIMESTAMP_MICROS in DateTimeOffsetDataTypeHandler by @dmishne in https://github.com/aloneguid/parquet-dotnet/pull/213

New Contributors

• @dmishne made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/212

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.2.1...4.2.2

What's Changed

• allow serialization of inherited properties by @DrewMcArthur in https://github.com/aloneguid/parquet-dotnet/pull/202

New Contributors

• @DrewMcArthur made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/202

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.2.0...4.2.1

Performance

This release is heavily inspired by performance optimisations but also brings breaking changes (see below) due to the work required to improve performance combined with general cleanup. Version 4.1 was not on the top of the list when it comes to performance which changes drastically in this release.

• Parquet data encoders are generally rewritten to make use of .NET's low-level Span and Memory API where possible.
• Code is much simpler.
• Many reads are heavily optimised!

Parquet.Net is now faster than ParquetSharp (a wrapper around native C++ implementation).

In summary, read performance is more or less the same (was much slower). But write performance is higher. For integers, this library is 5 times faster on write, and for floats it's about 6 times faster!

It's worth noting this is just the beginning, and performance work is only starting.

Breaking Changes

• Schema-related classes moved to the Parquet.Schema namespace.
• Attributes moved to the root Parquet namespace.
• Row-related classes moved to the Parquet.Rows namespace (from Parquet.Data.Rows).

Improvements:

• DateTime and TimeTimeOffset is now treated the same!

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.1.3...4.2.0

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.1.2...4.1.3

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.1.1...4.1.2

What's Changed

• Is this code correct/ by @jw56578 in https://github.com/aloneguid/parquet-dotnet/pull/187
• Fixed #188 - Reading full page from stream by @Sense545 in https://github.com/aloneguid/parquet-dotnet/pull/189
• Make ParquetWriter work with write-only streams by @Pansynchro-Admin in https://github.com/aloneguid/parquet-dotnet/pull/194

New Contributors

• @jw56578 made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/187
• @Sense545 made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/189
• @Pansynchro-Admin made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/194

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.1.0...4.1.1

What's Changed

• All compression methods are supported - Snappy, Gzip, LZO, Brotli, LZ4, ZSTD.
• ReadAsTableAsync: Return an empty table if there is no row by @dvasseur in https://github.com/aloneguid/parquet-dotnet/pull/186

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.0.2...4.1.0

What's Changed

• Fix ArgumentNullException during serialisation when schema data field… by @El-Gor-do in https://github.com/aloneguid/parquet-dotnet/pull/180
• Add RLE_DICTIONARY encoding support by @mukunku in https://github.com/aloneguid/parquet-dotnet/pull/181

New Contributors

• @El-Gor-do made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/180
• @mukunku made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/181

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.0.1...4.0.2

What's Changed

• Preserve ticks precision by @nilsgaeq in https://github.com/aloneguid/parquet-dotnet/pull/182

New Contributors

• @nilsgaeq made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/182

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/4.0.0...4.0.1

This release starts a new major version 4 for parquet-dotnet. Parquet interface becomes incompatible due to full switch to async API. You will see some performance improvements in high load scenarios, but performance in general will improve during v4.0 lifetime.

In addition to that, I have upgraded parquet.thrift to latest version and regenerated/fixed thrift contracts to enable progress and implementation of the newest Parquet features.

And:

What's Changed

• FIX: Check if Year is 0, set it to 1 then by @dvasseur in https://github.com/aloneguid/parquet-dotnet/pull/177

New Contributors

• @dvasseur made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/177

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/3.10.0...4.0.0

What's Changed

• fix typo in package description. by @MarkPflug in https://github.com/aloneguid/parquet-dotnet/pull/128
• Remove HasValueFlags array. by @MarkPflug in https://github.com/aloneguid/parquet-dotnet/pull/129
• Fix breaking change in v3.9 due to missing default fallback for Decimal Precision/Scale by @cajuncoding in https://github.com/aloneguid/parquet-dotnet/pull/132
• Fix issue #168 optionals without stats by @Teun in https://github.com/aloneguid/parquet-dotnet/pull/169
• 161 Reuse buffer for string encoding from ArrayPool by @stelmuhov in https://github.com/aloneguid/parquet-dotnet/pull/162
• Implement decimal validation according to spec by @ferenc-radius in https://github.com/aloneguid/parquet-dotnet/pull/157
• Enable support for nullable Boolean by @DBojsen in https://github.com/aloneguid/parquet-dotnet/pull/113
• Accept offset and count on array parameters by @aclemmensen in https://github.com/aloneguid/parquet-dotnet/pull/136
• Support LIsts for AWS Athena by @bdquig in https://github.com/aloneguid/parquet-dotnet/pull/150
• Corrected error message by @markpattison in https://github.com/aloneguid/parquet-dotnet/pull/155

New Contributors

• @MarkPflug made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/128
• @cajuncoding made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/132
• @Teun made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/169
• @stelmuhov made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/162
• @ferenc-radius made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/157
• @DBojsen made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/113
• @aclemmensen made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/136
• @markpattison made their first contribution in https://github.com/aloneguid/parquet-dotnet/pull/155

Full Changelog: https://github.com/aloneguid/parquet-dotnet/compare/3.9.1...3.10.0

thread safety fix #125

Another release. It's 3.9 now, is it going to be 4.0 any time soon? I don't know!

Breaking Changes

"parq" tool removed as according to stats it's rarely used by anyone, and I don't have time to support it. If you want to view parquet files, try Parquet Viewer.

Community Contributions

• #116 Fix for NullReference-Exception in case of Int16, UnsignedShort and UnsignedByte. (thanks @nikolapeja6).
• #109 Add support for decimal scale and precision. (thanks @bdquig).
• #112 Preserve ticks during the conversion from NanoTime to DateTimeOffset (thanks @MaratFaskhiev).

Small Improvements

• csproj: removed NET14 directive - wasn't in use at all.
• explicit dependencies on System.Buffers and System.Memory were removed.
• dependency on System.Reflection.Emit.Lightweight is now only required for netstandard 2.0 and lower.
• added 2 new .NET Core explicit LTS targets - .NET Core 2.1 and .NET Core 3.1. For you it only means that if you are targeting one of those frameworks, you will have 2 less transitive nuget dependencies.
• using official parquet graphics.

Bugfix release. Note to users: parquet-dotnet is pretty much alive. It's actively used in quite a few projects and is pretty solid.

• Many thanks to @felipepessoto for fixing and issue in RunLengthBitPackingHybridValuesReader and adding support for dots in field names.
• Thanks to @DmitryBaranov1986 for improving native serialiser interface (#92).
• Thanks to @ishepherd for handling empty data pages correctly (#95) and also congrats on the first OSS contribution 💛
• @aloncatz is also a hero - fixed reading plain dictionary of zero length (#96).

various bugfixes