datacompy

Breaking changes

This release has some breaking changes users should be aware of.

• For the new Pandas on Spark implementation only Pandas 1.5.3 and below will be supported due to Pandas 2.0 compatibility issues with Spark 3.
  • Users wishing to use the older SparkCompare can continue to do so using LegacySparkCompare
• Spark 3.1.x support will be dropped
• Python 3.8 support is dropped

What's Changed

• Pandas on Spark refactor by @fdosani in https://github.com/capitalone/datacompy/pull/275
• Fix suffix on all mismatches reports by @fdosani in https://github.com/capitalone/datacompy/pull/293
• Update pandas support to 2.2.2 by @mark-thm in https://github.com/capitalone/datacompy/pull/295
• bump pandas actions to 2.2.2 by @fdosani in https://github.com/capitalone/datacompy/pull/296
• Fugue count matching rows by @fdosani in https://github.com/capitalone/datacompy/pull/294
• Release v0.12.0 by @fdosani in https://github.com/capitalone/datacompy/pull/297

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.11.3...v0.12.0

What's Changed

• non-join column unicode fix by @fdosani in https://github.com/capitalone/datacompy/pull/285
• Fix callout in readme by @jdawang in https://github.com/capitalone/datacompy/pull/287
• Release v0.11.3 by @fdosani in https://github.com/capitalone/datacompy/pull/286

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.11.2...v0.11.3

What's Changed

• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/279
• Unicode column names fix by @fdosani in https://github.com/capitalone/datacompy/pull/281
• Release v0.11.2 by @fdosani in https://github.com/capitalone/datacompy/pull/282

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.11.1...v0.11.2

What's Changed

• bug fix when all columns match but no rows match by @fdosani in https://github.com/capitalone/datacompy/pull/277
• Release v0.11.1 by @fdosani in https://github.com/capitalone/datacompy/pull/278

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.11.0...v0.11.1

What's Changed

• ✨ Add mypy for static type checking and type annotations by @aguiddir in https://github.com/capitalone/datacompy/pull/249
• Test with and without spark by @jdawang in https://github.com/capitalone/datacompy/pull/252
• Native Polars support by @fdosani in https://github.com/capitalone/datacompy/pull/258
• copyright updating to 2024 by @fdosani in https://github.com/capitalone/datacompy/pull/261
• python 3.11 support by @fdosani in https://github.com/capitalone/datacompy/pull/263
• Move pandas upper bound to 2.2.0 by @mark-thm in https://github.com/capitalone/datacompy/pull/265
• Edgetest fix by @fdosani in https://github.com/capitalone/datacompy/pull/266
• Move numpy upper bound to 1.26.4 by @mark-thm in https://github.com/capitalone/datacompy/pull/270
• edgetest sparkfix by @fdosani in https://github.com/capitalone/datacompy/pull/269
• Release v0.11.0 by @fdosani in https://github.com/capitalone/datacompy/pull/271

New Contributors

• @aguiddir made their first contribution in https://github.com/capitalone/datacompy/pull/249
• @jdawang made their first contribution in https://github.com/capitalone/datacompy/pull/252
• @mark-thm made their first contribution in https://github.com/capitalone/datacompy/pull/265

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.10.5...v0.11.0

What's Changed

• Point GHA to edgetest-dev/run-edgetest-action by @gliptak in https://github.com/capitalone/datacompy/pull/241
• Fix code to work with Fugue 0.8.7 by @goodwanghan in https://github.com/capitalone/datacompy/pull/245
• adding all_rows_overlap to fugue by @fdosani in https://github.com/capitalone/datacompy/pull/244
• Release v0.10.5 by @fdosani in https://github.com/capitalone/datacompy/pull/246

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.10.4...v0.10.5

What's Changed

• Update CODEOWNERS by @fdosani in https://github.com/capitalone/datacompy/pull/234
• fix for logging intersect rows number by @BrainLogic in https://github.com/capitalone/datacompy/pull/236
• bump numpy to 1.26.0 by @fdosani in https://github.com/capitalone/datacompy/pull/238
• bump version by @fdosani in https://github.com/capitalone/datacompy/pull/239
• Release v0.10.4 by @fdosani in https://github.com/capitalone/datacompy/pull/240

New Contributors

• @BrainLogic made their first contribution in https://github.com/capitalone/datacompy/pull/236

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.10.3...v0.10.4

What's Changed

• adding in unq_columns and intersect_columns for Fugue by @fdosani in https://github.com/capitalone/datacompy/pull/217
• adding in all_columns_match by @fdosani in https://github.com/capitalone/datacompy/pull/219
• Migrate to pyproject.toml from setup.cfg by @NikhilJArora in https://github.com/capitalone/datacompy/pull/229
• Fix fugue empty partition bug by @goodwanghan in https://github.com/capitalone/datacompy/pull/220
• bump version 0.10.3 by @fdosani in https://github.com/capitalone/datacompy/pull/231
• Release v0.10.3 by @fdosani in https://github.com/capitalone/datacompy/pull/232

New Contributors

• @NikhilJArora made their first contribution in https://github.com/capitalone/datacompy/pull/229

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.10.2...v0.10.3

What's Changed

• Support for Pandas 2+ by @fdosani in https://github.com/capitalone/datacompy/pull/212
• bump version by @fdosani in https://github.com/capitalone/datacompy/pull/215
• Release v0.10.2 by @fdosani in https://github.com/capitalone/datacompy/pull/216

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.10.1...v0.10.2

What's Changed

• bump version by @fdosani in https://github.com/capitalone/datacompy/pull/210
• Release v0.10.1 by @fdosani in https://github.com/capitalone/datacompy/pull/211

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.10.0...v0.10.1

What's Changed

• Add Fugue support (Phase 1) by @goodwanghan in https://github.com/capitalone/datacompy/pull/201
• Add Fugue implementation of report by @goodwanghan in https://github.com/capitalone/datacompy/pull/208
• fixes #204 by @fdosani in https://github.com/capitalone/datacompy/pull/205
• Release v0.10.0 by @fdosani in https://github.com/capitalone/datacompy/pull/209

New Contributors

• @goodwanghan made their first contribution in https://github.com/capitalone/datacompy/pull/201

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.9.0...v0.10.0

What's Changed

• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/184
• Replace called_with with assert_called_with by @fdosani in https://github.com/capitalone/datacompy/pull/183
• Minimum Python version set to 3.8 by @fdosani in https://github.com/capitalone/datacompy/pull/186
• Update code with f-strings by @pronoym99 in https://github.com/capitalone/datacompy/pull/185
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/187
• bumping up to v0.9.0 by @fdosani in https://github.com/capitalone/datacompy/pull/191
• Convert all docs to markdown by @Vaporjawn in https://github.com/capitalone/datacompy/pull/198
• fixes #147 by @fdosani in https://github.com/capitalone/datacompy/pull/200
• Release v0.9.0 by @fdosani in https://github.com/capitalone/datacompy/pull/202

New Contributors

• @pronoym99 made their first contribution in https://github.com/capitalone/datacompy/pull/185
• @Vaporjawn made their first contribution in https://github.com/capitalone/datacompy/pull/198

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.8.4...v0.9.0

What's Changed

• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/176
• Bug fix for #177 by @fdosani in https://github.com/capitalone/datacompy/pull/178
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/180
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/181
• Release v0.8.4 by @fdosani in https://github.com/capitalone/datacompy/pull/179

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.8.3...v0.8.4

What's Changed

• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/168
• updating download badge by @fdosani in https://github.com/capitalone/datacompy/pull/169
• bump version to v0.8.3 by @fdosani in https://github.com/capitalone/datacompy/pull/171
• Update ROADMAP.rst by @fdosani in https://github.com/capitalone/datacompy/pull/174
• Release v0.8.3 by @fdosani in https://github.com/capitalone/datacompy/pull/172

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.8.2...v0.8.3

What's Changed

• WhiteSource Configuration Migration by @mend-for-github-com in https://github.com/capitalone/datacompy/pull/145
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/149
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/150
• Branch updates and HTML Report by @fdosani in https://github.com/capitalone/datacompy/pull/152
• Edgetest update by @fdosani in https://github.com/capitalone/datacompy/pull/153
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/156
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/158
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/161
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/162
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/164
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/165
• To address issue #42, added a parameter to the all_mismatch method to control the addition of matching columns to the output. by @azadekhalaj in https://github.com/capitalone/datacompy/pull/166
• Release v0.8.2 by @fdosani in https://github.com/capitalone/datacompy/pull/167

New Contributors

• @mend-for-github-com made their first contribution in https://github.com/capitalone/datacompy/pull/145
• @azadekhalaj made their first contribution in https://github.com/capitalone/datacompy/pull/166

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.8.1...v0.8.2

What's Changed

• Create publish-package.yml by @fdosani in https://github.com/capitalone/datacompy/pull/130
• adding edgetest action by @fdosani in https://github.com/capitalone/datacompy/pull/131
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/132
• Create whitesource.config by @fdosani in https://github.com/capitalone/datacompy/pull/138
• Changes by run-edgetest action by @github-actions in https://github.com/capitalone/datacompy/pull/137
• updating whitesource config by @fdosani in https://github.com/capitalone/datacompy/pull/139
• bump version to 0.8.1 by @fdosani in https://github.com/capitalone/datacompy/pull/140
• Release 0.8.1 by @fdosani in https://github.com/capitalone/datacompy/pull/141

New Contributors

• @github-actions made their first contribution in https://github.com/capitalone/datacompy/pull/132

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.8.0...v0.8.1

What's Changed

• Adding edgetest, general house keeping with GitHub Actions, and dropping Python 3.6 testing/support

GitHub Action updates:

• #119
• #120
• #126

Edgetest updates:

• #125

Full Changelog: https://github.com/capitalone/datacompy/compare/v0.7.3...v0.8.0

Release v0.7.3:

• Make column order in reports determinitistic (alphabetical) (#107)
• Use OrderedSet to maintain DataFrame column order throughout comparison set operations (#108)
• General house keeping (#109, #110, #111, #113)
• adding column count (#116)

Release v0.7.2:

Updates to CODEOWNERS (#80, #86)
Bumped min pandas version to 0.25 (#92)
Move from Travis to GH Actions (#90, #91, #96)
Doc cleanup (#98, #100)
Unit test cleanup (#99)
Suppress "Sample Rows" section in the output report (#102)

Release v0.7.1

• Lower casing columns and casting as string (#69)
• Adding whitesource config (#70)