crates-index-diff-rs

New Features

• use github fast path to check for changes before git pull

Chore (BREAKING)

• upgrade gix to v0.67

Commit Statistics

• 4 commits contributed to the release.
• 107 days passed between releases.
• 2 commits were understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Merge branch 'upgrade-gix' (76b2bdb)
  • Upgrade gix to v0.67 (3ea251d)
  • Merge pull request #47 from syphar/github-fast-path (1ada2f1)
  • Use github fast path to check for changes before git pull (3bd1c11)

Upgrade gix to 0.63 and crates-index to use the same version of gix that contain security fixes.

Commit Statistics

• 2 commits contributed to the release.
• 85 days passed between releases.
• 0 commits were understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Set version number for new release and prepare changelog. (f4f4592)
  • Upgrade gix, gix-testtools & crates-index (4419a58)

Chore (BREAKING)

• upgrade gix to v0.62

Commit Statistics

• 2 commits contributed to the release.
• 78 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Merge branch 'gix-upgrade' (9bf5d93)
  • Upgrade gix to v0.62 (6377d3d)

Chore (BREAKING)

• upgrade gix to 0.58

Commit Statistics

• 1 commit contributed to the release.
• 29 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Upgrade gix to 0.58 (7044076)

A maintenance release without any API changes. The major version bump is just to signal the change
of the gix dependency to version v0.57.

Chore (BREAKING)

• update gix to v0.57

Commit Statistics

• 5 commits contributed to the release.
• 53 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Prepare changelog prior to release (98f24fe)
  • Merge branch 'maintenance' (2170563)
  • Turns out that using the resource cache is too costly (ad50624)
  • Remove git-lfs (2206723)
  • Update gix to v0.57 (bc3a8a2)

Chore

• upgrade gix to 0.55.2

New Features

• deterministic order if multiple changes are applied in one modification.
  This can happen if a crate is yanked and unyanked in one commit, which happens
  in practice even though I am not sure how it's possible.

  Related issue: https://github.com/rust-lang/docs.rs/issues/2295

Other

• improve docs and be more clear about ordering.

Commit Statistics

• 5 commits contributed to the release.
• 41 days passed between releases.
• 3 commits were understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Merge branch 'improvements' (28f0d1f)
  • Upgrade gix to 0.55.2 (e8b2c90)
  • Deterministic order if multiple changes are applied in one modification. (c404f67)
  • Assure gix versions are synced in the entire tree. (8576bc7)
  • Improve docs and be more clear about ordering. (7957987)

New Features (BREAKING)

• upgrade to gix v0.54 (from v0.50)
  This may produce smaller binaries.

  Further, no performance option is set anymore to leave this in the hands of
  the final binary.

Commit Statistics

• 6 commits contributed to the release over the course of 83 calendar days.
• 83 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Upgrade to gix v0.54 (from v0.50) (34d3510)
  • Merge branch 'uprade-crates-index' (922e962)
  • Upgrade to latest version of crates-index to remove git2 (fa1b21c)
  • Merge branch 'test-version-delete' (e200a56)
  • Add a test specifically for the VersionDeleted variant. (959c566)
  • Refactor (b0f84d0)

Chore

• uprgade gix to v0.48

Bug Fixes (BREAKING)

• split Change::Delete into Change::VersionDeleted and Change::CrateDeleted.
  This resolves an ambiguity (facilitated by incorrect documentation) that would emit
  Change::Delete when a single version was deleted, even though downstream that would
  rightfully be interpreted as deleting the whole crate.

  Adimittedly, previously there were no single-version deletions, just yanks, so we didn't
  think of it either.

Commit Statistics

• 4 commits contributed to the release over the course of 10 calendar days.
• 11 days passed between releases.
• 2 commits were understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Split Change::Delete into Change::VersionDeleted and Change::CrateDeleted. (2c59dd4)
  • Uprgade gix to v0.48 (9946196)
  • Split VersionDeleted from crate-deletes (2caf814)
  • Fix stress-test invocation (967836b)

Chore (BREAKING)

• upgrade to gix v0.47 (from v0.45).
  This includes integration with gix negotiation algorithms and support for
  multi-round negotiation just like git does it.

Commit Statistics

• 2 commits contributed to the release over the course of 13 calendar days.
• 13 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Upgrade to gix v0.47 (from v0.45). (f726287)
  • Try to fix the baseline test which now starts failing on CI (892561a)

Chore (BREAKING)

• upgrade dependencies
  • gix 0.44 -> 0.45
  • hashbrown 0.13 -> 0.14

Commit Statistics

• 1 commit contributed to the release.
• 25 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Upgrade dependencies (7427603)

Chore (BREAKING)

• upgrade gix from v0.37 to v0.44.
  Note that there is no breaking change, but the major version bump is merely
  for safety and to make it easy to reatain a certain gix version within the tree.

Commit Statistics

• 3 commits contributed to the release.
• 82 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Upgrade gix from v0.37 to v0.44. (b00ca99)
  • Reduce code-size by reusing the same type for Progress. (c3d5518)
  • Upgrade gix dependency to 0.44.1 (4489ef7)

Bug Fixes

• make DependencyKind publicly accessible
  Previously it was used as field within an exported type, but it couldn't
  be used directly.

Commit Statistics

• 1 commit contributed to the release.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • make DependencyKind publicly accessible (8d7b7b6)

Bug Fixes (BREAKING)

• upgrade from git-repository to gix
  This change renames the public re-export from git to gix as well
  for consistency.

Commit Statistics

• 1 commit contributed to the release.
• 77 days passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • upgrade from git-repository to gix (8082588)

Bug Fixes

• Index::peek_changes_ordered() now actually provides ordered changes.

Commit Statistics

• 1 commit contributed to the release.
• 1 day passed between releases.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • Index::peek_changes_ordered() now actually provides ordered changes. (44a5579)

Bug Fixes

• correctly configure 'serde' features of dependencies.
  Due to surprising an to me still quite unclear behaviour cargo
  will turn on the serde features in the hex and smartstring
  dependencies when building it locally from a repository clone,
  but won't do it when building the crates-io version of the crate
  after adding it as dependency.

  This fixes the issue by correctly configuring it the serde
  feature.

Commit Statistics

• 4 commits contributed to the release over the course of 3 calendar days.
• 3 days passed between releases.
• 1 commit was understood as conventional.
• 1 unique issue was worked on: #33

Commit Details

• #33
  • correctly configure 'serde' features of dependencies. (c915992)
• Uncategorized
  • Merge branch 'fix-build' (57ca8f0)
  • a failing test to reproduce #33 (8bfd04d)
  • fix stress test (9afe15b)

This release adds support for obtaining changes in the correct order by automatically looking at changes one commit at a time
while handling squashed indices gracefully. It take about 20 seconds to catch up with all commits done in the crates index for
2 whole days and takes much longer than a similar unorderd acquisition of changes, but should be well worth it in situations
where changes are fetched more often than that.

The baseline validation to assure correctness was improved to also assert the yanked state of crates. Furthermore, it is
regularly run by CI, on the real crates-index, and in a more realistic manner mixing both unordered and ordered change requests.

The API is mostly the same, but has a few breaking changes to deal with order selection.

Last but not least, the user can now configure the HTTP backend to use, which allows for a pure-Rust
build as well.

Bug Fixes

• allow to configure the http transport layer in git-repository.
  This way the user of the library has a choice to activate any of these
  features:

  • http-curl (default) - use curl and a lot of C dependencies for HTTP/HTTPS.
    But also support advanced HTTP options.
  • http-request - use reqwest for http and https, which currently doesn't
    support any options.

Chore

• a pipeline to validate basic assumptions are stil fulfilled.
  Running stress-test like baseline tests regularly should help us
  assure that crates-index-diff operates as it should against a
  real crates-index.

New Features

• baseline validation now validates ordered and unordered mode.
• greatly improve performance and realism of baseline-atomic test.
  We now set a fixed amount of 'big' steps along with one of those chunks
  being a range where the step-size is one commit at a time, which
  might be the way changes are obtained in the future.
• baseline also validates the yanked status.
  That way we assure that the state we are interested in is indeed
  communicated perfectly through the events we emit.

New Features (BREAKING)

• Support for in-order changes via changes_between_ancestor_commits().
  This improvement also makes available an enum to select Order
  in higher-level methods like peek_changes_with_options().

  We also add peek_changes_ordered() and changes_ordered() for convenience.

Commit Statistics

• 8 commits contributed to the release.
• 6 commits were understood as conventional.
• 1 unique issue was worked on: #30

Commit Details

• #30
  • allow to configure the http transport layer in git-repository. (c3cff88)
  • prepare changelog (b887560)
  • baseline validation now validates ordered and unordered mode. (87e49b5)
  • greatly improve performance and realism of baseline-atomic test. (81c6dd2)
  • baseline also validates the yanked status. (4dd4a4c)
• Uncategorized
  • Merge branch 'baseline-improvements' (a80c7fa)
  • Support for in-order changes via changes_between_ancestor_commits(). (133f2f5)
  • a pipeline to validate basic assumptions are stil fulfilled. (d91afc9)

A massive release that increases performance up to 10x for diffing repositories[^1] and correctness at the same time. This release
wouldn't have been possible without the herculean efforts of Pascal Kuthe and I am grateful
for his contribution. Thank you!

[^1]: Needs to build with git-repository/max-performance and setup a pack cache, for example with GITOXIDE_PACK_CACHE_MEMORY=1g <you-application>

Other

• try to rewrite delegate to be map based…
  …but besides completely failing the normalization test which I don't
  understand, it also doesn't manage to get the correct amount of
  versions.
• try to chunk up baseline, but changes do not line up.
  When stepping through the changes in multiple steps, we end up with
  more crates then there are even though we identify them by
  checksum and consider deletions. Yanking doesn't remove them from
  the iteration either.

New Features

• a max-performance feature to tune git-repository.
  The performance difference is rather drastic at about 2.5x, and
  definitely worth having if there is no compatibility issue
  due to shared C dependencies in the same binary.

  Additionally we setup the makefile to use big object caches
  to avoid having to decompress the same object too often, accelerating
  the diffing process about 4x, for a total of 10x performance boost.

Changed (BREAKING)

• Change::Deleted variant now has versions field to include all deleted versions.
  That way it doesn't degenerate any information, previously the exact
  version information was lost.

  Not doing so helps to be able to reproduce the current state by
  aggregating all changes.

New Features (BREAKING)

• Reduce heap-allocations CrateVersion type and sub-types.
  This improves performance slightly when dealing with a lot of versions,
  like when all versions are obtained from the beginning of time.

Commit Statistics

• 21 commits contributed to the release over the course of 3 calendar days.
• 3 days passed between releases.
• 5 commits were understood as conventional.
• 1 unique issue was worked on: #26

Thanks Clippy

Clippy helped 1 time to make code idiomatic.

Commit Details

• #26
  • refactor (b42ac1e)
  • revert previous to FAILed commits (3b52cfd)
  • try to rewrite delegate to be map based… (4d53b04)
  • try to chunk up baseline, but changes do not line up. (c0c01bb)
  • refactor (097209c)
  • Now the baseline result is the same. (02cdb2e)
  • make baseline work with CI (b9a1850)
  • a baseline test which shows that we cannot reproduce the status quo with changes just yet. (3fcf96b)
  • Change::Deleted variant now as versions to include all deleted versions. (b538ad6)
  • Reduce heap-allocations CrateVersion type and sub-types. (f9be536)
  • layout baseline for exhaustive test against the latest available index (7e9d3cd)
• Uncategorized
  • prepare changelog prior to release (a93ba40)
  • Merge branch 'fix-diff' (ec9842a)
  • refactor (bd3bc22)
  • remove unnecessary unsafe code (1b5684f)
  • a max-performance feature to tune git-repository. (6f5b12a)
  • improve baseline tests to be more practical (bae80b0)
  • add baseline tests that steps trough each commit individually (a377ca4)
  • perform an unordered comparison instead of using a linear edit-sequence (8256cbb)
  • Merge branch 'complete-baseline' (61c6272)
  • thanks clippy (249d141)

Bug Fixes

• assure differences are handled exhaustively.
  Previously it was possible to have multiple diffs in one crate
  distributed over multiple commits to rightfully show up as multiple
  hunks of modified and added lines only register the modified lines,
  not the new ones (or the deleted ones for that matter).

  This would cause updates or removals to be missed.

  Now hunks of changes are exhaused properly, fixing the issue.

Commit Statistics

• 8 commits contributed to the release.
• 9 days passed between releases.
• 1 commit was understood as conventional.
• 1 unique issue was worked on: #26

Thanks Clippy

Clippy helped 1 time to make code idiomatic.

Commit Details

• #26
  • prepare changelog prior to release (46d94b5)
  • refactor (87678db)
  • thanks clippy (425fd57)
  • refactor (d059360)
  • assure differences are handled exhaustively. (51c5109)
  • reproduce issue (69c8f43)
  • Add test fixtures for reproduction (462d44f)
• Uncategorized
  • Merge branch 'fix-26' (7ea3d6e)

Features

• Switch diff implementation to imara-diff for performance, but also to get off the broken git-repository v0.25 release.

Commit Statistics

• 9 commits contributed to the release.
• 28 days passed between releases.
• 0 commits were understood as conventional.
• 0 issues like '(#ID)' were seen in commit messages

Commit Details

• Uncategorized
  • prepare changelog prior to release (adecda5)
  • Merge branch 'git-repository-upgrade' (b28b7ca)
  • upgrade to git-repository v0.27 (377065e)
  • all tests are green (6b47427)
  • one more test bytes the dust, but… (62cb6eb)
  • fix most of the tests (276e726)
  • A first stab at getting the diff back to work with imara-diff (5573529)
  • try to rewrite explicit trait impl as closure, same lifeimte issues. (2809c75)
  • upgrade to git-repoitory v0.26, fail to make diff compile. (3f47ec3)

Bug Fixes

• rename init::Error2 to init::Error.
  This name was left by mistake and shouldn't have made it into the
  release.

Commit Statistics

• 1 commit contributed to the release.
• 1 commit was understood as conventional.
• 0 issues like '(#ID)' where seen in commit messages

Commit Details

• Uncategorized
  • rename init::Error2 to init::Error. (a6975ce)