Summary

This release adds some long-awaited features and adds support for excluding
paths, code blocks and nofollow targets from checking. Please find details below.

Major Changes

• Add support for excluding paths from link checking (#623) @mre
• Add support for URI remapping (#620) @mre
• Exclude code blocks from checking (#523) @mre
• Add support for comments in .lycheeignore (#616) @mre
• Add support for nofollow (#572) @mre
• Allow input URLs without scheme (fixes #567) @mre
• Also dump excluded links (#615) @mre
• Exclude example domains as per RFC 2606 from checking (#627)
• Set HTTP connect timeout and a TCP keepalive (#617) @mre
• Extract more status information from reqwest (#577) @mre
• Refactor response formatting (#599) @mre
• Add missing panic documentation (#561) @mre
• Skip Docker login on PRs from forks (#586) @MichaIng
• Enhance verbosity check (#578) @MichaIng
• Optimize plaintext extractor for small strings (#565) @mre
• Update scheme to sequence in example config (#576) @itamarhaber
• Regression: Fix cache verbosity issue (fixes #562) @mre
• Regression: Ignore invalid URLs (#571) @mre
• Update dependencies

Full Changelog: https://github.com/lycheeverse/lychee/compare/v0.9.0...v0.10.0

Thanks to all the testers, contributors, and maintainers for improving lychee! ❤️

Major Changes

This is a bigger release for lychee with new features, bug fixes, performance improvements,
cleaner output formatting, and better docs.

We moved to a new HTML engine, html5gum by @untitaker.
The old engine, html5ever is still supported by setting the
LYCHEE_USE_HTML5EVER=1 environment variable, but it will be deprecated soon,
so let us know if you run into issues with the new engine.

Previously it could happen that lychee panicked on invalid links. There is a PR for reqwest to fix the issue, but in the meantime we added a workaround to avoid the panic.

Some more highlights:

• lychee output is now less verbose to focus on what matters.
• We moved from hubcaps to octocrab as our Github API client.
• The Docker image now also supports Apple Silicon.
• Across the board, we improved link checking speed and reduced allocations.

Pull Requests

• Improve Github API usage by @mre in https://github.com/lycheeverse/lychee/pull/453
• Add caching functionality (v2) by @mre in https://github.com/lycheeverse/lychee/pull/443
• Simplify error messages by @mre in https://github.com/lycheeverse/lychee/pull/462
• Add more information about mail errors by @mre in https://github.com/lycheeverse/lychee/pull/463
• Fix bugs in extractor; reduce allocs by @mre in https://github.com/lycheeverse/lychee/pull/464
• Add missing Github exclusions; sort entries by @mre in https://github.com/lycheeverse/lychee/pull/473
• Add Alpine page for benchmark; refactor code by @mre in https://github.com/lycheeverse/lychee/pull/481
• lychee-bin: replace lazy_static by const_format by @lebensterben in https://github.com/lycheeverse/lychee/pull/495
• Add html5gum as alternative link extractor by @untitaker in https://github.com/lycheeverse/lychee/pull/480
• fix constant updating of progressbar by @untitaker in https://github.com/lycheeverse/lychee/pull/488
• Fix default config by @mre in https://github.com/lycheeverse/lychee/pull/491
• update rust-toolchain to .toml format by @lebensterben in https://github.com/lycheeverse/lychee/pull/496
• Update license files by @lebensterben in https://github.com/lycheeverse/lychee/pull/497
• lychee-lib::client reworked by @lebensterben in https://github.com/lycheeverse/lychee/pull/500
• Add Docker semver tagging for new multiarch image by @mre in https://github.com/lycheeverse/lychee/pull/503
• Multiarch for main Docker workflow by @mre in https://github.com/lycheeverse/lychee/pull/506
• replace hubcaps by octocrab by @lebensterben in https://github.com/lycheeverse/lychee/pull/502
• Use captured identifiers in format strings by @mre in https://github.com/lycheeverse/lychee/pull/507
• Replace plain String with SecretString for GitHub token by @lebensterben in https://github.com/lycheeverse/lychee/pull/509
• Prevent flaky tests by @mre in https://github.com/lycheeverse/lychee/pull/514
• Only login to Dockerhub on image push by @mre in https://github.com/lycheeverse/lychee/pull/515
• Disable Docker login for dependabot by @mre in https://github.com/lycheeverse/lychee/pull/519
• Error cleanup by @mre in https://github.com/lycheeverse/lychee/pull/510
• Improved IPv6 filtering support by @lebensterben in https://github.com/lycheeverse/lychee/pull/501
• Response output overhaul by @mre in https://github.com/lycheeverse/lychee/pull/524
• Fix link to documentation by @mre in https://github.com/lycheeverse/lychee/pull/527
• Update link to documentation by @mre in https://github.com/lycheeverse/lychee/pull/528
• Remove background from logo by @mre in https://github.com/lycheeverse/lychee/pull/526
• Make retry wait time configurable by @mre in https://github.com/lycheeverse/lychee/pull/525
• Print errors with a different format for easier clickability (#532) by @mre in https://github.com/lycheeverse/lychee/pull/533
• Fix broken pipe error on failing writes to stdout by @mre in https://github.com/lycheeverse/lychee/pull/535
• Print Github token error once at the end by @mre in https://github.com/lycheeverse/lychee/pull/537
• Add more comments to path methods by @mre in https://github.com/lycheeverse/lychee/pull/543
• Add Docker pulls badge by @mre in https://github.com/lycheeverse/lychee/pull/550
• Avoid reqwest panic on invalid URIs by @mre in https://github.com/lycheeverse/lychee/pull/557
• Update dependencies

Full Changelog: https://github.com/lycheeverse/lychee/compare/v0.8.2...v0.9.0

Thanks to all the testers, contributors, and maintainers for improving lychee! ❤️

• Improve concurrency with streams (#330)
• Add ability to recursively walk a directory for links (e.g. lychee .)
• Fix false-positives for special Github URLs (like /sponsors or /marketplace)
• Use tokenizer for extraction; add benchmark (#424)
• Max concurrency moved to check (#419)
• Add support for .lycheeignore file #308 (#402)
• Only print source in verbose mode (#400)
• Use thiserror for error handling (#399)
• Update docs and tests

• Add support for different output formats (compact, detailed, markdown) (#375)
• Fix clippy lint (#383) by @dcroote
• Add more known false positive schema domains (#376)
• Remove srcset attribute from list of "link" attrs (#393) by @untitaker
• Update dependencies

This is a major milestone for lychee, because we added support for checking local
files. To do so, you can now set a a directory as --base (e.g. /path/to/files) to check relative URLs.
Check the docs for more info.

Below is a list of all major changes in this version.

• Add support for local files (#262)
• Add new flag --require-https for checking if a HTTPS URL is available (#195)
• Add support for dumping links without checking them (#99)
• Don't exclude mail on --exclude-all-private (#316)
• Don't panic on unknown status codes (#234)
• Make inputs required (show help message if not provided) (#329)
• Silently ignore absolute paths without base (#338)
• Cache absolute_path to decrease allocations (#346)
• Imply "localhost" when loopback IPs are excluded (#351)
• Extract base from the source URL if --base is empty (#358)
• Optimize Dockerfile (#326)
• Add an image description to DockerHub (#325)
• Move to debian/bullseye:slim as base image (#370)
• Fix publish workflow (#309)
• Use cargo-publish-all for releases (#312)
• Add http://www.w3.org/2000/svg to known false positives (#359)
• Add support for async profiling using tokio-console (#318)
• Refactor extractor (#354)
• Filter out directories with suffixes that look like known extensions
• Avoid double-encoding already encoded destination paths
• Make error message for broken file links more understandable
• Refactoring (e.g. use Or-Patterns from Rust 1.53)
• Bump crate dependencies
• Update Makefile and CI pipelines

Thanks to the following amazing devs who contributed features to this release
and reviewed pull requests (alphabetically ordered):

• @dblock
• @fauust
• @jorgelbg
• @lebensterben
• @MichaIng
• @pawroman

• Add support for --exclude-file by @dblock. See #302.
• Simple URI cache: Repeated links are only checked once. See #243.
• Silently ignore unsupported schemes (e.g. slack://). See #199.
• Add support for multiple schemes (#214 and #237).
• Add missing Rust docs for lychee-lib (#231).
• Update dependencies.

Refactoring work by @lebensterben

• Major changes in lychee-lib::filter module:
  • Fields in Excludes except the RegexSet is now moved to Filter.
  • Filter contains Option<Excludes> and Option<Includes>, which are
    wrapper struct of RegexSet instead of Option<RegexSet>. As a result
    the code now looks cleaner.
  • Factored out some filtering logics to dedicated functions.
    • It's possible to write tests for those functions in addition to tests
      for the Filter struct.
  • Added docs to Filter::is_excluded and reorgnized the code.
• placed derive_builder by typed_builder:
  • The internal interface very ugly, as admitted by the author, but we no
    longer have nested Options like before.
  • As a result, the Client building is much easier to read.
  • Main benefit of typed_builder is, the arguments feeded to builder is
    checked at compile time instead of run-time.
• Fixed a bug in lychee::tests::usage and lychee-lib::stats::test.
  • Now it will clear environment variable which would otherwise cause an
    issue if GITHUB_TOKEN is set.
• Updated dependencies.

Major refactor

Fix broken email exclusion parameter

Pre-release to fix issues with excluding mails

We are blocked on upstream dependencies for a proper 0.6.0. See https://github.com/lycheeverse/lychee/issues/59#issuecomment-798677637.
In the meantime this is an alpha release to unblock some users. It includes the fix for skipping emails (#177) and other smaller fixes.

Pre-release to allow packaging for homebrew

• Switch to master branch of if-mail-exists to fully migrate to tokio 1.x.
• Explicitly use latest ring crate to allow compiling on Apple Silicon (required for homebrew)
• Detect and properly handle mailto: addresses
• Add support for excluding all e-mails from being checked
• Update dependencies

• Use html5ever for HTML link extraction (#98)
• Upgrade to tokio 1.0 and reqwest 0.11.0
• Add JSON output feature
• Test that cli parameters / usage are in sync in README.md (#95)
• Update other dependencies

• Find more URLs in HTML files (see #83)
• Run clippy for all targets, including tests
• Update dependencies

• Build binaries for Linux, Windows, and macOS for every release (#74)
• Publish releases on crates.io (#65)
• Publish releases on FreeBSD (@pizzamig)

• Add connection timeout
• Add e-mail checking support
• Support exponential backoff in check_real #1
• Add progress bar
• use action-rs for CI
• Add Uri::host_ip method and tests
• Implement exclude private URLs feature
• Introduce cargo fmt and clippy checks, fix all clippy warnings
• Add --exclude-all-private flag and cli integration test
• Add comparison table and recording of demo run (#17)
• Support relative URLs (#15)
• Configuration file (lychee.toml) (#16)
• Add support for include patterns (#23)
• Add support for basic auth #18 (#20)
• Make GITHUB_TOKEN optional (#22)
• Refactor and add documentation
• Use builder pattern and channels (fixes #12) (#33)
• Properly announce lychee user agent (#38)
• Cleanup unused depependencies
• Update dependencies
• Update documentation

• Make README.md the default when no inputs are given
• Add troubleshooting guide and .editorconfig (#60)
• Add support for reading from stdin and make input handling more robust
• Add official Docker image #40
• Mention official Docker image
• Revamp documentation