New Features

• Configurable override of url generation for External Tool versions (Cherry-pick of #20074) (#20100)

Plugin API Changes

• Plumb the hidden _find_links through non-resolve Pex invocations (Cherry-pick of #20085) (#20095)

Bug Fixes

• Fix coursier download for arm64, bumps default to v2.1.6 (Cherry-pick of #19940) (#20092)

• Add best-effort limits on async file opens to reduce file handle counts (Cherry-pick of #20055) (#20077)

New Features

• Optionally retry failed tests (#19760)

• Create system-binaries subsystem (#19377)

User API Changes

• Upgrade Pex to 2.1.148. (#20048)

• Deprecate the platforms field on pex_binary. (#20012)

Bug Fixes

• Fix protoc support on M1 mac (#20058)

• Fix algorithm for gathering Go build requests with coverage. (#20030)

• docker_image: The image_tags field must not be empty. (#19980)

Documentation

• doc: Some guidance on auto-fixing goals in CI (#20040)

• docs: remove reference to use_deprecated_directory_cli_args_semantics (#20041)

• added a new section in Using Pants in CI on partial clones (#20028)

• Document how to build cross-platform pexes. (#20008)

• Specify pants as running with cp3.9 only in plugin docs (#20003)

• docs: Remove pants 2.15 example configuration. (#19995)

• Replace maintainer release duty with MOTW in docs (#19987)

Bug Fixes

• Fix algorithm for gathering Go build requests with coverage. (Cherry-pick of #20030) (#20031)

• docker_image: The image_tags field must not be empty. (Cherry-pick of #19980) (#19990)

Documentation

• added a new section in Using Pants in CI on partial clones (Cherry-pick of #20028) (#20029)

• Specify pants as running with cp3.9 only in plugin docs (Cherry-pick of #20003) (#20005)

• docs: Remove pants 2.15 example configuration. (Cherry-pick of #19995) (#19998)

• Replace maintainer release duty with MOTW in docs (Cherry-pick of #19987) (#19992)

Bug fixes

• Fix algorithm for gathering Go build requests with coverage. (Cherry-pick of #20030) (#20032)

Bug Fixes

• Fix algorithm for gathering Go build requests with coverage. (Cherry-pick of #20030) (#20033)

• docker_image: The image_tags field must not be empty. (Cherry-pick of #19980) (#19991)

Documentation

• docs: Remove pants 2.15 example configuration. (Cherry-pick of #19995) (#19997)

Bug Fixes

• Support using __defaults__ in same BUILD file as environment targets. (Cherry-pick of #19446) (#19969)

• Fetch shellcheck from vscode-shellcheck for M1 support (Cherry-pick of #19945) (#19952)

• Immediately invalidate written files in write_digest (Cherry-pick of #19903) (#19949)

• Fix "failed to create hardlink" error due to multiple mounts on the same device (Cherry-pick of #19894) (#19914)

• Add missing __init__.py file for the yamllint backend. (Cherry-pick of #19899) (#19906)

Documentation

• Add Tom and Gautham to the Contributors page. (Cherry-pick of #19955) (#19962)

• Remove references to PyPI/pantsbuild.pants wheels in docs (Cherry-pick of #19924) (#19936)

Bug fixes

• Support using __defaults__ in same BUILD file as environment targets. (Cherry-pick of #19446) (#19968)

• Add missing __init__.py file for the yamllint backend. (Cherry-pick of #19899) (#19908)

• Add additional directories with register.py to pants bin deps (Cherry pick of #19848) (#19857)

• Fix global report coverage for namespaced packages (Cherry-pick of #19821) (#19839)

• Have Pants create the immutable_inputs base parent(s) (Cherry-pick of #19646) (#19653)

• Use hardlinking when possible for mypy cache (Cherry-pick of #19633) (#19649)

• Undo pyenv install marking files as RO (Cherry-pick of #19645) (#19647)

• Use named cache for Docker-env PBS (Cherry-pick of #19611) (#19617)

• Make mypy cache move truly atomic (Cherry-pick of #19610) (#19614)

• Scala: support Enums in dependency inference (Cherry pick of #19588) (#19594)

• Fix the new tool lockfile deprecation message. (cherrypick of #19565) (#19571)

• Fix golangci-lint runner script (Cherry-pick of #19015) (#19472)

• Use the package_name variable to fix an UnboundLocalError. (Cherry-pick of #19433) (#19452)

Documentation

• Update release docs with latest changes (#19895)

• docs: fix code examples for setting-up-ide and interpreter-compatibility docs (Cherry-pick of #19624) (#19638)

• docs: add 2.16 entry to changelog (Cherry-pick of #19589) (#19598)

• Add ruff, adjust pyugprade, autoflake in docs (Cherry-pick of #19555) (#19577)

• Catch google artifact registry gotcha (Cherry-pick of #19568) (#19569)

• Stop referring to ~/.cache/pants/setup in CI docs (Cherry-pick of #19554) (#19560)

• Visibility: documentation fixes and clarifications. (Cherry-pick of #19438) (#19495)

New Features

• add support for mTLS in remote cache usage (#19887)

• Add a GitHub Actions Cache remote cache backend (#19831)

• Support pass-through args to the repl goal for ipython (#19858)

Plugin API Changes

• Always have coloured output and then strip later for pytest, helm unittest, shunit2 (#19923)

Bug Fixes

• Fix Buf expected configuration files (#18015)

• Don't crash in the presence of non-UTF-8 env vars. (#19926)

• Fetch shellcheck from vscode-shellcheck for M1 support (#19945)

• Fix missing venv path when running Pyright (#19430)

• Immediately invalidate written files in write_digest (#19903)

• Visibility: Fix issue with root paths. (#19935)

• Regex dependency inference module mapping (#19684)

Documentation

• Fix "the the" typos in the codebase. (#19967)

• Add Tom and Gautham to the Contributors page. (#19955)

New Features

• add support for mTLS in remote cache usage (Cherry-pick of #19887) (#19960)

Bug Fixes

• Fetch shellcheck from vscode-shellcheck for M1 support (Cherry-pick of #19945) (#19951)

• Immediately invalidate written files in write_digest (Cherry-pick of #19903) (#19948)

• Visibility: Fix issue with root paths. (Cherry-pick of #19935) (#19937)

Documentation

• Add Tom and Gautham to the Contributors page. (Cherry-pick of #19955) (#19961)

New Features

• with contextlib.suppress(ImportError) weakens imports (Cherry-pick of #19293) (#19789)

Bug Fixes

• Port pantsd fingerprinting to Rust, and validate from native client (Cherry-pick of #19833) (#19867)

• Add additional directories with register.py to pants bin deps (Cherry pick of #19848) (#19855)

• Fix global report coverage for namespaced packages (Cherry-pick of #19821) (#19837)

• Allow requesters of TargetRootsToFieldSets to turn off the secondary ownership warning (Cherry-pick of #19721) (#19734)

Performance

• Remove memoization of the Paths intrinsic to reduce memory usage (Cherry-pick of #19689) (#19717)

Documentation

• docs: remove Toolchain references (cherrypick #19508) (#19828)

• docs: Improve the help string for the global option 'PANTS_CONCURRENT'. (Cherry-pick of #19822) (#19824)

• Fix Documentation (InitialConfiguration) go backend (Cherry-pick of #19806) (#19809)

• Fix typoed artifact name in Lambda docs (Cherry-pick of #19739) (#19746)

• Update the docs changelog to include 2.17.x. (Cherry-pick of #19729) (#19736)

New Features

• Add support for OpenDAL-backed remote stores (file://, initially) (#19827)

User API Changes

• Move .pids under .pants.d by default (#19911)

• Enable lockfile diff report by default. (#19900)

Bug Fixes

• Support using parametrize() with a single value. (#19920)

• Don't include files targets in coverage sandbox. (#19913)

• bump javaparser and add test for sealed/permits/yield/switch expressions (#19870)

• Fix "failed to create hardlink" error due to multiple mounts on the same device (#19894)

• Add missing __init__.py file for the yamllint backend. (#19899)

Documentation

• Remove references to PyPI/pantsbuild.pants wheels in docs (#19924)

• Add kubeconfirm, fix typos in enabling-backends.md docs (#19880)

• List every single backend in docs (#19854)

Bug Fixes

• Fix "failed to create hardlink" error due to multiple mounts on the same device (Cherry-pick of #19894) (#19910)

• Add missing __init__.py file for the yamllint backend. (Cherry-pick of #19899) (#19907)

• Port pantsd fingerprinting to Rust, and validate from native client (Cherry-pick of #19833) (#19864)

Documentation

• Remove references to PyPI/pantsbuild.pants wheels in docs (Cherry-pick of #19924) (#19930)

• List every single backend in docs (Cherry-pick of #19854, #19880) (#19878)

New Features

• Add cache-to and cache-from options (#19811)

• Add lint backend to run semgrep (#18593)

• with contextlib.suppress(ImportError) weakens imports (#19293)

• Add support for openapi-format (#18553)

User API Changes

• Add missing openapi java codegen backend (#19835)

• Remove deprecations scheduled for 2.19.0.dev0. (#19759)

• Remove remaining Lambdex code, update FaaS docs (#19738)

• user api change: change the Pip version option to a string (#19643)

• Require scie-pants 0.9.2 or newer, for new distribution model (#19654)

• Add Python 3.12 to the default interpreter universe (#19641)

Plugin API Changes

• Optionally resolve rules by a provided id (#19755)

• Remove "old" Python dep parser (#19792)

• Remove TestResult.{stdout,stderr} and all uses. (#19768)

Bug Fixes

• Port pantsd fingerprinting to Rust, and validate from native client (#19833)

• Add additional directories with register.py to pants bin deps (#19848)

• Add --force-exclude to ruff args (#19823)

• Tweak fix partitioning to closer align to lint partitioning (#19796)

• Fix global report coverage for namespaced packages (#19821)

• Allow requesters of TargetRootsToFieldSets to turn off the secondary ownership warning (#19721)

• Include all binaries available in a venv when fulfilling a VenvPexRequest (#19315)

• Remove tensorboard mapping from our default mapping (#19673)

• Have Pants create the immutable_inputs base parent(s) (#19646)

• Use hardlinking when possible for mypy cache (#19633)

• Undo pyenv install marking files as RO (#19645)

Performance

• Stream large blobs to remote cache directly from local cache file (#19711)

• Remove memoization of the Paths intrinsic to reduce memory usage (#19689)

Documentation

• docs: Improve the help string for the global option 'PANTS_CONCURRENT'. (#19822)

• Fix Documentation (InitialConfiguration) go backend (#19806)

• docs: add chat archive link, more talks, more example repos (#19679)

• Instructions: pants repo debugging with pycharm (#19787)

• Tweak titles in the release doc (#19769)

• generate toml for reference-docs (#19718)

• Remove Toolchain from who uses Pants page (#19747)

• Fix typoed artifact name in Lambda docs (#19739)

• Update the docs changelog to include 2.17.x. (#19729)

• Remove PANTS_SHA from docs (#19719)

• docs: use Python 3.11+ as example (#18565)

• Additional mentions and tweaks in 2.17's "What's new" (#19629)

• docs: update go tracking link to a live ticket instead of the archived project (#19608)

• docs: fix code examples for setting-up-ide and interpreter-compatibility docs (#19624)

• docs: mention that python_distribution targets can use transitive excludes (#19627)

New Features

• with contextlib.suppress(ImportError) weakens imports (Cherry-pick of #19293) (#19788)

User API Changes

• Add Python 3.12 to the default interpreter universe (Cherry-pick of #19641) (#19693)

• Require scie-pants 0.9.2 or newer, for new distribution model (Cherry-pick of #19654) (#19694)

Bug Fixes

• Add additional directories with register.py to pants bin deps (Cherry pick of #19848) (#19856)

• Fix global report coverage for namespaced packages (Cherry-pick of #19821) (#19838)

• Allow requesters of TargetRootsToFieldSets to turn off the secondary ownership warning (Cherry-pick of #19721) (#19728)

• Have Pants create the immutable_inputs base parent(s) (Cherry-pick of #19646) (#19696)

• Remove tensorboard mapping from our default mapping (Cherry-pick of #19673) (#19699)

• Use hardlinking when possible for mypy cache (Cherry-pick of #19633) (#19692)

• Undo pyenv install marking files as RO (Cherry-pick of #19645) (#19691)

Performance

• Remove memoization of the Paths intrinsic to reduce memory usage (Cherry-pick of #19689) (#19716)

Documentation

• docs: Improve the help string for the global option 'PANTS_CONCURRENT'. (Cherry-pick of #19822) (#19825)

• Fix Documentation (InitialConfiguration) go backend (Cherry-pick of #19806) (#19810)

• Tweak titles in the release doc (Cherry-pick of #19769) (#19770)

• Fix typoed artifact name in Lambda docs (Cherry-pick of #19739) (#19745)

• Update the docs changelog to include 2.17.x. (Cherry-pick of #19729) (#19735)

• docs: fix code examples for setting-up-ide and interpreter-compatibility docs (Cherry-pick of #19624) (#19695)

• docs: use Python 3.11+ as example (Cherry-pick of #18565) (#19700)

• docs: update go tracking link to a live ticket instead of the archived project (Cherry-pick of #19608) (#19698)

• docs: mention that python_distribution targets can use transitive excludes (Cherry-pick of #19627) (#19697)

What's New

Individuals and companies can now sponsor Pants financially.

Pants is an open-source project that is not owned or controlled by any one company or organization, and does incur some expenses. These expenses are managed by Pants Build, a non-profit that was established for this purpose. This non-profit's only source of revenue is sponsorship by individuals and companies that use Pants.

We offer formal sponsorship tiers for companies, as well as individual sponsorships via GitHub.

Overall

The pants launcher binary (aka scie-pants) is now the recommended way to run Pants. Use of a ./pants shell script in a repository is deprecated.

The pants launcher has numerous benefits, such as:

• automatically download and maintain a Python distribution for running Pants
• run pants from any directory within a repository and not just the build root
• far less overhead when starting commands (up to 4-5×)

See the Pants installation instructions for additional information on how to migrate to the pants launcher binary.

Related to this, pants' distribution model is changing, and 2.17 will be the last version where:

• the pantsbuild.pants package is published to PyPI. Use the pants_requirements target for installing the requisite packages for plugins, rather than python_requirement or similar.
• the universal (cross-platform) pants PEX is published to GitHub Releases. For future releases, smaller per-platform PEXes will be attached instead.

Performance

As mentioned above, the new launcher binary reduces start-up overhead.

Large files are now cached as standalone files on disk, rather than stored directly in the internal database pants uses for cache (but still under the ~/.cache/pants/lmdb_store directory). This improves performance for manipulating these files, especially when the cache directory and sandbox temporary directory are on the same file system and thus allow hard-linking.

Backends

Python

Python dependencies are now analyzed via an intrinsic rule implemented in Rust for increased performance over the prior Python implementation of the rule. See #18854 for discusion of the performance improvements.

Exported mutable virtualenvs can now include PEP-660 editable installs of python_distribution targets. To enable this feature for a resolve, add that resolve's name to the [export].py_editables_in_resolves list in pants.toml.
See #18639 for details on Pants' PEP-660 implementation.

The default pip Pants uses under the hood has changed from 20.3.4 to 23.1.2. This newer pip has better dependency resolution performance in many cases, but may give different results than the earlier pip. Of course those results will still be correct in the sense that they will be compatible with your requirements and constraints.

The python_awslambda or python_google_cloud_function targets now support a new 'zip' layout, as recommended by the cloud vendors. This layout gives smaller packages and faster cold starts than the existing Lambdex layout, and will become the default in 2.18. See the docs for python_awslambda and for python_google_cloud_function.

The default value for the [python].interpreter_constraints option was deprecated in 2.16 and is now removed: pants.toml must provide this option. We recommend constraining to a single interpreter minor version if you can, for instance: interpreter_constraints = ['==3.11.*']. See the Interpreter Compatibility docs for more details.

Docker

The Docker backend now supports authenticating with registries via the DOCKER_HOST, DOCKER_CONFIG and DOCKER_CERT_PATH environment variables.

New: Javascript (experimental!)

Pants now has very early experimental support for Javascript thanks to ongoing efforts by Tobias Nilsson and Theo Ribeiro. Please note that the Javascript backend is still incomplete, is being actively developed, and probably has critical bugs. Despite that, the Pants maintainers would appreciate any feedback from the community to help guide our development efforts.

Supported goals are:

• test: Letting you run tests via runners installed with the package manager of your choice.
• package goal, either to run a customized package.json script that produces an artifact via node_build_script, or to pack a tarball for npm-registry publication via the npm_distribution target.
• tailor: Generates build file targets for *.js, *.test.js and package.json files.
• generate-lockfile: Creates the lockfile in the format matching a projects package manager.

The backend supports all package managers provided by corepack. yarn@v2 and PlugNPlay is not supported.

Enable the pants.backend.experimental.javascript backend to try out this support. Please file issues for any issues encountered, and follow along the stabilization ticket on github.

New: Taplo TOML Formatter

Pants now supports the Taplo TOML formatter. Enable the pants.backend.tools.taplo backend to add this support.

Plugin API Changes

Rules should now request output types which do not need an input type via the new one-argument form of Get. For example, rules can now write await Get(ChosenLocalEnvironmentName). Certain request types which exists only to work around the previous lack of such synax are now deprecated in favor of the one argument Get() form.

The PythonBinary type is now deprecated, use PythonBuildStandalone instead.

Bug Fixes

• Remove tensorboard mapping from our default mapping (Cherry-pick of #19673) (#19677)

Documentation

• Backport to 2.17: docs: set EnvironmentBehavior in plugin Goal examples (#19590) (#19664)

User API Changes

• Require scie-pants 0.9.2 or newer, for new distribution model (Cherry-pick of #19654) (#19655)

• Add Python 3.12 to the default interpreter universe (Cherry-pick of #19641) (#19651)

Bug Fixes

• Have Pants create the immutable_inputs base parent(s) (Cherry-pick of #19646) (#19652)

• Use hardlinking when possible for mypy cache (Cherry-pick of #19633) (#19650)

• Undo pyenv install marking files as RO (Cherry-pick of #19645) (#19648)

• Make mypy cache move truly atomic (Cherry-pick of #19610) (#19615)

• Use named cache for Docker-env PBS (Cherry-pick of #19611) (#19616)

• Add path to cache key for Rust dep inference, for relative imports (cherry-pick of #19630) (#19640)

Documentation

• Backport to 2.17: Fix dead doc links (#19546) (#19659)

• Additional mentions and tweaks in 2.17's "What's new" (Cherry-pick of #19629) (#19642)

• docs: fix code examples for setting-up-ide and interpreter-compatibility docs (Cherry-pick of #19624) (#19639)

• docs: fix markdown inconsistencies (Cherry pick of #19363) (#19622)

• docs: add 2.16 entry to changelog (Cherry-pick of #19589) (#19597)

New Features

• Add Docker global no build cache option (#19533)

• add latest helm versions; upgrade default (#19592)

• Add tfsec linter for Terraform (#19525)

Bug Fixes

• Use named cache for Docker-env PBS (#19611)

• Make mypy cache move truly atomic (#19610)

• Fix an erroneous options scope in an export error message. (#19605)

• javascript: Assign a default name to root nodejs resolves (#19245)

• Cache Go processes using full version string (#19558)

• Scala: support Enums in dependency inference (#19588)

Documentation

• docs: set EnvironmentBehavior in plugin Goal examples (#19590)

• docs: add 2.16 entry to changelog (#19589)

• initial draft of What's New for v2.17.x (#19168)

• Spelling fix in target docs (#19578)

• Update and streamline the release process docs. (#19575)

• Add ruff, adjust pyugprade, autoflake in docs (#19555)

• Catch google artifact registry gotcha (#19568)

• Stop referring to ~/.cache/pants/setup in CI docs (#19554)

User API Changes

• Upgrade the default pip to v23.1.2. (Cherry-pick of #19538) (#19541)

Bug Fixes

• Scala: support Enums in dependency inference (Cherry pick of #19588) (#19593)

• Fix the new tool lockfile deprecation message. (#19565)

• Fix golangci-lint runner script (Cherry-pick of #19015) (#19473)

Documentation

• Initial draft of What's New for v2.17.x (Cherry-pick of #19168) (#19595)

• Add ruff, adjust pyugprade, autoflake in docs (Cherry-pick of #19555) (#19576)

• Catch google artifact registry gotcha (Cherry-pick of #19568) (#19570)

• Stop referring to ~/.cache/pants/setup in CI docs (Cherry-pick of #19554) (#19561)

• Visibility: documentation fixes and clarifications. (Cherry-pick of #19438) (#19494)

New Features

• bump pyright to 1.1.316 (#19426)

User API Changes

• Upgrade the default pip to v23.1.2. (#19538)

Plugin API Changes

• Resolve pants_requirements from direct wheel links to GitHub Release assets (#19432)

Bug Fixes

• Support discarding inferred Helm snapshot dependencies (#19527)

Documentation

• Fix dead doc links (#19546)

• docs: fix rules-api docs typos (#19539)

• docs: remove Toolchain references (#19508)