pants

New Features

• javascript: Test goal with package manager installed test runner support (#18554)

• expose PANTS_VERSION and make it string-comparable (#18573)

User API Changes

• Upgrade Pex to 2.1.131. (#18626)

Bug fixes

• Ensure that sandboxed processes exit before their sandboxes are cleaned up (#18632)

• Plumb entire lockfile/internal only code through create_pex_from_targets (#18622)

• narrow pex_binary files warning (#18619)

• Fix PexPEX invocations (#18610)

• Improve debug information for docker auth, and expose an environment variable. (#18599)

Performance

• Only materialize immutable files once per process (#18600)

Documentation

• Fix a string that should have been an f-string (#18608)

• Fix ./pants -> pants in some docs and help strings. (#18596)

Bug fixes

• Ensure that sandboxed processes exit before their sandboxes are clean… (#18641)

• Plumb entire lockfile/internal only code through create_pex_from_targets (cherry-pick of #18622) (#18636)

• Fix export-codgen with multiple output types from same input type (Cherry-pick of #18373) (#18377)

• Improve debug information for docker auth, and expose an environment variable. (Cherry-pick of #18599) (#18603)

Documentation

• Fix ./pants -> pants in some docs and help strings. (cherrypick of #18596) (#18607)

New Features

• engine: add an option to clear the destination directory when writing digest to disk (#18534)

• javascript: NodeJS bootstrapping via binary paths, PATH, asdf or nvm (#18520)

• javascript: Initial NPM support (#18326)

• Always provide Python-for-Pants-scripts (#18433)

User API Changes

• Upgrade Pex to 2.1.130. (#18576)

• Deprecate old-style custom tool lockfiles. (#18494)

• Let a user resolve shadow a tool lockfile of the same name (#18481)

Bug fixes

• Ensure that a symlinked exported venv exists and is valid. (#18575)

• Fail fast for attempts to use test --debug with a docker environment (#18560)

• Declare ruff outputs as files. (#18551)

• Add support for Docker registry auth (#18541)

• Fix Poetry req synthesis for URLs with markers. (#18535)

• Re-enable hardlinking using fclonefileat on macOS. (#18501)

Performance

• Materialize "large" files in a new store location and hardlink them in sandboxes (#18153)

• Finish parallelizing materialization of Process inputs (#18469)

Documentation

• First pass at adhoc_tool documentation (#18531)

• Add Tobias Nilsson to team page (#18571)

• docs: fix an f-string in help (#18539)

• Clarify that experimental_ shell functions are deprecated and replaced (#18529)

• Update docs on running pants from sources with scie-pants. (#18513)

• Add ExoFlare to user list. (#18496)

• Fix malformed pantsbuild/example-python link in documentation (#18484)

• update scie-jump cache directory reference (#18485)

• Expand lockfile documentation to its own page. (#18471)

New Features

• Always provide Python-for-Pants-scripts (Cherry-pick of #18433) (#18495)

• Expose environment information on process metadata (Cherry-pick of #18458) (#18463)

User API Changes

• Upgrade Pex to 2.1.130. (Cherry-pick of #18576) (#18578)

• Deprecate old-style custom tool lockfiles. (#18494) (#18498)

• Let a user resolve shadow a tool lockfile of the same name (Cherry-pick of #18481) (#18493)

• Audit and update requirements for Python tools. (cherrypick #18431) (#18460)

Bug fixes

• Ensure that a symlinked exported venv exists and is valid. (Cherry-pick of #18575) (#18585)

• Fail fast for attempts to use test --debug with a docker environment (Cherry-pick of #18560) (#18568)

• Declare ruff outputs as files. (Cherry-pick of #18551) (#18557)

• Add support for Docker registry auth (Cherry-pick of #18541) (#18549)

• Fix Poetry req synthesis for URLs with markers. (Cherry-pick of #18535) (#18536)

• Preserve environment variables in pantsd to allow Docker auth. (Cherry-pick of #18465) (#18468)

• Only expose the equiv of from typing import * in prelude files (Cherry-pick of #18452) (#18455)

• Support type hints in prelude files. (Cherry-pick of #18442) (#18449)

• Synthesized lockfile targets should never err on missing lockfiles. (Cherry-pick of #18406) (#18447)

Documentation

• First pass at adhoc_tool documentation (Cherry-pick of #18531) (#18581)

• Update docs on running pants from sources with scie-pants. (Cherry-pick of #18513) (#18514)

• Remove the superfluous updated: front matter from readme docs. (cherrypick of #18475) (#18476)

• Expand lockfile documentation to its own page. (cherrypick of #18471) (#18474)

Bug fixes

• Fail fast for attempts to use test --debug with a docker environment (Cherry-pick of #18560) (#18567)

• Add support for Docker registry auth (Cherry-pick of #18541) (#18550)

• Fix Poetry req synthesis for URLs with markers. (Cherry-pick of #18535) (#18537)

• Preserve environment variables in pantsd to allow Docker auth. (Cherry-pick of #18465) (#18467)

• Synthesized lockfile targets should never err on missing lockfiles. (Cherry-pick of #18406) (#18446)

• go: skip go v1.20+ when searching for the GOROOT (#18429)

Documentation

• Some doc fixes suggested by users. (Cherry-pick of #18393) (#18394)

Bug fixes

• Fix Poetry req synthesis for URLs with markers. (Cherry-pick of #18535) (#18538)

• Make mypy cache per-repo (Cherry-pick of #18043) (#18060)

New Features

• Allow python tools to be installed from a user resolve. (#18418)

• golang: add support for passthrough args for gofmt (#18409)

• Add the ability for Pants to provide Python via a union (with a pyenv impl) (#18352)

• Add sources_fingerprint to peek on source-creating targets (#18383)

• New BUILD file symbols help info (#18378)

• Visibility: support matching on target names. (#18369)

• Support for peeking at dependency/dependents rules (#18112)

• Adds support for supplying "runnable" dependencies to the execution_dependencies field (#18347)

• javascript: Support esm modules and commonjs specific javascript file extensions (#18208)

• Adds system_binary target, which can be used to reproducibly find a pre-installed binary for use with adhoc_tool (#18332)

• Enable running third-party Go main packages (#17449)

User API Changes

• Add python311 runtime to python_google_cloud_function target (#18405)

• Delegate lockfile requirement validation to Pex when possible. (#18415)

• Upgrade Pex to 2.1.126. (#18408)

• Deprecate the default interpreter constraints. (#18390)

• Give pantsd more RAM by default. (#18389)

Plugin API Changes

• Support specifying a named resolve as the superset in a PexRequirements. (#18397)

• Deprecate the @rule_helper decorator. (#18330)

• Get rid of the LockfileContent class (#18305)

Bug fixes

• python: collect files with '[tool.ruff' sections as candidates for Ruff config files (#18413)

• Fix OSError when running pants in systemd-nspawn container (#18398)

• Fix paths issue with transitive excludes. (#18396)

• Fix export-codgen with multiple output types from same input type (#18373)

• Errors in --loop wait for changes rather than re-running immediately (#18367)

• Support using env() in prelude macro files (#18273)

• Docker environment uses a volume for named caches (#18327)

• Load fromfile files relative to build root (#18311)

• Ensure that the pep-517 backend shim works on Python 2.7. (#18295)

Documentation

• Update documentation on how to install the pants binary (#18372)

• Some doc fixes suggested by users. (#18393)

• docs: tidy up writing plugins tutorial (#18380)

• docs: upgrade plugin writing tutorial to 2.15.0 (#18371)

• Use valid container/app names in helm examples (#18358)

• Remove defunct newsletter (#18355)

• docs: mention code generation when troubleshooting missing imports from first-party code (#18073)

• Fix documentation example for python_awslambda targets (#18302)

• Change bool values to lowercase in TOML snippets (#18299)

The first stable release of the series, with no changes since the previous rc!

Pants 2.15 brings support for several new features:

Environments

Pants now supports alternate execution "environments" for build actions. These "environments" can be explicitly
configured as desired via several new target types (including having multiple environments of the same type).
The new environment field on many targets tells Pants which environment to use for a particular target.

Pants previously supported local execution (which is now configured by the local_environment target type) and
remote execution via the Remote Execution API (which is now configured by the remote_environment target type).

New in Pants 2.15 is support for executing build actions within a Docker container configured via the
docker_environment target type.

See the Environments documentation for more information.

Test Batching

./pants test now supports batching compatible tests together so that they execute in the same process.
This can improve performance of test execution by sharing test environment setup across multiple test files,
but must be balanced against the impact on caching and concurrency.

The Python backend is the first user of the test batching support.

New goal: fix

The fix goal has been added to run tools which generally will perform more invasive changes on your source
files than the fmt goal. The fmt goal remains for tools which merely format code without making
semantic changes.

Backends

Python

Several changes have been made to the Python backend:

• Support for the new Environments feature. Relevant targets now have a new environment field
  to configure which environment to use for that target.
• Batched execution of pytest tests.
• Support for lcov coverage format.
• Support for pyright typechecking.
• Python lockfiles are now represented in the build graph via synthetic targets.
• Upgrades to default version of several tools including coverage and Flake8.

Go

The Go backend received several changes including:

• Basic support for Cgo
• Coverage reports can be generated in HTML and not just the raw data file used by go tool cover.
• Support for the golangci-lint linter.
• Support for multiple go_mod targets in same repository.
• A new go-generate custom goal to run go generate on any //go:generate instructions in a package.
• $GOROOT/bin is now prepended to PATH when running tests to enable them to easily find the go binary for
  the Go toolchain in use.

Note: Due to certain changes in Go v1.20, Pants 2.15.x and earlier Pants versions will not work with Go v1.20
and later releases. Instead, use Pants v2.16.x or later Pants versions to support Go v1.20 and later releases.

New: OpenAPI

The new OpenAPI backend provides support for linting OpenAPI API specifications using the spectral tool
and generating Java code from the OpenAPI specifications.

Enable the pants.backend.experimental.openapi backend for base OpenAPI support,
the pants.backend.experimental.openapi.lint.spectral backend for linting support, and
the pants.backend.experimental.openapi.codegen.java backend for Java code generation support.

Plugin API changes

Several changes were made to the Plugin API including the changes highlighted below. Read the
Plugin upgrade guide for a full description
of the changes highlighted below (as well as other changes not highlighted here), including how to migrate
existing plugins.

Formatters / Linters

The schema used for formatters and linters has been changed in order to allow for formatters and linters which
do not require a target on which to operate. See the applicable section of the Plugin upgrade guide
for the specific changes.

Batched Tests

The plugin API used by the test goal has been changed significantly to support the explicit batching of tests. See the
applicable section of the Plugin upgrade guide
for the specific changes.

Synthetic Targets

Plugins may now add "synthetic targets" to the build graph to create freestanding targets programmatically.
These synthetic targets can also be generator targets in order to further create generated targets. See the
plugin documentation for the Synthetic Targets API
for more information.

Bug fixes

• Docker environment uses a volume for named caches (Cherry-pick of #18327) (#18337)

• go: add backends which were missing from distribution (Cherrypick of #18312) (#18316)

• Ensure that the pep-517 backend shim works on Python 2.7. (Cherry-pick of #18295) (#18300)

New Features

• Allow exporting codegenerated files/resources (#18236)

• python: teach tailor goal to create resource targets for py.typed marker files (#18174)

• The beginnings of a Django plugin. (#18173)

• Add backend for projects that use openstack/stevedore (#18132)

• [python-infer]: allow ignoring unowned imports (#18094)

• Print diff report when generating Python lockfiles. (#17347)

User API Changes

• Deprecates experimental_run_shell_command in favour of run_shell_command (#18266)

• Deprecates experimental_shell_command, replacing with shell_command (#18255)

• Extracts backend.adhoc package and backend.experimental.adhoc backend for adhoc_tool (#18237)

• Upgrade Pex to 2.1.122. (#18230)

• Relativize the pants_bin_name if necessary. (#18204)

• Adjust the default for root_output_directory (#18192)

• Visibility backend: adjust rule set selector syntax making path values default rather than target types. (#18154)

• Adds fields to capture stdout/stderr to experimental_run_in_sandbox (#18165)

• shell_command/run_in_sandbox workdir/output dir adjustments (#18157)

• Default to the new "Dependents" header in the py_constraints report. (#18142)

Plugin API Changes

• Improve type inference for @rules (#17947)

• deprecate frozen_after_init (#18127)

Bug fixes

• go: support embed configuration in stdlib packages (#18271)

• Upgrade to Lambdex 0.1.9 (#18257)

• Delete override of expected exe name for shfmt. (#18258)

• Fix bootstrap issue with __defaults__ and environments. (#18251)

• Allow breakpoints to be hit when run_in_sandbox is False (#18244)

• Avoid bind-mounts for docker environments on macOS (#18225)

• Move ruff from fmt to fix (#18228)

• Stream large remote cache downloads directly to disk (#18054)

• BUILD file target field default should respect values from __defaults__. (#18172)

• Fix published wheel RECORDs. (#18219)

• Always explicitly shutdown executors (#18216)

• go: upgrade golangci-lint to fix issue with Go v1.20 (#18207)

• go: disable coverage redesign experiment on Go v1.20+ (#18205)

• Use EnvironmentAwarePackageRequest in many more contexts. (#18203)

• Have lambdex create a new output file. (#18200)

• Shutdown all threads as part of Scheduler shutdown (#18166)

• Fix the dep inference scripts source root bug. (#18164)

• Add the environment field to most Python binary target types (#18144)

• go: don't check for Target attribute on stdlib packages (#18148)

• go: fix backend to work when run in non-local environments (#18140)

• Run pyupgrade until convergance (#18128)

Documentation

• docs: use an example black version that does not explode (#18270)

• docs: fix typo (#18238)

• Update the stated release target to match our desired reality. (#18234)

• docs: add What's New section for 2.15.x release notes (#18226) (#18232)

• Add warning about dependency inference + extra_type_stubs. (#17076)

• docs: further updates to 2.16.x What's New (#18227)

• Jacob Floyd/Huon Wilson updates to team page (#18221)

• docs: add "What's New" section to 2.16 release notes (#18146)

• Document the new get-pants.sh name. (#18137)

• Add Pylint post (#18079)

• Change to Pantsbuild's custom YouTube url (#18133)

• Recommend checking the pantsup.sh script in. (#18120)

• Don't update old release notes. (#18123)

• Change ./pants to pants in the docs (#18121)

New Features

• Allow exporting codegenerated files/resources (Cherry-pick of #18236) (#18241)

Bug fixes

• Fix bootstrap issue with __defaults__ and environments. (Cherry-pick of #18251) (#18262)

• Upgrade to Lambdex 0.1.9 (Cherry-pick of #18257) (#18261)

• Delete override of expected exe name for shfmt. (Cherry-pick of #18258) (#18259)

• Allow breakpoints to be hit when run_in_sandbox is False (Cherry-pick of #18244) (#18245)

Documentation

• docs: add What's New section for 2.15.x release notes (#18226)

Bug fixes

• Fix published wheel RECORDs. (Cherry-pick of #18219) (#18220)

• Always explicitly shutdown executors (Cherry-pick of #18216) (#18218)

• Use EnvironmentAwarePackageRequest in many more contexts. (Cherry-pick of #18203) (#18206)

• Have lambdex create a new output file. (Cherry-pick of #18200) (#18201)

Bug fixes

• Shutdown all threads as part of Scheduler shutdown (Cherry-pick of #18166) (#18188)

New Features

• Add environment= to experimental_shell_command. (Cherry-pick of #17575) (#18064)

Bug fixes

• Add the environment field to most Python binary target types (Cherry-pick of #18144) (#18150)

• Run pyupgrade until convergance (Cherry-pick of #18128) (#18130)

• Make mypy cache per-repo (Cherry-pick of #18043) (#18061)

• go: detect when go.sum needs additional entries filled in (Cherrypick of #17811) (#18118)

• Explicitly use "python" ijson backend (cherry-pick of #17787) (#18077)

• Fix caching of PATH lookups in remote execution (Cherry-pick of #18026) (#18062)

Documentation

• Document the new get-pants.sh name. (Cherry-pick of #18137) (#18139)

• Recommend checking the pantsup.sh script in. (Cherry-pick of #18120) (#18134)

• Change ./pants to pants in the docs (cherry-pick of #18121) (#18124)

• docs: fix markdown inline code typo in a docstring (Cherry-pick of #18002) (#18103)

• docs: add tutorials to get started writing plugins (Cherry-pick) (#18107)

• docs: mention name clashing between custom goals and options when writing plugins (Cherry-pick of #17899) (#18105)

• docs: tidy up Writing plugins : rules-api directory files (Cherry-pick of #17710) (#18102)

• Document new pants launcher binary, aka scie-pants. (Cherry-pick of #18056) (#18075)

• Docs: improve docs on handling option values in cli (Cherry-pick of #17506) (#18069)

• Update and reorganize resource hub (#17922) (#18068)

• Add Daniel Goldman and Darcy Shen to team page (Cherry-pick of #17921) (#18066)

• Nudge more technical support toward GH Discussions (Cherry-pick of #17906) (#18065)

New Features

• go: add environment field to go_binary (#18113)

• Add experimental backend for ruff with lint and fmt goals (#17945)

• Add support for fmt cue files. (#18106)

• Add new experimental backend for CUE with initial support for the lint goal. (#17943)

• Add yamllint as a lint tool for YAML sources (#17821)

• lint: add pydocstyle python backend (#17596)

User API Changes

• Upgrade Pex to 2.1.121. (#18037)

Plugin API Changes

• Make the python dep inference script extensible. (#17997)

• Remove some dead code from lockfile.py (#18041)

Bug fixes

• Inject a path-safe target spec into experimental_shell_command etc processes (#18092)

• docker: Allow overriding FROM args when determining upstream image dependencies (#18009)

• Make mypy cache per-repo (#18043)

• Fix caching of PATH lookups in remote execution (#18026)

Documentation

• docs: make tutorials to get started writing plugins visible on readme pages (#18083)

• Document new pants launcher binary, aka scie-pants. (#18056)

• Front matter for new tutorial pages. (#18074)

• Cosmetic fix for current value information in help output for pass-through arguments. (#18051)

• docs: add pydocstyle to the docs and help reference (#18033)

• docs: add tutorial to get started writing plugins: No 3 (#18018)

New Features

• Added support for discovering and patching pyright config files (cherry-pick of #17771) (#18030)

Bug fixes

• Streamline local dist handling when running python sources. (Cherry-pick of #17742) (#18035)

• Adds workunit for interactive processes. (Cherry-pick of #17544) (#17996)

• Pyright should grab non-transitive dependencies and make them available in the sandbox (Cherry-pick of #17235) (#17980)

• add skip_pyright field to python test targets (Cherry-pick of #17960) (#17968)

• Do not load the BUILD file prelude (macros) in the bootstrap scheduler. (Cherry-pick of #17939) (#17941)

Documentation

• Document how to update the pants script. (Cherry-pick of #17976) (#17979)

New Features

• New env() BUILD file function. (#17652)

• Support REPL command history. (#17959)

• Resolve Python dep inference ambiguity via locality. (#17931)

• Add targets to re-wrap source files in different SourcesField types. (#17877)

• Allow plugins to add custom schema/authority URL handler rules (#17898)

• Support freezing command line args and env vars into pex binaries. (#17905)

• Support catching @rule errors (#17911)

User API Changes

• Ugrade Pex to 2.1.120 (#17957)

• Adds workdir field for experimental_shell_command and friends (#17928)

Bug fixes

• Let pyright see dependencies in other source roots. (#18014)

• go: gather and link transitive prebuilt object files for cgo mode (#17971)

• add skip_pyright field to python test targets (#17960)

• Allow experimental_shell_command/experimental_run_in_sandbox to specify output_s from anywhere under the buildroot (#17938)

• Do not load the BUILD file prelude (macros) in the bootstrap scheduler. (#17939)

Documentation

• WSL disclaimer in Getting Started/Prerequisites (#18000)

• Add Daniel Goldman and Darcy Shen to team page (#17921)

• Clarify runtime vs. complete_platforms for serverless. (#18001)

• docs: add tutorial to get started writing plugins: No 2 (#17890)

• docs: generate a JSON schema file to be used in IDEs when editing pants.toml (#17915)

• docs: fix markdown inline code typo in a docstring (#18002)

• fix type in check help output (#17969)

• Document how to update the pants script. (#17976)

• Add Kaiko to "who uses Pants" page. (#17949)

• docs: provide a link to Google Cloud Function in the backend docstring (#17948)

• docs: add info about beta stage support of protobuf for Scala and Java (#17935)

• docs: add the first tutorial on writing plugins to readme docs directory (#17930)

• docs: reformat the kotlin docs to have better line breaks on readme.com (#17929)

• Update and reorganize resource hub (#17922)

• Add Payhere to user list. (#17917)

• Nudge more technical support toward GH Discussions (#17906)

• go: update docs to remove caveats about implemented features (#17913)

The second stable release of the 2.14.x series, with no changes since the previous release candidate!

New Features

• Add [buf] config and config_discovery options (#17885)

• New help topic: backends. (#17600)

• Make pants work when the uid doesn't map to a user. (#17881)

• Make JVM sources runnable (#17847)

• Support python_requirement as a run/experimental_run_in_sandbox target (#17864)

• Support experimental_run_in_sandbox/./pants run for jvm_artifact targets (#17827)

• go: support the go test profiling options (#17836)

• Add output_path field to python_distribution. (#17841)

User API Changes

• Upgrade default mypy-protobuf version to 3.4 (#17887)

• allow each resolve to be exported for multiple python interpreters (#17711)

• More default Python module mappings. (#17879)

• Make export cleanup a subset of dist/export, not the whole thing (#17849)

• Unify the messages for lockfile validation errors. (#17842)

• Unify validation of tool and user lockfile metadata. (#17834)

• Detect Pants-generated lockfiles before attempting to validate. (#17833)

Plugin API Changes

• Remove immutable_input_digests from shell code (#17886)

• Default use_pantsd to False in run_pants (#17874)

• Remove the ToolCustomLockfile/ToolDefaultLockfile classes. (#17843)

Bug fixes

• scala: support scala v3 which has different compiler/library artifacts (#17855)

Performance

• Bring back hardlinking big files (#17878)

• Don't invalidate watched files if non-perms metadata changed (#17875)

Documentation

• docs: mention name clashing between custom goals and options when writing plugins (#17899)

• docs: add tutorial to get started writing plugins: No 1 (#17732)

• Document batched pytest execution. (#17845)

• Tweak the docs about defaults. (#17829)

• docs: extend python test goal with info on collecting tests with pytests (#17894)

Bug fixes

• scala: support scala v3 which has different compiler/library artifacts (Cherry pick of #17855) (#17867)

• Fix twine env vars propagation for publish goal (Cherry-pick of #17650) (#17658)

• Allow for run --debug-adapter to match breakpoints when source is running in a sandbox (Cherry-pick of #17566) (#17583)

• Do not try to create "" output directory in CreateArchive (Cherry-pick of #17538) (#17547)

• Fix an unactionable InteractiveProcessRequest deprecation (#17546)

• Fix dep inference from shuint2_tests targets. (Cherry-pick of #17417) (#17421)

• Fix dirtying of uncacheable nodes (Cherry-pick of #17079) (#17396)

Documentation

• Fix broken docs links to manual build instructions (Cherry-pick of #17608) (#17609)

• Update team pages for SJ and Dan! (Cherry-pick of #17534) (#17569)

• Fix line breaks in a docs page. (Cherry-pick of #17527) (#17529)

• Clarify how __defaults__ apply to target generators. (Cherry-pick of #17493) (#17498)

• Fix typo in docs: debug-adaptor should be debug-adapter (Cherry-pick of #17452) (#17453)

• Update nuke_if_too_big documentation (Cherry-pick of #17429) (#17430)

• Fix version templating in vcs_version docs (Cherry-pick of #17423) (#17424)

• Add 2.14 blog post link to changelog page. (Cherry-pick of #17381) (#17384)

• Document the existence of the init-pants action. (Cherry-pick of #17378) (#17380)

• Fix linebreak issues in the targets docs page. (Cherry-pick of #17364) (#17366)