pants

Plugin API Changes

• Upgrade packaging from 20.4 to 20.7 (#11302)

• Update default values for run tracker and streaming work unit handlers to more commonly used values. (#11292)

Bug fixes

• Fix filtering of log messages generated in native code. (#11313)

• Upgrade to Pex 2.1.24 to fix macOS Big Sur. (#11312)

• Clean the graph speculatively, and cancel nodes when interest is lost (#11308)

• Implement native Process cache scoping to fix --test-force (#11291)

New Features

• python: add default module mapping of scikit-video Python package (#19498)

• feat: ptype type checker (#19440)

• Implement Helm chart versioning (#19493)

• Add build platform and hosts options to docker_image target (#19475)

• paths goal: allow finding paths between addresses that expand to multiple targets (#19482)

• Add Support for Kubeconform in Helm backend (#19427)

• Resolve Docker addresses relative to the helm_deployment (#19455)

• Wire pep621 into tailor (#19296)

Bug Fixes

• Support using __defaults__ in same BUILD file as environment targets. (#19446)

• Fix golangci-lint runner script (#19015)

• pex_binary sandbox: Traversing dep graph, stop after package targets (#19155)

Documentation

• docs: mention addressing targets in the current directory with : (#19510)

• Add Aviva Credito to user list. (#19511)

• docs: fix typo from docker.md (#19504)

• docs: explain that run_goal_use_sandbox field won't have any effect with the test goal execution (#19501)

• docs: explain that visibility rules do not apply transitively (#19476)

• Visibility: documentation fixes and clarifications. (#19438)

New Features

• Upload local Pants PEXs to release (#19450)

• Automate GitHub release creation, with wheels (#19444)

Bug Fixes

• Visibility: fix issue with using target type in verbose rule selector. (#19437)

• Use the "package_name" variable to fix an UnboundLocalError. (#19433)

• Raise error for conflicting Docker registries (#19344)

• Provide proper error message for unrecognized symbols during bootstrap. (#19225)

Documentation

• Fix unwanted line break in remote-caching-execution.md (#19461)

• Add Embark Studios to Who Uses Pants page. (#19442)

• Update adhoc-tool.md (#19434)

Bug Fixes

• Visibility: fix issue with using target type in verbose rule selector. (Cherry-pick of #19437) (#19457)

• Fix helm documentation example (Cherry-pick of #19271) (#19310)

Documentation

• Tweak getting-help docs (Cherry-pick of #19418) (#19420)

• Refine run_shell_command docs (Cherry-pick of #19413) (#19422)

• Add a sponsorships page to the docsite. (Cherry-pick of #19412) (#19417)

• Get doc titles from the in-repo markdown files (Cherry-pick of #19311) (#19320)

• docs: layout tweak of caching info, remove toolchain :( and additional info about bazel-remote-cache (Cherry-pick of #19299) (#19331)

Bug Fixes

• Visibility: fix issue with using target type in verbose rule selector. (Cherry-pick of #19437) (#19456)

• Use the "package_name" variable to fix an UnboundLocalError. (Cherry-pick of #19433) (#19453)

• Fix helm documentation example (Cherry-pick of #19271) (#19313)

Documentation

• Tweak getting-help docs (Cherry-pick of #19418) (#19419)

• Refine run_shell_command docs (Cherry-pick of #19413) (#19421)

• Add a sponsorships page to the docsite. (Cherry-pick of #19412) (#19416)

• Get doc titles from the in-repo markdown files (Cherry-pick of #19311) (#19321)

• docs: layout tweak of caching info, remove toolchain :( and additional info about bazel-remote-cache (Cherry-pick of #19299) (#19329)

New Features

• Support per backend requirements (#19406)

Bug Fixes

• Embed a hash of the relevant implementation into the dep inference cache (#19393)

• Improved error message when referencing undefined BUILD symbols (#19286)

Documentation

• Refine run_shell_command docs (#19413)

• Tweak getting-help docs (#19418)

• Add a sponsorships page to the docsite. (#19412)

New Features

• turn path/BUILD into path in the tailor goal (#19323)

• Terraform: Add experimental-deploy and wire-in dependency inference (#19185)

• add support for alias-flags (#19034)

• Add environment variable help options (#19078)

• Add build_network option to docker_image target (#19354)

User API Changes

• Remove support for referencing 2nd owner via file. (#19400)

Plugin API Changes

• Remove deprecated PythonBinary type. (#19399)

Bug Fixes

• Fix pytest-cov validation check. (#19366)

Documentation

• docs: recommend scie-pants on upgrade tips page (#19139)

• docs: fix markdown inconsistencies (#19363)

• docs: apply suggested edits to guide docs from readme.com (#19374)

• docs: add style guide instructions for writing Python help strings (#19365)

• docs: update plugin writing tutorials (#19369)

• docs: fix markdown inconsistencies (#19359)

The third stable release of the 2.15.x series, with no changes since the previous rc!

The first stable release of the series, with no changes since the previous rc!

BUILD files

The new env function in BUILD files allows access to environment variables in BUILD files.

Run pants help symbols for documentation of all builtin symbols and macros provided by prelude files
which are usable in BUILD files.

The visibility rules syntax now supports matching on target names. The rules glob syntax also now uses
the same syntax as target selectors.

Backends

Docker

The {pants.hash}
generation code was changed such that the generated hash for the same input will be different in pants 2.16.

Python

The Python backend added or improved support for various tools including:

• pydocstyle is now supported for linting Python doc strings. Enable the
  pants.backend.python.lint.pydocstyle backend to add this support.

• ruff is now supported for linting and formatting Python code. Enable the
  pants.backend.experimental.python.lint.ruff backend to add this support.

• pyenv is now supported as a way for Pants to provide a hermetic version of Python
  for each relevant Interpreter required to run user code. Enable the pants.backend.python.providers.experimental.pyenv
  backend to add this support.

• Pants now understands how to parse dependencies of projects using the OpenStack/Stevedore ((https://github.com/openstack/stevedore)
  project. Enable the pants.backend.python.framework.stevedore backend to add this support.

• More Python target types now have the environment field and thus can execute in non-local environments.

Deprecations:

• The default value of CPython>=3.7,<4 for the [python].interpreter_constraints option is now
  deprecated and will be removed in Pants v2.17.x. This options sets Python interpreter
  constraints for Python-related targets which did not otherwise specify their own interpreter
  constraints. Instead of relying on this default, explicitly set the
  [python].interpreter_constraints option appropriately. We recommend constraining to a single
  interpreter minor version if you can, e.g., interpreter_constraints = ['==3.11.*'], or at least
  a small number of interpreter minor versions, e.g., interpreter_constraints = ['>=3.10,<3.12'].

• The special Python "tool lockfile" functionality is deprecated. Repos can now use regular "user lockfiles" to provide custom versions of tools.

Shell command improvements and support for ad-hoc tools

The Shell backend along with the new pants.backend.experimental.adhoc backend have received a number of improvements so that
Pants is able to invoke workflows that include tools for which Pants does not yet have a
dedicated backend.

In recognition of these improvements, the experimental_shell_command and experimental_run_shell_command target types have
graduated to no longer have the experimental_ prefix; deprecating the prefixed variants with removal in 2.17. Upgrade to use
shell_command and run_shell_command respectively instead.

The improvements include:

• The shell_command continues to provide a way to run a tool for its side effects and capture
  outputs from the tool invocation for consumption as dependencies by other target types, but has had several improvements:

  • the dependencies field has been deprecated, and separated into three fields: execution_dependencies for
    dependencies required to execute the command, output_dependencies for dependencies required to
    use the output, and runnable_dependencies for targets that need to exist as directly
    executable on the sandbox's PATH
  • the outputs field has been separated into output_files and output_directories
  • the directory in which the command is invoked can be controlled via the new workdir field
  • the location of the output files when consumed as a dependency can be controlled via the new root_output_directory field

• Preserving symlinks as symlinks in process output, rather than resolving them to their target
  file. This eliminates the need for work-arounds in shell_commands that happen to generate
  semantically-relevant symlinks, such as the contents of node_modules/.bin if using shell_command
  to invoke npm ci or npm install.

• The new adhoc_tool target type supplied by the adhoc backend allows executing most "runnable" target types (those supported for pants run goal) exported by another Pants backend in an execution sandbox goal, allowing you to capture their outputs to be captured. The outputs can then be consumed by another Pants target in a similar manner to the shell_command target type.
  "Runnable" target types include:

  • Adhoc: system_binary
  • Docker: docker_image
  • Go: go_binary
  • JVM: jvm_artifact, and deploy_jar
  • Python: pex_binary, python_requirement, python_source, and pyoxidizer_binary

• The new system_binary target type supplied by the adhoc backend wraps any externally-managed tool so that the
  tool may be invoked as a "runnable" by the adhoc_tool target type, while also allowing tool versions to be pinned for reproducibility.

• Support for running in non-local environments as specified by the environment field on shell_command, adhoc_tool, and related target types.

• Defining arbitrary shell commands as tests which can be invoked using pants test via the
  new experimental_test_shell_command target type.

• The workdir field to run_shell_command has been switched to match the documented behavior: it now runs from the same directory as the BUILD file by default. To preserve the old behavior, explicitly set workdir="/". The stabilisation also fixes other bugs with relative paths passed to the workdir field and thus may require updates.

Ad-hoc tool support, with the adhoc_tool or shell_command targets, can be used to compile third party
libraries in unsupported languages, run tools from ecosystems Pants doesn't yet directly support, and other similar code-generation tasks.
These targets can be used for first-party code generation, making use of existing backends that define
"runnable" targets if needed. For instance, exporting a OpenAPI schema from a Python API server, letting Pants
manage the interpreters and dependencies required.

Enable the pants.backend.experimental.adhoc backend to add the features supplied by the adhoc backend.
Enable the pants.backend.shell backend for the shell-related functionality including shell_command and
run_shell_command target types.

Go

The Go backend has received a large number of changes in order to be ready for Go v1.20
and to make continued progress on stabilizing the Go backend.

The main change is that Pants will now compile Go SDK packages from scratch and no longer rely on
the prebuilt package archives distributed with the Go SDK. Go v1.20 will no longer ship with those
prebuilt package archives, and so this change was necessary for Pants to continue to work
with Go v1.20 and later releases. (It is also a correctness issue since Pants will now be able to
apply compiler options consistently to both first-party Go code and Go SDK packages.)

The Pants team would appreciate if the community could try out 2.16.x with your Go code as an additional
check to ensure there are no regressions.

The Go backend also now supports:

• Environments are supported via the environment field available now on go_binary targets.
• Code coverage can now analyze coverage for packages beyond just the package under test via the new
  --go-test-cover-packages option.
• Numerous bug fixes to the Cgo support.
• The data race detector is supported via the race field now available on go_binary targets.
• Compiler and other flags may be set via the compiler_flags, linker_flags, and assembler_flags fields
  available on various target types.

Buf

Pants is now able to discover and supply configuration files for the Buf protobuf linter.

New: Preamble

The new "preamble" plugin allows formatting/linting files to ensure they start with specific text, such as a copyright header. Enable the
pants.backend.tools.preamble backend to add this support.

New: CUE

Pants now have basic support for the CUE language for defining, generating, and validating configuration data.
Enable the pants.backend.experimental.cue backend to use this support.

New: YamlLint

Pants now the supports the yamllint linter for YAML files. Enable the pants.backend.experimental.tools.yamllint backend to add this support.

New: AWS S3 support for URLs

Pants now supports referencing S3 URLs via the scheme (E.g. s3://), or authority (E.g. *.s3*amazonaws.com or s3.*amazonaws.com)
anywhere a URL can be specified.
Credentials are loaded similar to the AWS CLI (most regularly ~/.aws/credentials) and attached as headers in the request (which aren't cached or logged).

Enable the pants.backend.url_handlers.s3 backend to add this support.

New

Plugin API Changes

The URLDownloadHandler union has been added to allow plugins to modify native URL downloading, such as transforming the URL or attaching headers for auth.

The frozen_after_init decorator on dataclasses has been removed from many parts of the Pants codebase. Plugin authors should avoid use of this decorator because it will likely be removed in a future Pants version.

The deprecated Platform.current has been removed. Instead, have the affected rules take a Platform directly as a parameter and the engine will supply the relevant Platform instance.

The ToolCustomLockfile and ToolDefaultLockfile classes have been removed.

Bug Fixes

• Fix S3 URL Handler for good (Cherry-pick of #19285) (#19288)

Bug Fixes

• Improve handling of additional files in Helm unit tests (Cherry-pick of #19263) (#19266)

Documentation

• update scie-jump cache directory reference (Cherry-pick of #18485) (#19153)

• Fix docs link to pants_from_sources script. (Cherry-pick of #19088) (#19089)

Bug Fixes

• Improve handling of additional files in Helm unit tests (Cherry-pick of #19263) (#19267)

• Workaround botocore bug in S3 URL Handler backend (Cherry-pick of #19056) (#19110)

Plugin API Changes

• Deprecate using PythonBinary (Cherry-pick of #19209) (#19222)

Bug Fixes

• Improve handling of additional files in Helm unit tests (Cherry-pick of #19263) (#19268)

• Add taplo to the release (Cherry-pick of #19258) (#19259)

• Handle from foo import * wildcard imports in Rust dep inference parser (Cherry-pick of #19249) (#19255)

• Fix secondary ownership warning semantics (Cherry-pick of #19191) (#19224)

New Features

• Add support for packaging python AWS Lambda layers (#19123)

• Add visibility linter to report all violations. (#17671)

User API Changes

• Remove the old Python tool lockfile functionality. (#19218)

• Rename python_awslambda to python_aws_lambda_function (#19216)

• Deprecate the version/extra_requirements options on python tools. (#19204)

Plugin API Changes

• Deprecate using PythonBinary (#19209)

Bug Fixes

• store stdout/stderr output from tests as bytes (#19184)

• Discover Scala REPL main class and coordinates (#19189)

• Scrub the Pyenv install dir before attempting an install. (#19193)

• Include scala_artifact classpath rule (#19190)

• docker_image: secrets paths are now expanding ~ to the users home directory. (#19130)

• Support explicit dependencies with scala_artifact (#19187)

• Avoid extra . in from .. import parent imports in Rust Python dep inference (#19175)

Documentation

• Update FaaS docs for 2.18, including Lambda layers (#19217)

• Add tobni to Maintainers (#19195)

• Adjust FaaS docs for layout = "zip" (#19180)

User API Changes

• Deprecate the version/extra_requirements options on python tools. (Cherry-pick of #19204) (#19211)

• Terraform: Hashes for MacOS binaries have changed (Cherry-pick of #19004) (#19136)

• Implement layout="zip" for Lambda/GCF, deprecating lambdex (Cherry-pick of #19076) (#19120)

• Deprecate [mypy].extra_type_stubs and its lockfile. (Cherry-pick of #19084) (#19087)

• Bring the tool --export option deprecation forward. (Cherry-pick of #19079) (#19083)

Bug Fixes

• Discover Scala REPL main class and coordinates (Cherry-pick of #19189) (#19203)

• Scrub the Pyenv install dir before attempting an install. (Cherry-pick of #19193) (#19197)

• Avoid extra . in from .. import parent imports in Rust Python dep inference (Cherry-pick of #19175) (#19177)

• Don't try and garbage collect tempfiles (Cherry-pick of #19103) (#19113)

• If a shebang already exists when adding a preamble, preserve it. (Cherry-pick of #19133) (#19138)

• Workaround botocore bug in S3 URL Handler backend (Cherry-pick of #19056) (#19111)

Documentation

• Add tobni to Maintainers (Cherry-pick of #19195) (#19207)

• Adjust FaaS docs for layout = "zip" (Cherry-pick of #19180) (#19198)

• Add botocore-a-la-carte reference to s3 URL Handler (Cherry-pick of #19055) (#19109)

• add bazel-remote to list of compatible remote caches (Cherry-pick of #19041) (#19115)

• Fix docs link to pants_from_sources script. (Cherry-pick of #19088) (#19091)