rebar3

New Features:

• add a new shell completion provider
• zsh support for aliases in completion
• Add new 'shell_hooks_env' config to extend shell hooks' OS env vars
• Add (newer) Dialyzer info. about invalid_contract
• Add command rebar3 alias

Bug fixes:

• Bump relx and erlware commons versions
• REBAR_SRC_DIRS and REBAR_APP_DIRS fix by making rebar_dir:src_dirs/1 options default to ["src"]
• Eliminate use of experimental function code:lib_dir/2
• Fix "clean" hooks in non-umbrella apps and when override are presents
• Initial version of the Manifest plugin
• Make log_level entry in rebar.app work again
• Append . to --eval arg if missing
• Handle 'or' in versions in a primitive manner
• Fall back to reading versions from hex metadata
• Dedupe compiler DAG edge insertion for artifacts
• Add missing doc. for incremental on Dialyzer analysis
• rebar_utils: add metadata to primary logger config
• Prevent XRef issue when analysis runs on generated code
• Fix unicode output printed as charlist
• Rework argument parsing in do/as providers
• Fix for rebar3 shell in Erlang 26 when ShellArgs==undefined
• Erlang 26 custom shell fix
• Add app name to "OTP release ~ts or later is required" msg

This is a small patch release for some bugfixes:

• Defer hostname check to OTP when supported
• Print diagnostic when failing to get VSN with Git
• Add simple Erlang LS config

This release most importantly contains patches to fix blocking issues on Windows:

• Avoid setting a shell-encoding in OTP-26, which causes the shell command to hang on windows
• Support CRLF in shell ENV files

This release also contains an experimental and optional change that warrants a bit of explanation: Support rich compiler messages in errors and in warnings

Given the module:

-module(fake_mod).

-export([diagnostic/1]).

diagnostic(A) ->
    X = add(5 / 0),
    {X,X}.

add(X) -> X.

add(X, Y) -> X + Y.

Calling rebar3 compile can now yield:

...
===> Compiling apps/rebar/src/fake_mod.erl failed
   ┌─ apps/rebar/src/fake_mod.erl:
   │
 5 │  diagnostic(A) ->
   │             ╰── variable 'A' is unused

   ┌─ apps/rebar/src/fake_mod.erl:
   │
 6 │      X = add(5 / 0),
   │                ╰── evaluation of operator '/'/2 will fail with a 'badarith' exception

    ┌─ apps/rebar/src/fake_mod.erl:
    │
 11 │  add(X, Y) -> X + Y.
    │  ╰── function add/2 is unused

and in a terminal supporting color output:

By default, this format is turned off, but can be turned on optionally by configuring values with

{compiler_error_format, rich}.

You can get this enabled. You may want to put it in your global rebar3 config file. That being said, this may break some tooling that could parse rebar3-specific output. The default value is {compiler_error_format, minimal}. and putting that value back in should fix any breakage. We're also expecting color output to have a few bugs related to a rather naive heuristic used to delimit language constructs, but we wanted to make the feature available for feedback and this bug-fixing release was a decent opportunity.

This functionality relies on an expanded compiler API exposed by Rebar3, but since we're not yet quite sure what form we want it to take, we'll keep the new API undocumented in the short term.

New Features:

• Support OTP-26, Deprecate OTP-23 support.
• Add incremental dialyzer support for OTP-26

Various fixes:

• bumping a bunch of deps, including certificates, smaller patches in dependencies, and the following relx changes:
  • Also set -dist_listen false for relx_nodetool calls
  • Force use of nodetool if proto_dist set
  • Only overwrite erl with dyn_erl if dyn_erl exists
• Add rebar_parallel pool, use in DAG scans which limits memory usage on large projects
• Maintain cth_log_redirect default in CT runs to show logger messages in HTML test logs
• Change license template to Markdown
• rebar3_dialyzer_format: Add missing message formatting
• Always dereference symbolic links when copying files in ct
• Add ?RTX_LOG macro option to top-level rebar.config
• fix help shell output, --eval is misindeted
• Ignore _vendor folder
• Do not ignore everything starting with underscore recursively
• scope local install VSN variable
• pkgs command should support empty lists

This release took a while to build; OTP-25.1 had incompatibilities with EUnit that interrupted the CI process, and which are corrected in later releases.

New features:

• Add --eval option to shell
• Add support for dialyzer --incremental when available
• Vendor structure

This new vendor structure should be of no impact to anyone using Rebar3 as a built artifact, but should make the life of corporate users and package managers much easier by having pre-bundled in all dependencies and allowing effective offline builds. It also simplifies our bootstrap code and reduces security issues by not having to download a certificate bundle without certificates, at least until our oldest supported version can use Erlang's new built-in on-host certificate lookup mechanisms.

If you do rely on Rebar3 source for your tooling or any other mechanisms, then you will want to cd apps/rebar before running other commands.

Other fixes:

• Warn of EUnit issues on OTP-25.1
• Respect HEX_MIRROR environment variable
• Use shell:start_interactive to start shell if available
• Do not send empty If-None-Match in tarball request
• Respect rebar_packages_cdn set in rebar.config
• Dereference symbolic links when copying files from ct data folder
• Reduce warnings around bootstrap and DAG operations

New features:

• Add --offline option and REBAR_OFFLINE environment variable
• Add support for project-local plugins in umbrella projects only
• Add eunit --test flag

Experimental features for which we promise no backwards compatibility in
the near future:

• Experimental vendoring provider
  • Support plugins in experimental vendor provider

Other changes:

• Support OTP 23..25 inclusively
• Bump Relx to 4.7.0
  • Use erlexec directly in relx helper functions
  • Make rlx_util:parse_vsn parse integer versions
  • fix awk script check_name() in extended_bin
  • avoid crash when overlay is malformed
  • keep attributes when stripping beams
  • Fix {include_erts,true} in Windows releases
  • ensure the erl file is writable before copying dyn_erl to it
  • Various tests added
• Properly carry overlay_vars settings for files in relx
• Track mib compilation artifacts
• Attempt to find apps in git subdirs (sparse checkouts)
• Do not discard parameters --system_libs and --include-erts when duplicate values exist
• Use default depth parameter for SSL
• Fix global cache config overriding
• Error out on unknown templates in 'new' command
• Fix a typo
• Bump certifi to 2.9.0
• add a debug message in internal dependency fetching code
• Use SPDX id for license in template and test
• Use default branch for git and git_subdir resources with no revision

• add support for otp24 optional applications
• use git command in non interactive mode
• git_resource: ignore stderr in get_patch_count when calling git rev-list
• fix NIF compilation on Apple M1.
• fix issue for duplicates options of 'parserfile' in 'yrl_opts'
• support for --all argument in unlock
• support for --all argument in upgrade
• support for --all argument in upgrade plugin
• add --relnames option to allow selectively building of multiple releases
• bump bbmustache to support unicode attributes
• proper respect of GLOBAL_CACHE_DIR
• bump certifi to 2.8.0
• don't crash on printing non-ASCII characters
• upgrade providers to 1.9.0

This release is mostly bug fixes, and repairs prior broken builds of elixir dependencies, but also includes a couple of useful features.

New Features:

• Add escript_incl_priv so escriptize priv dir inclusion works with _checkouts and profiles
• Added SSL cacert patch to support Corporate MITM Proxies

Bug fixes:

• rebar3 release to honour the ignore-xref attribute
• Bump relx to 4.5.0
  • use copied erts dir when tar'ing even when the user sets the erts
  • improve error message when a symlink creation fails
  • State extended with a filter method to filter xref_warnings
  • Do not require logger
  • Add default time warp mode of multi
  • optionally allow static node name prefixes
  • use random:uniform instead of os:pid when constructing node name in nodetool
  • Fix eval command to use ERL_DIST_PORT, consistency with rpc command
  • Fix bin script arguments to erlexec
• Give compiler hints when handling run-time deps in parse_transforms
• Make rebar_file_utils:system_tmpdir/1 take TMPDIR env var into account on *nix
• Handle abstract code starting with a non-file attribute
• Prefer erlang app to mix project
• Drop bootstrap URI handling functions warnings
• Prevent crashes on recursive src_dir definitions in deps
• Fix ssl check hostname options for wildcard certificate
• {xref_ignores} now properly works on module,
• Normalize Dialyzer PLT paths
• Fix optimistic registry update of plugins being upgraded

Security release. See https://ferd.ca/you-ve-got-to-upgrade-rebar3.html

Major security fix around TLS validation. See https://ferd.ca/you-ve-got-to-upgrade-rebar3.html

Also:

• local upgrade: extract vsn of rebar3 from downloaded escript

Major security fix around TLS validation. See https://ferd.ca/you-ve-got-to-upgrade-rebar3.html

This release drops multiple OTP versions as officially supported. We're aligning rebar3 development with the OTP team's compatibility guarantees, as some deprecations from OTP-21 were removed in OTP-24 and we just could not maintain compatible code bases. We're using the breakage opportunity to change just how much we support to something more sustainable.

The release also contains extra fixes that should prove useful.

• Only support 3 newest OTP versions
• add support to configure application resource naming
• Support ERL_LIBS libraries in escript building
• Fix CLI arg order in makefile template
• Increase OTP 24 readiness + move to OTP 20 min.
• Bump erlware_commons and certifi
• Treat applications and included_applications the same way when builing Dialyzer PLTs
• Strip CLI args from provider hooks
• Fix plugins upgrade for non-tuple specification

• Fix bad compiler path nesting

• Make dialyzer output file format configurable
• Fix DAG resolving for recursive directories
• Add EUnit support for sys_config options
• Catch and report missing escript dependency
• Relay unexpected relx return values
• Bump Relx and cth_readable
  • relx_resolve:to_app raise error with .app file pathname
  • Concurrent Builds - xref interactions refinement
  • Add richer return values for nodetool rpc
  • add exports of RELEASE_NAME, RELEASE_VERSION and RELEASE_PROG to script
• add an option to select the application to build an escript for
• Do not discard parameters --system_libs and --include-erts
• Have "rebar3 plugins upgrade" work without specifying plugin name
• rebar3 shell: implement various modes to print "booted" message
• More path juggling fixes in the compiler
• Fix OTP24 complaining about underscored variables
• Handle OTP-24 function location in xref
• pass no_spawn_compiler_process to compile:file/2

This is a point release containing various bug fixes, some performance enhancements, and adding support for OTP-24.

• Common test: make group paths work for --group "[g1,g2],g3" option
• Adapt to upcoming OTP 24 dialyzer output
• Sanitize repo configuration in DEBUG output
• Modify algorithm to decide compilation order between apps to resolve header dependencies
• Use rebar_packages_cdn for fetching package resource
• bump relx to 4.3.0
• Compile nothing if there are no extra virtual apps (performance increase)
• Print actural download repo in DEBUG output
• bump cth readable to 1.5.0
• Make sure dialyzer static analysis is performed at least once, in CI
• Use "git rev-list --count" to count refs (performance increase)
• OTP-24 Readiness
• fix injection of cth_readable to allow for hook configuration
• Fixed link to list of plugins.
• rebar_compiler: fix DAG and speed-up analysis for large repositories
• use newest certifi release

• bump relx to v4.2.0
  • See changes in relx v4.2.0 here
• Add support for relx config file

Here's all the changes that may impact some behaviour:

• Lots of Relx-related fixes as 4.x stabilizes, including:
  • fixes in handling of system_libs
  • bringing back start and attach commands (with warnings)
  • additional bugfixes for embedded, NixOS, and windows systems
• Rework DEBUG=1 mode to be user-centric output and introduce separate DIAGNOSTIC=1 mode for implementer-level logging (see also: Reworking more of the DEBUG output to be friendly)
• Trigger rebuilds when OTP compiler version changes (also includes fixes to plugins using checkout directories)
• Fix endless recursion in del_dep overrides
• Fix lock file not being generated when no deps are present in the config
• Change xref paths to only those of runtime deps
• Fix compile issue on windows when OS language is not english
• fix logger reset for the shell provider
• Re-vendor hex_core in at 0.7.x
• Fix relup command arguments and add validation
• Add checkouts location back to rebar3 path
• Support pre-fetched deps for nix
• set explicit encoding when reading and writing hex config
• Update references to changed API elements
• Remove verbosity option that was taken out in relx 4.x
• Run pkg updates on plugin upgrades (see also: Fixup plugin upgrade autoupdate)
• Make rebar3 templates check for name clashes
• Patch rebar_base_compiler to handle extensions with no leading dot (legacy compiler interface)
• Handle unicode in author's name in templates
• Specify dependencies for MIB files
• Ignore global and default in empty profiles warning for as
• Use infinity timeout for rebar_compiler_epp resolve call
• Add alias to example rebar.config.sample
• Fix dialyzer warning output for option unknown
• Add xref_ignores to sample config file
• Prevent xref crash when function deemed undefined has no local context
• Output relative paths in Dialyzer output (now controlled by the compiler_source_format option)

And some Internal Changes:

• Prevent dialyzer warnings on OTP 23
• Add a docker recipe for rebar3 itself
• Fix spec for download callback
• Add rebar_hex_repos:remove_from_auth_config/2
• Bumping cth_readable version and other small cleanups
• Add regression test for dep override infinite loop
• Fix README (see also: Update README.md and Update README with supported Erlang/OTP versions)
• Update link for Rebar 2.x to Rebar3
• use 'aws cp' when copying escript to s3

Contributions from 17 people are part of this release, and the project health is starting to look better; thanks to everyone involved!

• git lock: abort if version is not equal to or greater than 1.8.5
• require the app name as the last part of git_subdir path
• Fix Recursive Behaviour Lookup Regression
• bump relx to 4.0.2

New Features (since 3.13.2):

• git_subdir dependencies (and #2290) allow to only get a subdirectory of a given git repository. This change will not work with older rebar3 versions and may error out their build if used, although an identical plugin could be written to patch up for that use case. Declare the dependency as:
  • {git_subdir, Url, {branch, Name}, Dir}
  • {git_subdir, Url, {tag, Ref}, Dir}
  • {git_subdir, Url, {ref, Ref}, Dir}
• Add powershell script generation option for escriptize on windows; use for rebar3 itself
• dialyzer support: run single-app success typing and store project apps in PLT
• Add symlink to the ct_run of the last common test run
• ipv6 support for proxy configuration
• Source file encoding specifier support for templates
• Allow passing arguments in r3:do and r3:async_do
• Add templating feature for "from" file/template

Relx Changes (including more new features):

• relx 4.0 support, see the full release notes for the lot of new features and changes it contains
• add --all option for release building in parallel

Compiler Changes:

• Update compiler hooks order and internal compiler phases
• Split up the compiler DAG
• Switch to EPP analysis covering all apps at once when compiling
• Track build artifacts in DAG
• Another Compiler Refactor: Performance & Cleanup
• Name the app being escriptized to help readability when escriptize is run as a hook
• Additional performance improvements for large projects
• Fix directory recursion in compiler
• prv_plugins_upgrade: fix compilation of upgraded plugins

Other Changes:

• OTP-23 readiness
• Update hex core to v0.6.8, which includes some security work, and creates a new lockfile version. Related: eliminate lock file warnings when upgrading from 1.1.0 locks
• Add vendor_hex_core.sh script
• Allow users to override default hex auth config
• Fix tests in Windows; adjust absolute paths behaviour for OTP-22 and later
• output checkouts compilation to _build/<profile>/checkouts/, which helps prevent long-standing issues with the ebin/ directory being left around in the source location
• Support ENV var for mix in the bare compiler (related to - Allow specifying the outdir in bare compiler added in 3.13.1)
• checking epmd is started without a crash for features requiring named nodes
• Fix typo in abort message
• Newer version of the certifi certs bundle; a good thing to keep up
• Annotate generated escript with last release version for quicker version checks without calling the tool
• Simplify windows script
• Return an error if no apps are found when using bare compiler
• xref now outputs relative file paths
• Move absolute_path/2 and normalized_path to rebar_file_utils
• rebar_git_resource: fix {vsn, git} parser
• Fix race condition in rebar_prv_shell boot
• re-enable use of rebar_packages_cdn in rebar_pkg_resource
• only write lock file if locks have changed
• rebar_git_resource: allow non-single-digit git versions
• rebar_dir: return _checkouts path in a canonical form
• Bump cth_readable (1.4.8) to fix assertion macros
• Proper Unicode handling in port process output
• get a sorted list of modules for coverage output (screen and HTML)
• Run dialyzer for extra_src_dirs
• remove warning when booting applications in the shell
• Fix bootstrap crashes when compilers get modified
• Remove erl_interface from makefile template which is deprecated.
• Improve warnings about checkout dependencies when locking and upgrading
• Bump a bunch of dependencies
• Extract rebar3 libs in a version dependent directory when using rebar3 local commands
• Feature shell.script_file in rebar.config.sample
• Add extra pattern matching clause to not crash handling with keep_logs in rebar3 ct
• Fix symlink to last CT logs directory with logdir option present
• Fix keep_logs option crashing when having more than N directories
• shell: provide error message when applications list is bad
• Match on dep name in overrides
• Fix use of ct on projects without CT suites and configuration
• properly track app vsn that is the evaluated application vsn rather than just the source one
• Escape more characters for shell stuff

New Features (since 3.13.2):

• git_subdir dependencies (and #2290) allow to only get a subdirectory of a given git repository. This change will not work with older rebar3 versions and may error out their build if used, although an identical plugin could be written to patch up for that use case. Declare the dependency as:
  • {git_subdir, Url, Dir}
  • {git_subdir, Url, {tag, Ref}, Dir}
  • {git_subdir, Url, {ref, Ref}, Dir}
• Add powershell script generation option for escriptize on windows; use for rebar3 itself
• dialyzer support: run single-app success typing and store project apps in PLT
• Add symlink to the ct_run of the last common test run
• ipv6 support for proxy configuration
• Source file encoding specifier support for templates
• Allow passing arguments in r3:do and r3:async_do

Relx Changes (including more new features):

• relx 4.0 support, see the full release notes from rc-1 and the changes in rc-2 for the lot of new features and changes it contains
• add --all option for release building in parallel

Compiler Changes:

• Update compiler hooks order and internal compiler phases
• Split up the compiler DAG
• Switch to EPP analysis covering all apps at once when compiling
• Track build artifacts in DAG
• Another Compiler Refactor: Performance & Cleanup
• Name the app being escriptized to help readability when escriptize is run as a hook

Other Changes:

• OTP-23 readiness
• Update hex core to v0.6.8, which includes some security work, and creates a new lockfile version. Related: eliminate lock file warnings when upgrading from 1.1.0 locks
• Add vendor_hex_core.sh script
• Allow users to override default hex auth config
• Fix tests in Windows; adjust absolute paths behaviour for OTP-22 and later
• output checkouts compilation to _build/<profile>/checkouts/, which helps prevent long-standing issues with the ebin/ directory being left around in the source location
• Support ENV var for mix in the bare compiler (related to - Allow specifying the outdir in bare compiler added in 3.13.1)
• checking epmd is started without a crash for features requiring named nodes
• Fix typo in abort message
• Newer version of the certifi certs bundle; a good thing to keep up
• Annotate generated escript with last release version for quicker version checks without calling the tool
• Simplify windows script
• Return an error if no apps are found when using bare compiler
• xref now outputs relative file paths
• Move absolute_path/2 and normalized_path to rebar_file_utils
• rebar_git_resource: fix {vsn, git} parser
• Fix race condition in rebar_prv_shell boot
• re-enable use of rebar_packages_cdn in rebar_pkg_resource
• only write lock file if locks have changed
• rebar_git_resource: allow non-single-digit git versions
• rebar_dir: return _checkouts path in a canonical form
• Bump cth_readable (1.4.8) to fix assertion macros
• Proper Unicode handling in port process output
• get a sorted list of modules for coverage output (screen and HTML)
• Run dialyzer for extra_src_dirs
• remove warning when booting applications in the shell
• Fix bootstrap crashes when compilers get modified