A flexible package manager that supports multiple versions, configurations, platforms, and compilers.
OTHER License
Bot releases are hidden (Show)
Published by alalazo 9 months ago
Published by tgamblin 11 months ago
v0.21.0
is a major feature release.
Better error messages with condition chaining
In v0.18, we added better error messages that could tell you what problem happened, but they couldn't tell you why it happened. 0.21
adds condition chaining to the solver, and Spack can now trace back through the conditions that led to an error and build a tree of causes potential causes and where they came from. For example:
$ spack solve hdf5 ^[email protected]
==> Error: concretization failed for the following reasons:
1. Cannot satisfy '[email protected]'
2. Cannot satisfy '[email protected]'
required because hdf5 ^[email protected] requested from CLI
3. Cannot satisfy '[email protected]:' and '[email protected]
required because hdf5 ^[email protected] requested from CLI
required because hdf5 depends on [email protected]: when @1.13:
required because hdf5 ^[email protected] requested from CLI
4. Cannot satisfy '[email protected]:' and '[email protected]
required because hdf5 depends on [email protected]:
required because hdf5 ^[email protected] requested from CLI
required because hdf5 ^[email protected] requested from CLI
More details in #40173.
OCI build caches
You can now use an arbitrary OCI registry as a build cache:
$ spack mirror add my_registry oci://user/image # Dockerhub
$ spack mirror add my_registry oci://ghcr.io/haampie/spack-test # GHCR
$ spack mirror set --push --oci-username ... --oci-password ... my_registry # set login creds
$ spack buildcache push my_registry [specs...]
And you can optionally add a base image to get runnable images:
$ spack buildcache push --base-image ubuntu:23.04 my_registry python
Pushed ... as [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
$ docker run --rm -it [image]:python-3.11.2-65txfcpqbmpawclvtasuog4yzmxwaoia.spack
This creates a container image from the Spack installations on the host system, without the need to run spack install
from a Dockerfile
or sif
file. It also addresses the inconvenience of losing binaries of dependencies when RUN spack install
fails inside docker build
.
Further, the container image layers and build cache tarballs are the same files. This means that spack install
and docker pull
use the exact same underlying binaries. If you previously used spack install
inside of docker build
, this feature helps you save storage by a factor two.
More details in #38358.
Multiple versions of build dependencies
Increasingly, complex package builds require multiple versions of some build dependencies. For example, Python packages frequently require very specific versions of setuptools
, cython
, and sometimes different physics packages require different versions of Python to build. The concretizer enforced that every solve was unified, i.e., that there only be one version of every package. The concretizer now supports "duplicate" nodes for build dependencies, but enforces unification through transitive link and run dependencies. This will allow it to better resolve complex dependency graphs in ecosystems like Python, and it also gets us very close to modeling compilers as proper dependencies.
This change required a major overhaul of the concretizer, as well as a number of performance optimizations. See #38447, #39621.
Cherry-picking virtual dependencies
You can now select only a subset of virtual dependencies from a spec that may provide more. For example, if you want mpich
to be your mpi
provider, you can be explicit by writing:
hdf5 ^[virtuals=mpi] mpich
Or, if you want to use, e.g., intel-parallel-studio
for blas
along with an external
lapack
like openblas
, you could write:
strumpack ^[virtuals=mpi] intel-parallel-studio+mkl ^[virtuals=lapack] openblas
The virtuals=mpi
is an edge attribute, and dependency edges in Spack graphs now track which virtuals they satisfied. More details in #17229 and #35322.
Note for packaging: in Spack 0.21 spec.satisfies("^virtual")
is true if and only if the package specifies depends_on("virtual")
. This is different from Spack 0.20, where depending on a provider implied depending on the virtual provided. See #41002 for an example where ^mkl
was being used to test for several mkl
providers in a package that did not depend on mkl
.
License directive
Spack packages can now have license metadata, with the new license()
directive:
license("Apache-2.0")
Licenses use SPDX identifiers, and you can use SPDX expressions to combine them:
license("Apache-2.0 OR MIT")
Like other directives in Spack, it's conditional, so you can handle complex cases like Spack itself:
license("LGPL-2.1", when="@:0.11")
license("Apache-2.0 OR MIT", when="@0.12:")
More details in #39346, #40598.
spack deconcretize
command
We are getting close to having a spack update
command for environments, but we're not quite there yet. This is the next best thing. spack deconcretize
gives you control over what you want to update in an already concrete environment. If you have an environment built with, say, meson
, and you want to update your meson
version, you can run:
spack deconcretize meson
and have everything that depends on meson
rebuilt the next time you run spack concretize
. In a future Spack version, we'll handle all of this in a single command, but for now you can use this to drop bits of your lockfile and resolve your dependencies again. More in #38803.
UI Improvements
The venerable spack info
command was looking shabby compared to the rest of Spack's UI, so we reworked it to have a bit more flair. spack info
now makes much better use of terminal space and shows variants, their values, and their descriptions much more clearly. Conditional variants are grouped separately so you can more easily understand how packages are structured. More in #40998.
spack checksum
now allows you to filter versions from your editor, or by version range. It also notifies you about potential download URL changes. See #40403.
Environments can include definitions
Spack did not previously support using include:
with The definitions section of an environment, but now it does. You can use this to curate lists of specs and more easily reuse them across environments. See #33960.
Aliases
You can now add aliases to Spack commands in config.yaml
, e.g. this might enshrine your favorite args to spack find
as spack f
:
config:
aliases:
f: find -lv
See #17229.
Improved autoloading of modules
Spack 0.20 was the first release to enable autoloading of direct dependencies in module files.
The downside of this was that module avail
and module load
tab completion would show users too many modules to choose from, and many users disabled generating modules for dependencies through exclude_implicits: true
. Further, it was necessary to keep hashes in module names to avoid file name clashes.
In this release, you can start using hide_implicits: true
instead, which exposes only explicitly installed packages to the user, while still autoloading dependencies. On top of that, you can safely use hash_length: 0
, as this config now only applies to the modules exposed to the user -- you don't have to worry about file name clashes for hidden dependencies.
Note: for tcl
this feature requires Modules 4.7 or higher.
Updated container labeling
Nightly Docker images from the develop
branch will now be tagged as :develop
and :nightly
The :latest
tag is no longer associated with :develop
, but with the latest stable release. Releases will be tagged with :{major}
, :{major}.{minor}
and :{major}.{minor}.{patch}
. ubuntu:18.04
has also been removed from the list of generated Docker images, as it is no longer supported. See #40593.
spack env activate
without arguments now loads a default
environment that you do not have to create (#40756).spack find -H
/ --hashes
: a new shortcut for piping spack find
output to other commands (#38663)spack checksum --verify
, fix --add
(#38458)default_args
context manager factors out common args for directives (#39964)spack compiler find --[no]-mixed-toolchain
lets you easily mix clang
and gfortran
on Linux (#40902)spack external find
execution is now much faster (#39843)spack location -i
now much faster on success (#40898)This release has the best Windows support of any Spack release yet, with numerous
improvements and much larger swaths of tests passing:
make
is no longer a required system dependency of Spack (#40380)--allow-root
the default and deprecate the option (#38878)/
in git versions (#39398)v0.20.0
Published by haampie 12 months ago
spack mirror set-url
would drop configured connection info (reverts #34210)Published by haampie about 1 year ago
Spack now supports Python 3.12 (#40155)
spack install --overwrite
(#40252)NameError
to be thrown (#39017)pyproject.toml
for mypy
(#38769)Published by alalazo over 1 year ago
--force
was not given (#37877)archspec
fixes (#37793)Full Changelog: https://github.com/spack/spack/compare/v0.20.0...v0.20.1
Published by tgamblin over 1 year ago
v0.20.0
is a major feature release.
requires()
directive and enhanced package requirements
We've added some more enhancements to requirements in Spack (#36286).
There is a new requires()
directive for packages. requires()
is the opposite of
conflicts()
. You can use it to impose constraints on this package when certain
conditions are met:
requires(
"%apple-clang",
when="platform=darwin",
msg="This package builds only with clang on macOS"
)
More on this in the docs.
You can also now add a when:
clause to requires:
in your packages.yaml
configuration or in an environment:
packages:
openmpi:
require:
- any_of: ["%gcc"]
when: "@:4.1.4"
message: "Only OpenMPI 4.1.5 and up can build with fancy compilers"
More details can be found here
Exact versions
Spack did not previously have a way to distinguish a version if it was a prefix of
some other version. For example, @3.2
would match 3.2
, 3.2.1
, 3.2.2
, etc. You
can now match exactly 3.2
with @=3.2
. This is useful, for example, if you need
to patch only the 3.2
version of a package. The new syntax is described in the docs.
Generally, when writing packages, you should prefer to use ranges like @3.2
over
the specific versions, as this allows the concretizer more leeway when selecting
versions of dependencies. More details and recommendations are in the packaging guide.
See #36273 for full details on the version refactor.
New testing interface
Writing package tests is now much simpler with a new test interface.
Writing a test is now as easy as adding a method that starts with test_
:
class MyPackage(Package):
...
def test_always_fails(self):
"""use assert to always fail"""
assert False
def test_example(self):
"""run installed example"""
example = which(self.prefix.bin.example)
example()
You can use Python's native assert
statement to implement your checks -- no more
need to fiddle with run_test
or other test framework methods. Spack will
introspect the class and run test_*
methods when you run spack test
,
More stable concretization
Now, spack concretize
will only concretize the new portions of the environment
and will not change existing parts of an environment unless you specify --force
.
This has always been true for unify:false
, but not for unify:true
and
unify:when_possible
environments. Now it is true for all of them (#37438, #37681).
The concretizer has a new --reuse-deps
argument that only reuses dependencies.
That is, it will always treat the roots of your environment as it would with
--fresh
. This allows you to upgrade just the roots of your environment while
keeping everything else stable (#30990).
Weekly develop snapshot releases
Since last year, we have maintained a buildcache of develop
at
https://binaries.spack.io/develop, but the cache can grow to contain so many builds
as to be unwieldy. When we get a stable develop
build, we snapshot the release and
add a corresponding tag the Spack repository. So, you can use a stack from a specific
day. There are now tags in the spack repository like:
develop-2023-05-14
develop-2023-05-18
that correspond to build caches like:
We plan to store these snapshot releases weekly.
Specs in buildcaches can be referenced by hash.
spack buildcache list
and see the hashes inspack spec
and spack install
and refer tospack install /abc123
(#35042)New package and buildcache index websites
Our public websites for searching packages have been completely revamped and updated.
You can check them out here:
Both are searchable and more interactive than before. Currently major releases are
shown; UI for browsing develop
snapshots is coming soon.
Default CMake and Meson build types are now Release
Spack has historically defaulted to building with optimization and debugging, but
packages like llvm
can be enormous with debug turned on. Our default build type for
all Spack packages is now Release
(#36679, #37436). This has a number of benefits:
NDEBUG
disables assertions, which may lead to further speedups.You can still get the old behavior back through requirements and package preferences.
spack checksum
can automatically add new versions to package (#24532)spack pkg grep
to easily search package files (#34388)maintainers
directive (#35083)spack buildcache push
(alias to buildcache create
) (#34861)-j
to control the parallelism of concretization (#37608)--exclude
option to 'spack external find' (#35013)SPACK_EDITOR
environment variableruamel.yaml
to the latest versionspack.lock
(#32801)packages
subdirectory in repositories (#36643)satisfies(..., strict=True/False)
into two functions (#35681)v0.19.0
and has been removed. v0.20.0
onlygraviton
is now cortex_a72
graviton2
is now neoverse_n1
graviton3
is now neoverse_v1
blacklist
and whitelist
in module configuration were deprecated in v0.19.0
and areexclude
and include
instead.ignore=
parameter of the extends()
directive has been removed. It was not used byv0.19.0
(#34347).spack find --bootstrap
has been removed. It was deprecated in v0.19.0
. Use spack --bootstrap find
instead (#33964).spack bootstrap trust
and spack bootstrap untrust
are now removed, having beenv0.19.0
. Use spack bootstrap enable
and spack bootstrap disable
.--mirror-name
, --mirror-url
, and --directory
options to buildcache andv0.19.0
and have now been removed. They have beenenv:
as top level environment key (#37424)$_$@$%@+$+$=
) has beenv0.15
(#10556).installer.py
: drop build edges of installed packages by default (#36707)v0.19.0
Published by haampie over 1 year ago
spack config update
now works on active environments (#36542)Published by haampie over 1 year ago
buildcache create
: make "file exists" less verbose (#35019)spack mirror create
: don't change paths to urls (#34992)install_args
(#34481)combine_phase_logs
text encoding issues (#34657)config_values.py
fixture (#33886)Published by tgamblin almost 2 years ago
v0.19.0
is a major feature release.
Package requirements
Spack's traditional package preferences
are soft, but we've added hard requriements to packages.yaml
and spack.yaml
(#32528, #32369). Package requirements use the same syntax as specs:
packages:
libfabric:
require: "@1.13.2"
mpich:
require:
- one_of: ["+cuda", "+rocm"]
More details in the docs.
Environment UI Improvements
Fewer surprising modifications to spack.yaml
(#33711):
spack install
in an environment will no longer add to the specs:
list; you'll
need to either use spack add <spec>
or spack install --add <spec>
.
Similarly, spack uninstall
will not remove from your environment's specs:
list; you'll need to use spack remove
or spack uninstall --remove
.
This will make it easier to manage an environment, as there is clear separation
between the stack to be installed (spack.yaml
/spack.lock
) and which parts of
it should be installed (spack install
/ spack uninstall
).
concretizer:unify:true
is now the default mode for new environments (#31787)
We see more users creating unify:true
environments now. Users who need
unify:false
can add it to their environment to get the old behavior. This will
concretize every spec in the environment independently.
Include environment configuration from URLs (#29026, docs)
You can now include configuration in your environment directly from a URL:
spack:
include:
- https://github.com/path/to/raw/config/compilers.yaml
Multiple Build Systems
An increasing number of packages in the ecosystem need the ability to support
multiple build systems (#30738, docs),
either across versions, across platforms, or within the same version of the software.
This has been hard to support through multiple inheritance, as methods from different
build system superclasses would conflict. package.py
files can now define separate
builder classes with installation logic for different build systems, e.g.:
class ArpackNg(CMakePackage, AutotoolsPackage):
build_system(
conditional("cmake", when="@0.64:"),
conditional("autotools", when="@:0.63"),
default="cmake",
)
class CMakeBuilder(spack.build_systems.cmake.CMakeBuilder):
def cmake_args(self):
pass
class Autotoolsbuilder(spack.build_systems.autotools.AutotoolsBuilder):
def configure_args(self):
pass
Compiler and variant propagation
Currently, compiler flags and variants are inconsistent: compiler flags set for a
package are inherited by its dependencies, while variants are not. We should have
these be consistent by allowing for inheritance to be enabled or disabled for both
variants and compiler flags.
Example syntax:
package ++variant
:package +variant
:package ~~variant
:package ~variant
:package cflags==-g
:cflags
will be propagated to dependenciespackage cflags=-g
:cflags
will NOT be propagated to dependenciesSyntax for non-boolan variants is similar to compiler flags. More in the docs for
variants and compiler flags.
Enhancements to git version specifiers
v0.18.0
added the ability to use git commits as versions. You can now use the
git.
prefix to specify git tags or branches as versions. All of these are valid git
versions in v0.19
(#31200):
foo@abcdef1234abcdef1234abcdef1234abcdef1234 # raw commit
[email protected] # commit with git prefix
[email protected] # the develop branch
[email protected] # use the 0.19 tag
v0.19
also gives you more control over how Spack interprets git versions, in case
Spack cannot detect the version from the git repository. You can suffix a git
version with =<version>
to force Spack to concretize it as a particular version
(#30998, #31914, #32257):
# use mybranch, but treat it as version 3.2 for version comparison
[email protected]=3.2
# use the given commit, but treat it as develop for version comparison
[email protected]=develop
More in the docs
Changes to Cray EX Support
Cray machines have historically had their own "platform" within Spack, because we
needed to go through the module system to leverage compilers and MPI installations on
these machines. The Cray EX programming environment now provides standalone craycc
executables and proper mpicc
wrappers, so Spack can treat EX machines like Linux
with extra packages (#29392).
We expect this to greatly reduce bugs, as external packages and compilers can now be
used by prefix instead of through modules. We will also no longer be subject to
reproducibility issues when modules change from Cray PE release to release and from
site to site. This also simplifies dealing with the underlying Linux OS on cray
systems, as Spack will properly model the machine's OS as either SuSE or RHEL.
Improvements to tests and testing in CI
spack ci generate --tests
will generate a .gitlab-ci.yml
file that not only does
builds but also runs tests for built packages (#27877). Public GitHub pipelines now
also run tests in CI.
spack test run --explicit
will only run tests for packages that are explicitly
installed, instead of all packages.
Experimental binding link model
You can add a new option to config.yaml
to make Spack embed absolute paths to
needed shared libraries in ELF executables and shared libraries on Linux (#31948, docs):
config:
shared_linking:
type: rpath
bind: true
This can improve launch time at scale for parallel applications, and it can make
installations less susceptible to environment variables like LD_LIBRARY_PATH
, even
especially when dealing with external libraries that use RUNPATH
. You can think of
this as a faster, even higher-precedence version of RPATH
.
spack spec
prints dependencies more legibly. Dependencies in the output now appearpackage.py
attributes like url
, directly in packages.yaml
RPATH
-like library model on Windows (#31930)xdist
(used in GitHub Actions) (#32361)oneapi
in our buildcache (#31781, #31804,x86_64_v3
, CUDA, ROCmSupport for Python 3.5 is dropped (#31908). Only Python 2.7 and 3.6+ are officially
supported.
This is the last Spack release that will support Python 2 (#32615). Spack v0.19
will emit a deprecation warning if you run it with Python 2, and Python 2 support will
soon be removed from the develop
branch.
LD_LIBRARY_PATH
is no longer set by default by spack load
or module loads.
Setting LD_LIBRARY_PATH
in Spack environments/modules can cause binaries from
outside of Spack to crash, and Spack's own builds use RPATH
and do not need
LD_LIBRARY_PATH
set in order to run. If you still want the old behavior, you
can run these commands to configure Spack to set LD_LIBRARY_PATH
:
spack config add modules:prefix_inspections:lib64:[LD_LIBRARY_PATH]
spack config add modules:prefix_inspections:lib:[LD_LIBRARY_PATH]
The spack:concretization:[together|separately]
has been removed after being
deprecated in v0.18
. Use concretizer:unify:[true|false]
.
config:module_roots
is no longer supported after being deprecated in v0.18
. Use
configuration in module sets instead (#28659, docs).
spack activate
and spack deactivate
are no longer supported, having been
deprecated in v0.18
. Use an environment with a view instead of
activating/deactivating (docs).
The old YAML format for buildcaches is now deprecated (#33707). If you are using an
old buildcache with YAML metadata you will need to regenerate it with JSON metadata.
spack bootstrap trust
and spack bootstrap untrust
are deprecated in favor of
spack bootstrap enable
and spack bootstrap disable
and will be removed in v0.20
.
(#33600)
The graviton2
architecture has been renamed to neoverse_n1
, and graviton3
is now neoverse_v1
. Buildcaches using the old architecture names will need to be rebuilt.
The terms blacklist
and whitelist
have been replaced with include
and exclude
in all configuration files (#31569). You can use spack config update
to
automatically fix your configuration files.
buildable:true
for an MPI implementation now overrides buildable:false
for mpi
(#18269)spack stage
: add missing --fresh and --reuse (#31626)cmake
to package scope (#31739)binutils
(#32253)v0.18.0
Published by alalazo over 2 years ago
spack external find
(#31144,#31201,#31173.#31186)spack containerize
(#29741,#31321)Full Changelog: https://github.com/spack/spack/compare/v0.18.0...v0.18.1
Published by alalazo over 2 years ago
spack stage
with custom paths (#30448)spack buildcache save-specfile
(#30637)Full Changelog: https://github.com/spack/spack/compare/v0.17.2...v0.17.3
Published by tgamblin over 2 years ago
v0.18.0
is a major feature release.
Concretizer now reuses by default
spack install --reuse
was introduced in v0.17.0
, and --reuse
is now the default concretization mode. Spack will try hard to
resolve dependencies using installed packages or binaries (#30396).
To avoid reuse and to use the latest package configurations, (the
old default), you can use spack install --fresh
, or add
configuration like this to your environment or concretizer.yaml
:
concretizer:
reuse: false
Finer-grained hashes
Spack hashes now include link
, run
, and build
dependencies,
as well as a canonical hash of package recipes. Previously, hashes
only included link
and run
dependencies (though build
dependencies were stored by environments). We coarsened the hash to
reduce churn in user installations, but the new default concretizer
behavior mitigates this concern and gets us reuse and provenance.
You will be able to see the build dependencies of new installations
with spack find
. Old installations will not change and their
hashes will not be affected. (#28156, #28504, #30717, #30861)
Improved error messages
Error handling with the new concretizer is now done with
optimization criteria rather than with unsatisfiable cores, and
Spack reports many more details about conflicting constraints.
(#30669)
Unify environments when possible
Environments have thus far supported concretization: together
or
concretization: separately
. These have been replaced by a new
preference in concretizer.yaml
:
concretizer:
unify: [true|false|when_possible]
concretizer:unify:when_possible
will try to resolve a fully
unified environment, but if it cannot, it will create multiple
configurations of some packages where it has to. For large
environments that previously had to be concretized separately, this
can result in a huge speedup (40-50x). (#28941)
Automatically find externals on Cray machines
Spack can now automatically discover installed packages in the Cray
Programming Environment by running spack external find
(or spack external read-cray-manifest
to only query the PE). Packages from
the PE (e.g., cray-mpich
are added to the database with full
dependency information, and compilers from the PE are added to
compilers.yaml
. Available with the June 2022 release of the Cray
Programming Environment. (#24894, #30428)
New binary format and hardened signing
Spack now has an updated binary format, with improvements for
security. The new format has a detached signature file, and Spack
verifies the signature before untarring or decompressing the binary
package. The previous format embedded the signature in a tar
file, which required the client to run tar
before verifying
(#30750). Spack can still install from build caches using the old
format, but we encourage users to switch to the new format going
forward.
Production GitLab pipelines have been hardened to securely sign
binaries. There is now a separate signing stage so that signing
keys are never exposed to build system code, and signing keys are
ephemeral and only live as long as the signing pipeline stage.
(#30753)
Bootstrap mirror generation
The spack bootstrap mirror
command can automatically create a
mirror for bootstrapping the concretizer and other needed
dependencies in an air-gapped environment. (#28556)
Initial Windows support
Spack now has initial support for Windows. Spack core has been
refactored to run in the Windows environment, and a small number of
packages can now build for Windows. More details are
in the documentation
(#27021, #28385, many more)
Makefile generation
spack env depfile
can be used to generate a Makefile
from an
environment, which can be used to build packages the environment
in parallel on a single node. e.g.:
spack -e myenv env depfile > Makefile
make
Spack propagates gmake
jobserver information to builds so that
their jobs can share cores. (#30039, #30254, #30302, #30526)
New variant features
In addition to being conditional themselves, variants can now have
conditional values
that are only possible for certain configurations of a package. (#29530)
Variants can be
declared "sticky",
which prevents them from being enabled or disabled by the
concretizer. Sticky variants must be set explicitly by users
on the command line or in packages.yaml
. (#28630)
run
dependencieslink:run
(#29336)spack external find --all
finds library-only packages inconfig:license_dir
option (#30135)spack external find --path PATH
takes a custom search path (#30479)spack spec
has a new --format
argument like spack find
(#27908)spack concretize --quiet
skips printing concretized specs (#30272)spack info
now has cleaner output and displays test info (#22097)/hash
syntax to refer to concrete specs in an environment/hash
is not installed. (#30276)0.19.0-dev0
unify: when_possible
isoneapi
and dpcpp
flag support (#30783)M1
and a64fx
(#30683)Spack no longer supports Python 2.6
(#27256)
Removed deprecated --run-tests
option of spack install
;
use spack test
(#30461)
Removed deprecated spack flake8
; use spack style
(#27290)
Deprecate spack:concretization
config option; use
concretizer:unify
(#30038)
Deprecate top-level module configuration; use module sets (#28659)
spack activate
and spack deactivate
are deprecated in favor of
environments; will be removed in 0.19.0
(#29430; see also link:run
in #29336 above)
--reuse
(#30357, #30092, #29835, #29933, #28605, #29694, #28848)CMakePackage
uses CMAKE_INSTALL_RPATH_USE_LINK_PATH
(#29703)lua
support: lua-lang
virtual supports bothlua
and luajit
via new LuaPackage
build system(#28854)pip
(#27798)find_libraries
: search for both .so and .dylib on macOS (#28924)?full_index=1
for all github patches (#29239)v0.17.0
Published by alalazo over 2 years ago
Published by alalazo almost 3 years ago
--enable-locks
behavior (#24675)develop
(#27472)spack audit
: fix API calls to variants (#27713)MANPATH
can use system defaults (#21682)setdefault
subcommand to spack module tcl
(#14686)Published by tgamblin almost 3 years ago
v0.17.0
is a major feature release.
New concretizer is now default
The new concretizer introduced as an experimental feature in v0.16.0
is now the default (#25502). The new concretizer is based on the
clingo logic programming system,
and it enables us to do much higher quality and faster dependency solving
The old concretizer is still available via the concretizer: original
setting, but it is deprecated and will be removed in v0.18.0
.
Binary Bootstrapping
To make it easier to use the new concretizer and binary packages,
Spack now bootstraps clingo
and GnuPG
from public binaries. If it
is not able to bootstrap them from binaries, it installs them from
source code. With these changes, you should still be able to clone Spack
and start using it almost immediately. (#21446, #22354, #22489, #22606,
#22720, #22720, #23677, #23946, #24003, #25138, #25607, #25964, #26029,
#26399, #26599).
Reuse existing packages (experimental)
The most wanted feature from our
2020 user survey and
the most wanted Spack feature of all time (#25310). spack install
,
spack spec
, and spack concretize
now have a --reuse
option, which
causes Spack to minimize the number of rebuilds it does. The --reuse
option will try to find existing installations and binary packages locally
and in registered mirrors, and will prefer to use them over building new
versions. This will allow users to build from source far less than in
prior versions of Spack. This feature will continue to be improved, with
configuration options and better CLI expected in v0.17.1
. It will become
the default concretization mode in v0.18.0
.
Better error messages
We have improved the error messages generated by the new concretizer by
using unsatisfiable cores. Spack will now print a summary of the types
of constraints that were violated to make a spec unsatisfiable (#26719).
Conditional variants
Variants can now have a when="<spec>"
clause, allowing them to be
conditional based on the version or other attributes of a package (#24858).
Git commit versions
In an environment and on the command-line, you can now provide a full,
40-character git commit as a version for any package with a top-level
git
URL. e.g., spack install hdf5@45bb27f58240a8da7ebb4efc821a1a964d7712a8
.
Spack will compare the commit to tags in the git repository to understand
what versions it is ahead of or behind.
Override local config and cache directories
You can now set SPACK_DISABLE_LOCAL_CONFIG
to disable the ~/.spack
and
/etc/spack
configuration scopes. SPACK_USER_CACHE_PATH
allows you to
move caches out of ~/.spack
, as well (#27022, #26735). This addresses
common problems where users could not isolate CI environments from local
configuration.
Improvements to Spack Containerize
For added reproducibility, you can now pin the Spack version used by
spack containerize
(#21910). The container build will only build
with the Spack version pinned at build recipe creation instead of the
latest Spack version.
New commands for dealing with tags
The spack tags
command allows you to list tags on packages (#26136), and you
can list tests and filter tags with spack test list
(#26842).
spack diff
command can diff two installed specs (#22283, #25169)spack -c <config>
can set one-off config parameters on CLI (#22251)spack load --list
is an alias for spack find --loaded
(#27184)spack gpg
can export private key with --secret
(#22557)spack style
automatically bootstraps dependencies (#24819)spack style --fix
automatically invokes isort
(#24071)--include-build-deps
(#19955)spack audit
command for checking package constraints (#23053)CVS
repositories (yep, really) (#23212)spack monitor
lets you upload analysis about installations to aspack python --path
shows which python
Spack is using (#22006)spack env activate --temp
can create temporary environments (#25388)--preferred
and --latest
options for spack checksum
(#25830)cc
is now pure posix and runs on Alpine (#26259)SPACK_PYTHON
environment variable sets which python
spack uses (#21222)SPACK_SKIP_MODULES
lets you source setup-env.sh
faster if you don't need modules (#24545)spec.yaml
files are now spec.json
, yielding a large speed improvement (#22845)x86_64_v2
, x86_64_v3
, x86_64_v4
targetsspack arch --generic
lets you get the best generic architecture forarm
compiler on graviton2
(#24904)a64fx
(#24524),spackbot
to help package maintainers with notifications. Seespack ci rebuild
(#22887)--no-add
installs only specified specs and only if already present in… (#22657)spack buildcache sync
command (#25470)develop
and it will be unsupported in 0.18.spack setup
was deprecated in v0.16.0, and has now been removed.spack develop
and spack dev-build
.--dependencies
flag from spack load
(#25731)spack module [refresh|find|rm|loads]
, all of whichspack install
(#21319, #27012, #25314)spack config edit
now works even if spack.yaml
is broken (#24689)1.1.0:1.1
(#26402).99
's from many version ranges (#26422)CachedCMakePackage
for using *.cmake initial config files (#19316)lua-lang
allows swapping lua
and luajit
(#22492)ld.gold
and ld.lld
(#25626)$prefix/.spack
(#21179)pypi
attribute to infer homepage
/url
/list_url
(#17587)config.guess
file replacement (#26035)v0.16.0
spack test
usage in packages is increasing
Published by becker33 about 3 years ago
Published by tgamblin over 3 years ago
spack load
and other commands. (#23661)spack fetch
is now environment-aware. (#19166)clingo
-based concretizer. (#23016, #23307,clingo
from source. (#20652, #20657collections.abc
(#20441)__import__
instead of Spack package import. (#23288, #23290)--source-dir
argument for spack location
. (#22755, #22348, #22321)spack build-env
now prefers specs defined in the active environment. (#21642)from_sourcing_files
. (#22767)InternalConfigScope
. (#22609)SingleFileScope
able to repopulate the cache after clearing it. (#22559)package.py
(#21811)autogen.sh
. (#20319)-k/verify-ssl-false
in _existing_url
method. (#21864)Published by tldahlgren over 3 years ago
This minor release includes a new feature and associated fixes:
This release also contains bug fixes/enhancements for:
plus assorted documentation (#20021, #20174) and package bug fixes/enhancements
(#19617, #19933, #19986, #20006, #20097, #20198, #20794, #20906, #21411).
Published by alalazo almost 4 years ago
v0.16.0
is a major feature release.
New concretizer (experimental) Our new backtracking concretizer is
now in Spack as an experimental feature. You will need to install
clingo@master+python
and set concretizer: clingo
in config.yaml
to use it. The original concretizer is not exhaustive and is not
guaranteed to find a solution if one exists. We encourage you to use
the new concretizer and to report any bugs you find with it. We
anticipate making the new concretizer the default and including all
required dependencies for it in Spack v0.17
. For more details, see
#19501.
spack test (experimental) Users can add test()
methods to their
packages to run smoke tests on installations with the new spack test
command (the old spack test
is now spack unit-test
). spack test
is environment-aware, so you can spack install
an environment and
spack test run
smoke tests on all of its packages. Historical test
logs can be perused with spack test results
. Generic smoke tests for
MPI implementations, C, C++, and Fortran compilers as well as specific
smoke tests for 18 packages. This is marked experimental because the
test API (self.run_test()
) is likely to be change, but we encourage
users to upstream tests, and we will maintain and refactor any that
are added to mainline packages (#15702).
spack develop New spack develop
command allows you to develop
several packages at once within a Spack environment. Running
spack develop foo@v1
and spack develop bar@v2
will check
out specific versions of foo
and bar
into subdirectories, which you
can then build incrementally with spack install
(#15256).
More parallelism Spack previously installed the dependencies of a
single spec in parallel. Entire environments can now be installed in
parallel, greatly accelerating builds of large environments. get
parallelism from individual specs. Spack now parallelizes entire
environment builds (#18131).
Customizable base images for spack containerize
spack containerize
previously only output a Dockerfile
based
on official Spack images from Dockerhub. You may now specify
any base image of your choosing (#15028).
more external finding spack external find
was added in v0.15
,
but only cmake
had support. spack external find
can now find
bison
, cuda
, findutils
, flex
, git
, lustre
m4
, mpich
,
mvapich2
, ncurses
, openmpi
, perl
, spectrum-mpi
, tar
, and
texinfo
on your system and add them automatically to
packages.yaml
.
Support aocc, nvhpc, and oneapi compilers We are aggressively
pursuing support for the newest vendor compilers, especially those for
the U.S. exascale and pre-exascale systems. Compiler classes and
auto-detection for aocc
, nvhpc
, oneapi
are now in Spack (#19345,
#19294, #19330).
spack mark
command can be used to designate packages as explicitlyspack gc
will not garbage-collect them (#16662).install_tree
can be customized with Spack's projection format (#18341)sbang
now lives in the install_tree
so that all users can access it (#11598)csh
and tcsh
users no longer need to set SPACK_ROOT
beforesetup-env.csh
(#18225)variant=*
syntax for finding any packageSPACK_GNUPGHOME
variable for custom GPG directories (#17139)sbang
is an external package now (https://github.com/spack/sbang, #19582)archspec
is an external package now (https://github.com/archspec/archspec, #19600)spack bootstrap
was deprecated in v0.14.0, and has now been removed.spack setup
is deprecated as of v0.16.0.spack test
is now called spack unit-test
. spack test
isSome of the most notable bugfixes in this release include:
packages.yaml
(#18013)buildcache list --allarch
now works properly (#17827)Spack now has 5050 total packages, 720 of which were added since v0.15
.
hip
, aomp
, more) added by AMD (#19957, #19832, others)llvm-flang
, flang
, and f18
removed, as llvm
has real flang
spack external find
and spack test
in packages.gitlab.spack.io
develop