Executor plugin interfacing Covalent with Slurm
APACHE-2.0 License
Bot releases are visible (Hide)
ssh_key_file
and cert_file
paths will now be expanded and resolved to their absolute paths instead of requiring to be absolute in the first place.ignore_versions
parameter to SlurmExecutor
to allow for ignoring the versions of python
, covalent
and cloudpickle
when submitting jobs on the remote machine.aiofiles
to be >=
instead of ==
.variables
parameter for environment variablescovalent
and cloudpickle
versionsCOVALENT_CONFIG_DIR=/tmp
inside sbatch script to enable filelockBaseModel
instead of dict
Path
everywhere instead of os.path
operationspoll_freq >= 10
seconds, instead of 60 secondsCOVALENT_CONFIG_DIR=/tmp
to sbatch scriptsshproxy
interface.,
to README.Published by wjcunningham7 about 1 year ago
Published by wjcunningham7 over 1 year ago
bashrc_path
, which is the path to the bashrc script to source.$HOME/.bashrc
from the SLURM submit script.conda_env
is set to False
or ""
.conda_env
from None
to ""
.ValueError
will now be raised if ssh_key_file
is not supplied.Published by wjcunningham7 over 1 year ago
use_srun
, that allows for the user to specify whether to use srun
when running the pickled Python function.sshproxy
Published by wjcunningham7 over 1 year ago
create_unique_workdir
that will create unique subfolders of the type <DISPATCH ID>/node_<NODE ID>
within remote_workdir
if set to True
cleanup = False
would be ignored.cache_dir
was not present, Covalent would crash.Published by wjcunningham7 over 1 year ago
Published by wjcunningham7 over 1 year ago
Published by wjcunningham7 over 1 year ago
README
for the remote machine's dependencies."parsable": ""
option by default if not set by the user.srun
in slurm script, instead of injecting python code and calling python directly.SlurmExecutor
to allow finer control of jobs via options for srun
and in-script commands (see README.md).srun_append
parameter allowing insertion of intermediate command (see README.md).prerun_commands
and postrun_commands
parameters allowing execution of in-script shell commands before and after the workflow submission via srun
(see README.md).cert_file
, to SlurmExecutor
that allows for a certificate file to be passed._client_connect
function to output the connection object only since the first positional argument cannot get used.Published by wjcunningham7 almost 2 years ago
Published by wjcunningham7 about 2 years ago
SlurmExecutor
can now be import directly from covalent_slurm_plugin
asyncssh
added as a requirementslurm_path
for users to provide a path for slurm commands if they aren't detected automaticallySlurmExecutor
initialization parametersasyncssh
module for a uniform interface to run ssh commands on remote machinescp
instead of rsync
result
from run
method instead of returning stdout
and stderr
as well, which are now printed directly appropriatelyPublished by wjcunningham7 about 2 years ago
covalent
version to stable
Published by wjcunningham7 about 2 years ago
cache_dir
parameter to constructorPublished by wjcunningham7 about 2 years ago
Published by wjcunningham7 about 2 years ago
Published by wjcunningham7 over 2 years ago
Published by wjcunningham7 over 2 years ago
Published by wjcunningham7 over 2 years ago
Published by wjcunningham7 over 2 years ago
self.get_status
instead of str
Published by wjcunningham7 over 2 years ago
slurm.py
Published by wjcunningham7 over 2 years ago
proc.stdout
using regex