asyncio-based parallel ssh
OTHER License
apssh
- asynchronous parallel sshThis tool leverages the Python asynchronous paradigm and comes with
apssh
command for running remote commands on a bunch of hostsappull
and appush
commands to perform parallelSshJob
objectsasynciojob
's Scheduler
forPlease refer to http://apssh.readthedocs.io/ for a complete documentation bundle.
nepi-ng
?Originally, the idea presented here addresses the needs of experimental research, where an experiment often boils down to running jobs like preparing a set of nodes, initializing them, running some bash script, collecting results, all of them having temporal relationships.
To that end, the 2 libraries asynciojobs
and apssh
complement each other
to offer a powerful programing environment that we have branded nepi-ng
; its 2 parts are thus:
asynciojobs
asynciojobs
is a microscopic orchestration scheduler for asyncio-based jobs -
see this link for
details.
This is the part that handles the temporal relationships.
apssh
apssh
ships with a few classes that allow you to write jobs in the
asynciojobs
sense, that will actually run on ssh:
SshNode
: describe how to reach a node (possible through a gateway)SshJob
: to run one or several remote commands; each of these can be
Run
: that is designed to run a command readily available on the targetRunScript
: when you have a local script file to run remotely, so there isRunString
: same idea, but you do not even have a local file, it's just aPull
or Push
for file transfers over SFTPAs the names may suggest:
SshNode
instance contains the details of the target node (hostname,SshJob
, is suitable to run as asynciojobs's
jobs, i.e. inside a scheduler;SshJob
instance contains a list of the actual commands to run, that canYou can see a very simple example of that idea implemented in 2 files
Another, simpler example, illustrates how to take advantage of the YAML loader, see
my-first-nepi-ng-script
folder indemo-v1.py
and demo-v2.py
that