Docker image for Nomad, based on official Consul image
MPL-2.0 License
This repo contains the Dockerfiles and shell scripts for my Nomad Docker image on Docker Hub, based on the official Consul image.
Make sure you check out the Nomad homepage for more information on Nomad itself.
As this image is based on the Consul image, many of the same idioms apply to this image. The construction is similar, using Alpine Linux as the base. However, since Nomad uses cgo and is dynamically linked, the binary is custom built against a Alpine beforehand and uploaded to the image in the place of the upstream binary.
This doc assumes you are familiar with Nomad - if you need to learn how to use it specifically, make sure you check out the homepage.
Running this container with no arguments will load the container in development mode.
docker run --net=host --rm vancluever/nomad
--net=host
is important to ensure that you will be able to reach respective
ports from the host.
Note that in this very basic form you will not be able to do much as Nomad's privileges will be highly restricted within the container. Continue reading for further details.
Keep in mind that by running this container you are, in fact, running any scheduling operations executed by this instance of Nomad within the container. This means a few things:
NOMAD_RUN_ROOT
. For more details, see below.SYS_ADMIN
capabilities, which are not granted to thecap-add
(again, see/nomad/data
) from the hostNOMAD_DATA_DIR
must also be set with thisNote that depending on how you run your Nomad jobs, privileges, capabilities, or shared directories above what is mentioned may also be needed.
The config and data directories have their permissions reset to that
of the running Nomad process (usually root:nomad
or nomad:nomad
, depending
on the value of NOMAD_RUN_ROOT
) to ensure proper access to the data. Keep this
in mind when designing your system's filesystem layout.
The following examples are basic examples of what is necessary to run the Nomad image and have it successfully execute containers. The key takeaways are:
/tmp
is shared with the host so that the default syslog sockets that Nomad/tmp/nomad/data
is shared from the host at the same pathNOMAD_DATA_DIR
is set to the appropriateNOMAD_RUN_ROOT
works to grant appropriate permissions for access such toSYS_ADMIN
capabilities are added so that Nomad can perform generaldocker run \
--cap-add=SYS_ADMIN \
--env=NOMAD_DATA_DIR=/tmp/nomad/data \
--env=NOMAD_RUN_ROOT=1 \
--net=host \
--rm \
--volume=/tmp:/tmp \
--volume=/tmp/nomad/data:/tmp/nomad/data \
--volume=/var/run/docker.sock:/var/run/docker.sock \
vancluever/nomad
Note the extra options as shown below to not only publish the API endpoint, but also to ensure Nomad listens on all IPv4 addresses.
docker run \
--cap-add=SYS_ADMIN \
--env=NOMAD_DATA_DIR=/tmp/nomad/data \
--env=NOMAD_RUN_ROOT=1 \
--publish=4646:4646 \
--rm \
--volume=/tmp:/tmp \
--volume=/tmp/nomad/data:/tmp/nomad/data \
--volume=/var/run/docker.sock:/var/run/docker.sock \
vancluever/nomad agent -dev -bind=0.0.0.0
If you run the Nomad image as a service (such as on CoreOS), you will want to do something along the lines of the following:
[Unit]
Description=HashiCorp Nomad
After=docker.service
Requires=docker.service
[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill nomad
ExecStartPre=-/usr/bin/docker rm nomad
ExecStartPre=/usr/bin/docker pull nomad:VERSION
ExecStart=/usr/bin/docker run \
--name=nomad \
--cap-add=SYS_ADMIN \
--env=NOMAD_RUN_ROOT=1 \
--net=host \
--rm \
--volume /nomad/config:/nomad/config \
--volume /nomad/data:/nomad/data \
--volume=/tmp:/tmp \
--volume=/var/run/docker.sock:/var/run/docker.sock \
vancluever/nomad:VERSION agent -server -client
[Install]
WantedBy=multi-user.target
Note that you will need to set VERSION to the version of Nomad you want to run.
Also, unless you plan on using a directory other than /nomad
, you don't need
to set NOMAD_DATA_DIR
, as the default data directory is not changing from the
default.
The container's entrypoint script adds some pre-existing configuration. See the script for futher details and take care to not overwrite any of the values, namely having to do with directories.
Finally note that this example also does not include any bootstrapping data or what not, and won't work on its own. You may want to set it up in tandem with the Consul image.
For further details or information on how to work with the image, or to submit an issue or a pull request, check the GitHub repository.