A minimal CoreOS + i686 ArchLinux setup
MIT License
Helper scripts and a manual for a minimal CoreOS + ArchLinux i686 setup
coreos_production_vmware_image.vmdk.bz2
somewhere for easy reversion. You can also repack it with another archiver for faster unpacking.$ETCD_PORT
, $SSH_PORT
and $REGISTRY_PORT
with 3 random integers between 1025 and 65535. Use 4001, 22 and 5000 if you don't care.user_data.registry.yaml
and user_data.slave.yaml
.- hostname: ${COREOS_HOSTNAME}
- ssh_authorized_keys:
- ${SSH_KEY}
registry
and slave
as respective hostnames if you don't care.vmx
and .vmdk
to 2 folders: registry
and slave
..vmx
from registry/
folder in Vmware Player.$public_ipv4
in user_data.registry.yaml
bash make_config_iso.sh user_data.registry.yaml registry
registry-config.iso
into it.$REGISTRY_IP
with same ip as $public_ipv4
bash make_config_iso.sh user_data.slave.yaml slave
.vmx
from slave/
in VmWare Player. Stop it immediately.slave-config.iso
into it.Proceed to CoreOS - Validation
step.
$ETCD_PORT
, $SSH_PORT
and $REGISTRY_PORT
with 3 random integers between 1025 and 65535. Use 4001, 22 and 5000 if you don't care.Create Droplet
registry
if you don't careams3
for Europe, nyc3
for US. Note that not all regions support User data
feature we need.User Data
checkbox, paste contents of user_data.registry.yaml
$REGISTRY_IP
with IP of the master node.Create Droplet
slave
if you don't careUser data
support.User Data
checkbox, paste contents of user_data.slave.yaml
You can hack create.sh
to fully automate node creation using DigitalOcean API v2. It needs node.js for proper JSON escaping so it works together with create.js
.
Before running create.sh
you need to get your personal token from DigitalOcean control panel. Use read TOKEN; export TOKEN
to paste the token to a terminal to avoid your token in shell history and similar logs.
If you are curious, @
syntax is the feature of curl
and <()
is a feature of shell.
At this point steps are the same for all platforms.
ssh -P $SSH_PORT core@$public_ipv4
fleetctl list-machines
. It should show 2 machines.If both steps succeed - it means that SSH, fleet and etcd work.
registry.service
to the master node using sftp
or scp
or terminal copy-paste.fleetctl start registry.service
.fleetctl list-units
shows that registry
is running.app-image/$APP_NAME
, so .hg
folder is at app-image/$APP_NAME/.hg
.depends=
. The syntax is shell array syntax, so depends=('foo' bar')
works. If in doubt, use pkgname -r
tool to find which package provides a file.pkgbuild
in app-image/
folder.At this point $APP_NAME-r$rev_num-$rev_hash-$pkgrel.pkg.tar.xz
file should be created. $pkgrel
is taken from PKGBUILD
and $rev_num
/rev-hash
from Mercurial, to provide a reliable identification of build. Note that the build ignores the working tree completely and only operates on what is stored in .hg
. So it is impossible to include uncommitted changes by mistake.
pacstrap.sh package.pkg.tar.xz
tar --numeric-owner --xattrs --acls -C newroot -c . | docker import - $APP_NAME-intermediate
docker run --name $APP_NAME-intermediate -ti $APP_NAME-intermediate command_line
. Use other arguments such as -v
for volumes, -w
for current directory etc.docker commit $APP_NAME-intermediate $REGISTRY_IP:$REGISTRY_PORT/$APP_NAME:r$rev_num-$rev_hash-$pkgrel
$REGISTRY_IP:$REGISTRY_PORT/$APP_NAME:r$rev_num-$rev_hash-$pkgrel
appears in docker images
.docker run --rm $REGISTRY_IP:$REGISTRY_PORT/$APP_NAME:r$rev_num-$rev_hash-$pkgrel
to ensure that -w
and command_line
arguments are remembered and no longer necessary. If no testing is necessary, docker create
steps can be used instead of docker run
.docker push $REGISTRY_IP:$REGISTRY_PORT/$APP_NAME:r$rev_num-$rev_hash-$pkgrel
.yourapp.service
to $APP_NAME.service
and upload it to the master node.fleetctl start $APP_NAME.service
fleetctl list-units
shows 2 instances of $APP_NAME.service
are running - one on master, one on slave.docker build -t archlinux-builder
in docker-builder
archlinux-builder
image is created using docker images
Proposed workflow:
Makefile
supporting make
and make DESTDIR=... install
PKGBUILD
.pkg.tar.xz
Pacman package versioned by Mercurial revision number and hash.pacman
docker import
. Container introspection to see versions of constituents is possible.docker push
fleetctl destroy && fleetctl start
and a global fleet unit.Features:
etcd
redundancy, no locksmith
reboot scheduling, no etcd
authenticationcloud-config
for one master node: contains fleet
, image registry and etcd
cloud-config
for slaves: contains only fleet
fleet
: the app is deployed everywhere including the master nodecloud-config
.iso files for VmWare Player - so a "secure" production VmWare image can be usedntpd
is disabled: out of the box it answers remote NTP queries so it is a potential remote exploitation loopholesystemd-timesyncd
is enabled instead. It still comes with stock CoreOS along with ntpd
so no third-party software is usedetcd
as there are no peersfleetctl
only needs a local Unix socketetcd
and registry
are only running during software updates.fleetd
operates by polling etcd
so it can start SSH back on request.PKGBUILD
pulling version tags from a local Mercurial DBarchlinux-base.sh
to build a base archlinux-i686 image.pacstrap.sh
to build a minimal image using a novel approach: the image doesn't include base so it's truly minimal!docker-builder
image to serve as Arch-based toolbox
$APP_NAME
- a short name used as Archlinux package name, image name, systemd unit name, container name and folder name of Mercurial source tree.$DESCRIPTION
- used as Archlinux package description and systemd unit description$APP_PORT
- TCP port application listens to.$REGISTRY_PORT
, $ETCD_PORT
, $SSH_PORT
- TCP ports to be used for image registry, etcd and sshd.$REGISTRY_IP
- IP address of the master node. Only used on slave nodes.[1234, 5678]
- DigitalOcean API numeric ids of SSH keys to add to provisioned nodes$HOSTNAME
- host name of DigitalOcean node. It also shows up in DigitalOcean control panel, so choose a unique value for each node. It can be either a hostname or a FQDN and it is also used for DigitalOcean reverse DNS record for node IP.SEARCH_BACKEND=
, SKIP
, DESTDIR
etc are not placeholders
$TOKEN
is not a placeholder either! Use read TOKEN; export TOKEN
to paste your personal token to a terminal to avoid your token in shell history and similar logs.
$public_ipv4
is not a placeholder. Well, it is technically a placeholder, but for DigialOcean API.