a docker development server for painless testing
A development server for serving branches of your docker-based web application an on-demand basis.
It can build images for any hash, run containers, stop containers that haven't been accessed in a while, proxy requests to the right container based on query params, etc.
git clone [email protected]:Automattic/dserve.git
cd dserve
nvm use
yarn
yarn start
You will need to modify your hosts file to include the following line:
127.0.0.1 calypso.localhost
Then you may either specify a branch or a hash to load.
At the end of the day, dserve is node express server written in typescript. It can trigger docker image builds and deletions, start and stop containers, and a few other tricks.
Here is an example flow of what happens when requesting a never-requested-before commit sha:
https://calypso.live?hash=hash
.hash
has never been requested before and needs to build an image for it. Therefore it will add the hash to the build queue and send the user a screen saying "starting a build for requested hash"./tmp/dserve-calyspo-hash/repo
and logs will be stored in /tmp/dserve-calypso-hash/dserve-build-log.txt
.index.ts: this acts as the entry point for dserve. it sets up the server, initializes the routes, and contains the request handling for each incoming route.
middlewares.ts this file contains all of the middlewares that dserve uses.
commitHash
to the express request based on the subdomain.api.ts: Contains all of the code that interfaces with external things like the fs, docker, or github. there are two kinds of entities that exist in this file, those that periodically update data and the other is helper functions for things that need to be done on-demand.
periodically repeating: updating the git branches to commit hash mapping, updating which docker images are available from the local docker server, stopping unused containers.
on-demand helper functions: this includes functionality for recording how recently a commitHash was accessed, a helper for proxying a request to the right container, helpers for checking the progress/state of a a commit, etc.
builder.ts: Contains all of the code for building the docker images for a specific commit hash. This includes making build queue and rate limiting dserve to N builds at a time.
logger.ts: Exports a couple key items around loggers including the application logger and a getter for configuring a specific logger per-docker build of commits.
Are you in a situation where you are suddently tasked with maintaining dserve even though you didn't write it? Once the flood of mixed feelings towards the original authors settles, it'd be a good idea to read this section. You probably want to know how to do things like, deploy new code, debug issues, and e2e test dserve locally. Here goes nothing:
deploying This GitHub repo is polled every 15 minutes. If there have been updates to the repo, then the latest sha is deployed. Thats it. Merge, and it'll be deployed. In a high severity situation where dserve is broken, you'll want to make sure you time your attempts to fix it before the next 15 minute mark.
debugging
dserve has a couple helpful urls for debugging issues for times when you don't have ssh access.
Note that any time you see branch=${branchName}
you can subsitute hash=${sha}
.
e2e test locally
yarn start
docker images
you can find repo-tags with the right sha to specify). After a succesfful build you should be proxied to that branch's version of calypso.fixing errors
things that have broken in the past
buildCache
issues. disabling the build cache (as a setting in the buildImage
function) until we could upgrade docker versions solved the issuething/thing2
then you cannot pull down a remote branch with the name thing
. The reason the remote repo was capable of having branch thing/thing2
is because thing
had already been deleted in its repo. The fix here is to always run a git prune
when pulling down new branches which automatically deletes the appropriates local branches that no longer exist in the remote repo.