The Ultimate Provisioner: the modern configuration management, build and automation tool
No worries of replacing anything you already setup, UPcmd does not dictate and work exclusively with other tools, rather it incorporates and collaborates with others, but it is capable to be a framework in case you do need it
You could use a quick terminal GUI to select the task to execute
All the project build, test, regression tests, the documentation site generation, publishing the tagged release and latest rolling release are using Up tasks
This is how publishing latest bleeding edge release look like when you run UP task
UP is designed and implemented to shine as a modern tool for below:
It is a build tool like Ansible, Make, Rake, Ant, Gradle, Puppet, Taskfile etc, but it is little smarter to try to make things a little easier
The goal of UP is to provide a quick (I'd say the quickest) solution to enable continuous integration and continuous deployment (CI/CD). It is simple to use and yet powerful to address many common challenges nowadays devop teams facing in the Cloud environment.
It is designed with mindful consideration of collaboration with automation in Kubernetes, helm charts, api call.
It follows best practices integrating with common CI/CD tools, such as GOCD, Jenkins, Drone, Gitlab CI and so on and is a good company of all types of CLI tools.
It brings a fun DSL programming interface, a way of modeling and engineering into CLI. It enables OO design and rapid Test Driven Development (TDD) and shorter software delivery cycle.
tasks:
-
name: task
task:
-
func: shell
do:
- echo "hello, world"
Below shows:
To deploy, simply specify a instanceid to be associated with an environment, eg: dev, staging or prod
up ngo -i staging
The config:
scopes:
- name: global
vars:
db_driver: postgres
port: 5432
- name: nonprod
members:
- dev
- staging
vars:
db_host: nonpord_database.test.host
db_user: test_db_user
db_password: could_be_encrypted_using_upcmd_too
dvars:
- name: db_password
value: '6HmsmiJIW1PfIXcF4WwOKOMDiL7PstgfKs2aRFajrwY='
- name: prod
members: [prod]
vars:
host_alias: prod
- name: dev
vars:
host_alias: dev
- name: staging
vars:
host_alias: staging
- name: prod
vars:
host_alias: prod
db_host: pord_database.proddb.host
db_user: prod_db_user
dvars:
- name: db_password
value: 'prod_encrypte_aes'
dvars:
- name: db_hostname
value: '{{.host_alias}}.myapp.com'
- name: db_url
value: 'jdbc:{{.db_driver}}://{{.db_hostname}}:{{.db_port}}/test?user={{.db_user}}&password={{.db_password}}&ssl=true'
tasks:
-
name: Main
desc: deploy my web app stack
task:
-
func: call
do:
- deploy_database
- deploy_web
-
func: shell
do:
- systemctl start my_database
- systemctl start my_web_server
-
name: deploy_web
task:
-
func: shell
do:
- deploy myweb_server
-
name: deploy_database
task:
-
func: shell
do:
- deploy mydatabase
With the evolving of the up.yml file, you could externalize the configuration to individual files or make them as module to be reused or shared. Please check out the doc for more details.
At high level, UPcmd process flows like below:
UPcmd is a generic automation tool, given your automation solution being backed by Unix Shell. You do not need Shell executable though, as it has default GOSH builtin just in case you will need one to fall back to.
There could be application as below, but not limit to:
There are 32 different distro for different combination of OS and Arch type, check them out: release
Always try to use the latest unless you have CI/CD pipeline to progressively to promote to production, then use tagged version
install_latest(){
if [ "$1" == "" ];then
echo "syntax exaple: install_latest darwin | linux | windows"
else
os=$1
curl -s https://api.github.com/repos/upcmd/up/releases \
|grep ${os}_amd64_latest \
|grep download \
|head -n 1 \
|awk '{print $2}' \
|xargs -I % curl -L % -o up \
&& chmod +x up
fi
}
install_latest darwin
install_latest linux
install_latest windows
Ensure you use go 1.14 (prefered)
go get -v github.com/upcmd/up/app/up
The up CLI command will be installed to: $HOME/go/bin, make sure you have this in your PATH
This will map your current working directory as /workspace directory inside of docker container:
docker run -it --rm -v `pwd`:/workspace docker.pkg.github.com/upcmd/up/upcli:latest /bin/sh
Or you can source this from the funcs.rc
. ./funcs.rc
run_upcli_docker
In the container:
cd /workspace
up ngo
Below is a simple greeting example, and a list, inspect and execution view of the task.
This shows:
Vars:
greet_to: Tom
weather: sunny
tasks:
-
name: task
desc: main task of hello world demo of UPcmd
task:
-
func: call
desc: greet to Tom
do:
- greet
-
func: call
desc: greet to Grace
vars:
greet_to: Grace
do:
- greet
-
func: cmd
desc: do you get the idea?
do:
- name: print
cmd: |
Have you got a little taste of using the UPcmd?
-
func: call
desc: greet to a team
vars:
team:
- Jason
- Connie
weather: stormy
loop: team
do:
- sayhi
-
name: greet
desc: greet to some one
task:
-
func: shell
desc: say hello
do:
- echo "Hello, {{.greet_to}}"
-
func: cmd
desc: talk about weather
do:
- name: print
cmd: 'It is {{.weather}}'
-
func: cmd
desc: ice break
do:
- name: print
cmd: 'What a great day!'
if: '{{eq .weather "sunny"}}'
else:
-
func: cmd
do:
- name: print
cmd: 'What a bad day!!'
-
name: sayhi
desc: say hi to some one
task:
-
func: cmd
desc: say hi to someone
do:
- name: print
cmd: 'Hi {{.loopitem}}, how are you?'
-
func: call
desc: greet to the team member
dvars:
- name: greet_to
value: '{{.loopitem}}'
do:
- greet
It demos:
Check it out yourself: source and try to have fun to run though the examples by yourself
Make was initially designed and used for building C program, even though it could be adopted for other purpose, the hard to learn trivial often causes problems than the benefits added to the team, and it is burning the brain. It is hard to make automation task extended to a more advanced level, readability degrades rapidly, and it is risky to implement critical logic using Make. Make is just a little old for modern business requirements. (Sorry, maybe this is just from one not good at using Makefile)
Rake is smart and powerful. If you don't mind learning Ruby, it is a good choice of building tool. Similarly Ant and Gradle are all bind to a language specific, it is just not right when it comes to the case that you want to automate things in cloud environment. In most cases when it requires automation in a cloud environment, in a given spun up AWS EC2 instance, a shell session, a kubernete pod, you would want something just works without any dependencies. You simply do not want to maintain the consistency of chain of upgrading path for all language packages in multiple environments. In these cases, Rake, Gradle, Ant are not best options. Due to history reasons, devops teams might have adopted them and take the advantages in the early phase. When it comes to gradual improvement and upgrade consistently in long term, the effort and cost to upgrade the whole ecosystem is just too huge and often wrong solution used in order to keep it going, until it is start sinking.
Ansible, Puppet and chef are configuration management tools. They are powerful, there are many builtin well tested modules you could use. However, Ansible might be too huge for little job. Most of the time it tends to over kill, also it suffers the same problem of python/python packages dependencies. Using it means bringing the whole hard to maintain forest of software packages, os libraries all into your execution context.
A common usage of Ansible for many teams is to use the local ssh execution with group/host vars for templating and workflow automation, which is simply not right. The way the vars managed are not fine-grained. The ansible role as a reusable module is not flexible to implement for more complicated tasks. Specifically, it does not support leaf level merge; it is hard and nearly not possible(elegantly) to do a simple validation of command line input; its controller and role one-way communication is not flexible ...
Inspired by https://taskfile.dev/, it is tiny tool making build and automation easier and elegant, however it lacks some features in a practical cloud environment for CI/CD, devops automation
With all above considerations, UP is designed to be a generic, tiny footprint (zero depedency), effective automation tool in a cloud environment.
Both UPcmd project build and the docs entire site build use the UPcmd itself
up ngo publish
build of the entire doc site using one build task:
There are over 230+ test cases, every release come with a full passed regression test of all cases defined, source
These test cases are not only about the tests, they are the usage examples with documentation self explaned
This project is under MPL-2.0 License