Just Another Boring Ops System - Jabos attempts to be a fully automated K8s GitOps framework.
MIT License
Jabos attempts to be a fully automated K8s GitOps framework.
This is WIP - any comments, requests or issues would be welcome! please use this link
Video version:
Instructions:
jabos
jabos
kubectl create namespace jabos
. Use the same namespace with the next command.kubectl apply -n <NAMESPACE> -f https://github.com/srfrnk/jabos/releases/latest/download/jabos-manifests.yaml
Jabos uses CRDs in order for users to define a codebase and how to build and deploy that.
See API Docs here
Video version:
All resources show current status using Status Sub-resource
and Event Resources
.
These can be viewed as with any K8s resource. i.e. kubectl describe git-repositories.jabos.io
GitRepository
resources have a Syncing
condition in the status.
If it becomes False
an Event
will describe the error.
They also have a Latest Commit
(latestCommit
) status containing the latest git
commit id found.
DockerImage
and ***Manifest
resources have a Synced
condition in the status.
If it becomes False
an Event
will describe the error.
They also have a Latest Commit
(latestCommit
) status containing the latest git
commit id found
and a Built Commit
(builtCommit
) status containing the git
commit id last built.
Create a file example.jsonnet
:
function(latestCommitId) {
apiVersion: 'apps/v1',
kind: 'Deployment',
metadata: {
name: 'test-deployment',
labels: {
app: 'test-deployment',
},
},
spec: {
replicas: 1,
selector: {
matchLabels: {
app: 'test-deployment',
},
},
template: {
metadata: {
labels: {
app: 'test-deployment',
},
},
spec: {
containers: [
{
name: 'test-deployment',
image: 'registry.kube-system:80/example-image:' + latestCommitId,
},
],
},
},
},
}
kubectl create secret generic -n example-env first-repo-private --from-file=git_ssh_passphrase=./build/passphrase --from-file=git_ssh_key=./build/key
)ssh
property to each applicable GitRepository
object to point to the secret.kubectl create secret generic -n example-env docker-hub --from-file=docker_hub_username=./build/docker_hub_username --from-file=docker_hub_password=./build/docker_hub_password
)dockerHub
property to any applicable DockerImage
object to point to the secret.kubectl create secret generic -n example-env gcp --from-file=gcp_service_account.json=./build/gcp_service_account.json
)gcp
property to any applicable DockerImage
object to point to the secret.Access key ID
and Secret Access Key
.kubectl create secret generic -n example-env aws --from-file=aws_access_key_id=./build/aws_access_key_id --from-file=aws_secret_access_key=./build/aws_secret_access_key
)aws
property to any applicable DockerImage
object to point to the secret.Note: You can use instance roles instead when pushing to ECR from a EC2 instance or from EKS, by configuring the instance role permissions.
All metrics are exported into Prometheus
using the ServiceMonitor
API by kube-prometheus-stack
.
To otherwise configure Prometheus
to collect the metrics you need to point it to 'OPERATOR_POD_IP:3000/metrics'.
All metrics exported are prefixed with jabos_operator_
.
Numerous metrics are exported most of them describe nodsjs
and expresjs
operations. Removed due to security audit fails
Important metrics for the operation of Jabos are:
# HELP jabos_operator_latest_commit_changed new "latest commit" detected for git repository
# TYPE jabos_operator_latest_commit_changed counter
# HELP jabos_operator_docker_image_build_trigger new build triggered for a docker image
# TYPE jabos_operator_docker_image_build_trigger counter
# HELP jabos_operator_jsonnet_manifests_build_trigger new build triggered for jsonnet manifests
# TYPE jabos_operator_jsonnet_manifests_build_trigger counter
# HELP jabos_operator_git_repository_updater_start GitRepositoryUpdater start
# TYPE jabos_operator_git_repository_updater_start counter
# HELP jabos_operator_git_repository_updater_end GitRepositoryUpdater end
# TYPE jabos_operator_git_repository_updater_end counter
# HELP jabos_operator_git_repository_updater_duration GitRepositoryUpdater duration
# TYPE jabos_operator_git_repository_updater_duration gauge
# HELP jabos_operator_docker_image_builder_start DockerImageBuilder start
# TYPE jabos_operator_docker_image_builder_start counter
# HELP jabos_operator_docker_image_builder_end DockerImageBuilder end
# TYPE jabos_operator_docker_image_builder_end counter
# HELP jabos_operator_docker_image_builder_duration DockerImageBuilder duration
# TYPE jabos_operator_docker_image_builder_duration gauge
# HELP jabos_operator_jsonnet_manifests_builder_start JsonnetManifestsBuilder start
# TYPE jabos_operator_jsonnet_manifests_builder_start counter
# HELP jabos_operator_jsonnet_manifests_builder_end JsonnetManifestsBuilder end
# TYPE jabos_operator_jsonnet_manifests_builder_end counter
# HELP jabos_operator_jsonnet_manifests_builder_duration JsonnetManifestsBuilder duration
# TYPE jabos_operator_jsonnet_manifests_builder_duration gauge
Diagrams for supported and future planned use-cases are here
Build images in DEV/QA only and reuse when commit is promoted to other environments.
To mark a DockerImage
for reuse of an image built in another environnement add build: false
to the spec.
Jabos
images and manifest are being scanned by CodeQL
and Snyk
as part of the release process using GitHub Actions.
Jabos
makes no attempt at protecting applications, networks, disks from malicious access. It is the responsibility of the user to put in place such measures.
Jabos
should always be contained inside a dedicated namespace to reduce risk to other systems.
Special attention must be given to the Jabos
docker image builder pods which use Kaniko
. At this time it is required for Kaniko
to be executed with root
user and with a writable file system. This known limitation is a low risk as these pods have a very short life span... however it does pose a risk especially when the code pulled from a Git
repository may contain vulnerabilities.
It is advisable to always scan all code which is pulled by Jabos
from Git
!
It is advisable to use NetworkPolicy
and other methods to ensure any egress from docker image builder pods is limited to what is required by your images to build!
The scan results can be found here
As of version 1.x there are no known vulnerabilities.
Create an issue here.
Please add a security
label for quicker response.
make
installed (Depending on your OS - start here)docker
installed (To install see here)minikube
installed (To install minikube see this)NodeJS
installed (To install NodeJS see this)Typescript
development tools installed npm install -g ts-node typescript '@types/node'
GNU Parallel
installed for your OS. For Debian based you can use sudo apt-get install parallel
.K9s
installed (To install see here). For automated port forwarding set K9s configuration with scanForAutoPf: true
. Make sure K9s version supports the feature (https://github.com/derailed/k9s/pull/1498).git clone [email protected]:srfrnk/jabos.git
(or using HTTPS/GitHub CLI - see instructions here)minikube start
make setup
oncemake build
after each code changeexample-env