Create Kubernetes ConfigMaps from configuration files
APACHE-2.0 License
Turn configuration files on disk into structured Kubernetes ConfigMaps
At Tumblr, we have many sources for logical configuration data that defines how deployed services operate. Some of these sources are rapidly changing, some are updated automatically, and some are changed by humans. Because we always strive to limit complexity in our stack, we wanted to come up with a way that applications can be made aware of these changes as they happen, without becoming tightly coupled to our internal processes. Additionally, because there are many configuration sources (git repos, bots, cronjobs, tools like Collins, etc), and we wanted to write tools to manipulate this configuration data after being generated, we generate intermediary representations in a config
git repo. However, because each application has very unique needs for what data it requires to run (eg TLS certs, comma separated lists of host names, env vars, specifically formatted JSON or YAML), we needed a method to transform the common configuration files into formats each application could consume or further process, without hardcoding configuration into our images, or requiring a rebuild/redeploy of the app each time configuration data changes. To accomplish this task, we built the k8s-config-projector
.
The k8s-config-projector is a program to project config files from disk into Kubernetes ConfigMap
s through Continuous Integration (CI). The ConfigMap
s can be applied to a k8s cluster with kubectl apply -f <filename>
(or create
+replace
) during Continuous Delivery (CD). These ConfigMap
s can then be consumed by applications in Kubernetes, by mounting them as volumes or environment variables inside pods.
This is quite useful to inform applications in Kubernetes about changing configuration. For example, this can be used to discover the nodes participating in a memcached ring, configure an application with feature flags, etc.
To use this tool, it is expected that:
config
repoIf you want to use this to manage secrets, you should check out k8s-secret-projector instead!
+--------------------------+
| |
| Projection Mappings Repo +-------------+ YAML Projection
| (--manifests) | | Manifests
| | |
+--------------------------+ |
|
|
+-----------------+ +---v--------------------+ +------------------+
| | config sources | | | |
| Config Repo +------------------> | generates configmaps | Output Directory |
| (--config-repo) | | k8s-config-projector +-----------------------> (--output) |
| | | | | |
+-----------------+ | | +---------+--------+
+------------------------+ |
|
|
| creates ConfigMaps
+----------------------+ | in Kubernetes
| | |
| kubectl create -f <----------------+
| |
+----------------------+
The projector can:
You should get started by checking out some example projection manifests.
DataSource
schema documentation: datasource.md
Check out hacking.md for how to build this!
--config-repo
checked out, and location in ${CONFIG_REPO}
--manifests
) in ${MANIFESTS_REPO}
${OUTPUT_DIR}
to keep the generated manifests the tool spits out$ docker run -it --rm \
-v "${OUTPUT_DIR}:/output" \
-v "${MANIFESTS_REPO}:/manifests:ro" \
-v "${CONFIG_REPO}:/config:ro" \
tumblr/k8s-config-projector:latest
Collect your generated ConfigMaps in ${OUTPUT_DIR}
!
$ make && ./bin/k8s-config-projector --debug --manifests=${MANIFESTS_REPO} --config-repo=${CONFIG_REPO} --output=${OUTPUT_DIR}
...
Collect your generated ConfigMaps in ${OUTPUT_DIR}
!
Example config map:
apiVersion: v1
kind: ConfigMap
metadata:
creationTimestamp: null
name: test-config-1
namespace: foo-bar
data:
test_config.txt: |
this is a config file with a whole bunch of stuff
to show that configs can be any form
blah
blah
blah
Example pod config:
apiVersion: v1
kind: Pod
metadata:
name: test-pod1
namespace: mynamespace
spec:
containers:
- name: busybox-test
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "cat /etc/config/test-config.txt" ]
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
name: test-config-1
items:
- key: test_config.txt
path: test-config.txt
restartPolicy: Never
There are a few things to note when using the Config Projector. They are actually limits of Kubernetes/etcd
, but it still is useful to be aware of them.
etcd
. This has a 1M limit of each object stored.kubectl apply
creates annotations, which has a size limit of 256K. This translates to a ConfigMap that can't be over ~512K. If you want to avoid this, use kubectl create
instead.For reference, these are size limits discussions:
Copyright 2019, Tumblr, Inc.