Mcrouter operator for Kubernetes built with Ansible and the Operator SDK.
The Mcrouter Operator was built with the Ansible Operator SDK. It is not yet intended for production use.
Mcrouter is a memcached protocol router for scaling memcached deployments, written by Facebook.
Dylan Murray's memcached operator was the original inspiration for this operator, and this project was originally forked from the AnsOpDemo repository.
This Kubernetes Operator is meant to be deployed in your Kubernetes cluster(s) and can manage one or more mcrouter instances in the same namespace as the operator.
First you need to deploy Mcrouter Operator into your cluster:
kubectl apply -f https://raw.githubusercontent.com/geerlingguy/mcrouter-operator/master/deploy/mcrouter-operator.yaml
Then you can create instances of mcrouter, for example:
Create a file named my-mcrouter.yml
with the following contents:
---
apiVersion: mcrouter.example.com/v1alpha3
kind: Mcrouter
metadata:
name: my-mcrouter
spec:
memcached_pool_size: 3
# The memcached pool can be 'sharded' or 'replicated'.
pool_setup: replicated
Use kubectl
to create the mcrouter instance in your cluster:
kubectl apply -f my-mcrouter.yml
What's the difference between
sharded
andreplicated
:sharded
uses a key hashing algorithm to distribute Memcachedset
s andget
s among Memcached Pods; this means a keyfoo
may always go to pod A, while the keybar
always goes to pod B.replicated
sends all Memcachedset
s to all Memcached pods, and distributesget
s randomly.
There are a number of configurable options in the spec
for your Mcrouter resources. For full details on each available variable, see the mcrouter role README.
spec:
# The image to run for the `mcrouter` Deployment.
mcrouter_image: devan2502/mcrouter:latest
# The port Mcrouter will run on.
mcrouter_port: 5000
# The image to run for the `memcached` StatefulSet.
memcached_image: memcached:1.5-alpine
# The size of the memcached pool.
memcached_pool_size: 3
# The port Memcached will run on.
memcached_port: 11211
# The memcached pool can be 'sharded' or 'replicated'.
pool_setup: replicated
# Set to '/var/mcrouter/fifos' to debug mcrouter with mcpiper.
debug_fifo_root: ''
Ensure you have the testing dependencies installed (in addition to Docker):
pip install docker molecule openshift jmespath
Run the local molecule test scenario:
molecule test -s test-local
Get the Kubernetes network IP address for the mcrouter pod:
kubectl describe pod -l app=mcrouter
Then run a telnet
container to connect to mcrouter directly:
kubectl run -it --rm telnet --image=jess/telnet --restart=Never <mcrouter_pod_ip> 5000
After a few seconds you will see a message like If you don't see a command prompt, try pressing enter.
. Don't press enter, because telnet doesn't display a prompt. Instead, enter the below commands:
In the telnet prompt send commands like the following:
set mykey 0 0 5
hello
get mykey
stats
quit
You can also inspect Mcrouter fifos using mcpiper
, by setting spec.debug_fifo_root
to /var/mcrouter/fifos
, then running mcpiper
inside the mcrouter pod once it's reconfigured: /usr/local/mcrouter/install/bin/mcpiper
. Note that you will not see any output (besides maybe an error message) until requests are sent to mcrouter.
One simple way to verify Mcrouter operator is working correctly is to change the memcached_pool_size
in your Mcrouter resource, then observe what happens in the cluster:
kubectl get pods -l app=mcrouter-cache
kubectl describe pod -l app=mcrouter
--config-str
in the mcrouter container command.kubectl edit mcrouter my-mcrouter
memcached_pool_size
to 4
kubectl describe mcrouter my-mcrouter
kubectl get pods -l app=mcrouter-cache
kubectl describe pod -l app=mcrouter
--config-str
.Another thing you could do is delete one of the Memcached pods and see what happens:
kubectl delete pod mcrouter-memcached-2
Within a few seconds, you should see that the deleted pod has been replaced by Kubernetes. Because mcrouter uses DNS to distribute requests to the memcached instances, there might be a single dropped connection if there was an active request to the deleted instance, but other than that rare occurrence, the loss of a single memcached pod should have no affect on the availability of the entire cache backend.
Note that because Memcached does not include any kind of replication, and Mcrouter does not backfill new Memcached instances (they will start with an empty cache), advanced usage of Mcrouter and Memcached in high-availability, high-performance environments requires more fine tuning in the replication strategy.
Also, your application should always be able to fall back and re-set a key if a particular cache key has vanished. Basically, don't rely on Mcrouter or Memcached for a data persistence layer!
There are a few moving parts to this project:
mcrouter-operator.yaml
Kubernetes manifest file which initially deploys the Operator into a cluster.Each of these must be appropriately built in preparation for a new tag:
Run the following command inside this directory:
operator-sdk build geerlingguy/mcrouter-operator:0.2.2
Then push the generated image to Docker Hub:
docker push geerlingguy/mcrouter-operator:0.2.2
mcrouter-operator.yaml
fileUpdate the mcrouter-operator version in two places:
deploy/mcrouter-operator.yaml
: in the ansible
and operator
container definitions in the mcrouter-operator
Deployment.build/chain-operator-files.yml
: the operator_image
variable.Once the versions are updated, run the playbook in the build/
directory:
ansible-playbook chain-operator-files.yml
After it is built, test it on a local cluster:
minikube start
kubectl apply -f deploy/mcrouter-operator.yaml
kubectl apply -f deploy/crds/mcrouter_v1alpha3_mcrouter_cr.yaml
<test everything>
minikube delete
If everything works, commit the updated version, then tag a new repository release with the same tag as the Docker image pushed earlier.