One can extend the power of Kubernetes using a so called Operator. A Operator is a piece of software, that runs constantly in a loop and checks for resources to manage. A Operator normally is written in GoLang, but the Operator Framework of Kubernetes enables you also to create one using Ansible roles or playbooks.
Why to use Ansible to create an Operator you might ask? I admit, it comes with some downsides on the flexibility and power. In the end, it’s more flexible to write a pure Go program that can handle all sort of states. But I’m not Go battleproof (far from it), and so I took a look into the Ansible Operator.
In Part 1 of this series, we create an Operator, that does watch for Helloworld resources and spins up hello-world pods in a customizeable count. It’s just to get started. In part 2, we will dive down more in detail and create something with an actual use. You can find all code written in this post also in my Github repository.
So what it takes to get this started? Here are the prerequisites:
- A Kubernetes cluster
- Docker installed
- Python 3.8.6+ installed
- Ansible 2.9+ installed
- ansible-runner 2.0.2+ installed
- kubectl installed
- Operator-SDK installed
- A Docker registry access of some kind (Docker Hub, Github Container Registry…)
To create our own Operator, we use the Operator SDK. A framework that builds us the basis for an Operator. In case of an Ansible Operator, it creates the role structure and all configurations we need. Let’s start from scratch and initialize our hello-world-operator with the Operator SDK.
mkdir hello-world-operator
cd hello-world-operator
operator-sdk init hello-world-operator \
--domain thedatabase.me \
--plugins=ansible
Writing kustomize manifests for you to edit...
Next: define a resource with:
$ operator-sdk create apiAs told us from the output, the next step is to create the API.
operator-sdk create api --version v1alpha1 \
--kind Helloworld \
--generate-role
Writing kustomize manifests for you to edit..Let’s have a look what has been created by the SDK.
❯ tree .
.
├── config
│ ├── crd
│ │ ├── bases
│ │ │ └── thedatabase.me_helloworlds.yaml
│ │ └── kustomization.yaml
│ ├── default
│ │ ├── kustomization.yaml
│ │ ├── manager_auth_proxy_patch.yaml
│ │ └── manager_config_patch.yaml
│ ├── manager
│ │ ├── controller_manager_config.yaml
│ │ ├── kustomization.yaml
│ │ └── manager.yaml
│ ├── manifests
│ │ └── kustomization.yaml
│ ├── prometheus
│ │ ├── kustomization.yaml
│ │ └── monitor.yaml
│ ├── rbac
│ │ ├── auth_proxy_client_clusterrole.yaml
│ │ ├── auth_proxy_role_binding.yaml
│ │ ├── auth_proxy_role.yaml
│ │ ├── auth_proxy_service.yaml
│ │ ├── helloworld_editor_role.yaml
│ │ ├── helloworld_viewer_role.yaml
│ │ ├── kustomization.yaml
│ │ ├── leader_election_role_binding.yaml
│ │ ├── leader_election_role.yaml
│ │ ├── role_binding.yaml
│ │ ├── role.yaml
│ │ └── service_account.yaml
│ ├── samples
│ │ ├── kustomization.yaml
│ │ └── _v1alpha1_helloworld.yaml
│ ├── scorecard
│ │ ├── bases
│ │ │ └── config.yaml
│ │ ├── kustomization.yaml
│ │ └── patches
│ │ ├── basic.config.yaml
│ │ └── olm.config.yaml
│ └── testing
│ ├── debug_logs_patch.yaml
│ ├── kustomization.yaml
│ ├── manager_image.yaml
│ └── pull_policy
│ ├── Always.yaml
│ ├── IfNotPresent.yaml
│ └── Never.yaml
├── Dockerfile
├── Makefile
├── molecule
│ ├── default
│ │ ├── converge.yml
│ │ ├── create.yml
│ │ ├── destroy.yml
│ │ ├── kustomize.yml
│ │ ├── molecule.yml
│ │ ├── prepare.yml
│ │ ├── tasks
│ │ │ └── helloworld_test.yml
│ │ └── verify.yml
│ └── kind
│ ├── converge.yml
│ ├── create.yml
│ ├── destroy.yml
│ └── molecule.yml
├── playbooks
├── PROJECT
├── requirements.yml
├── roles
│ └── helloworld
│ ├── defaults
│ │ └── main.yml
│ ├── files
│ ├── handlers
│ │ └── main.yml
│ ├── meta
│ │ └── main.yml
│ ├── README.md
│ ├── tasks
│ │ └── main.yml
│ ├── templates
│ └── vars
│ └── main.yml
└── watches.yaml
28 directories, 58 filesWe concentrate for now on two things. The watches.yaml and all what’s under the roles directory. The watches.yaml tells Kubernetes for which custom resource (we configured a kind: Helloworld as custom resource. This then calls our Ansible role.
---
# Use the 'create api' subcommand to add watches to this file.
- version: v1alpha1
group: thedatabase.me
kind: Helloworld
role: helloworld
#+kubebuilder:scaffold:watchEverything that’s under the roles directory, describes our Ansible role which runs then as the upper mentioned loop. So every time, a new custom resource of kind Helloworld is created or updated, our role will be called to do it’s magic. So let’s implement in our role what we want to get done on the Kubernetes cluster. We do so in the roles/helloworld/tasks/main.yml. Let’s add this deployment:
---
- name: Start hello-world
kubernetes.core.k8s:
definition:
kind: Deployment
apiVersion: apps/v1
metadata:
name: '{{ ansible_operator_meta.name }}-helloworld'
namespace: '{{ ansible_operator_meta.namespace }}'
spec:
replicas: "{{ size }}"
selector:
matchLabels:
app: helloworld
template:
metadata:
labels:
app: helloworld
spec:
containers:
- name: helloworld
image: "docker.io/hello-world:latest"
In order to get a default value for size (so the amount of replicas), we also add the following to roles/helloworld/defaults/main.yml.
size: 1To get the operator rolled out, we need to create the container image. Here comes the container registry in place which I mentioned above. Before you push it, you need to update the Makefile you can find in the projects root folder. Change the line IMG ?= controller:latest fitting to your needs. In my case, I push my image to the Github Container Registry. Keep in mind, that you either need to provide the credentials to your private Docker registry or you make it public.
IMG ?= ghcr.io/thedatabaseme/hello-world-operator:$(VERSION)Now build and push the Operator image.
make docker-build docker-push And finally, deploy the Operator:
make deploy
cd config/manager && /usr/local/bin/kustomize edit set image controller=ghcr.io/thedatabaseme/hello-world-operator:0.0.1
/usr/local/bin/kustomize build config/default | kubectl apply -f -
namespace/hello-world-operator-system created
customresourcedefinition.apiextensions.k8s.io/helloworlds.thedatabase.me created
serviceaccount/hello-world-operator-controller-manager created
role.rbac.authorization.k8s.io/hello-world-operator-leader-election-role created
clusterrole.rbac.authorization.k8s.io/hello-world-operator-manager-role created
clusterrole.rbac.authorization.k8s.io/hello-world-operator-metrics-reader created
clusterrole.rbac.authorization.k8s.io/hello-world-operator-proxy-role created
rolebinding.rbac.authorization.k8s.io/hello-world-operator-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/hello-world-operator-manager-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/hello-world-operator-proxy-rolebinding created
configmap/hello-world-operator-manager-config created
service/hello-world-operator-controller-manager-metrics-service created
deployment.apps/hello-world-operator-controller-manager createdLet’s check the status of the Operator then:
kubectl get all -n hello-world-operator-system
NAME READY STATUS RESTARTS AGE
pod/hello-world-operator-controller-manager-5d654f8689-8stj4 2/2 Running 0 6m26s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/hello-world-operator-controller-manager-metrics-service ClusterIP 10.107.246.135 <none> 8443/TCP 8m10s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/hello-world-operator-controller-manager 1/1 1 1 8m10s
NAME DESIRED CURRENT READY AGE
replicaset.apps/hello-world-operator-controller-manager-5d654f8689 1 1 1 8m10sSo what’s left now, is to create a CRD of the kind Helloworld. Therefore, a sample manifest has been created by the SDK under config/samples/_v1alpha1_helloworld.yaml. Let’s adjust the size specification right away.
apiVersion: thedatabase.me/v1alpha1
kind: Helloworld
metadata:
name: helloworld-sample
namespace: helloworld
spec:
size: 3And apply the manifest (but create the namespace helloworld first).
kubectl create ns helloworld
namespace/helloworld created
kubectl apply -f config/samples/_v1alpha1_helloworld.yaml
helloworld.thedatabase.me/helloworld-sample created
kubectl get all -n helloworld
NAME READY STATUS RESTARTS AGE
pod/helloworld-sample-helloworld-698884b845-4jvll 0/1 Completed 3 (30s ago) 48s
pod/helloworld-sample-helloworld-698884b845-tx999 1/1 Running 3 (30s ago) 48s
pod/helloworld-sample-helloworld-698884b845-vgvfb 0/1 CrashLoopBackOff 2 (27s ago) 48s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/helloworld-sample-helloworld 1/3 3 1 48s
NAME DESIRED CURRENT READY AGE
replicaset.apps/helloworld-sample-helloworld-698884b845 3 3 1 48sAs you can see, the Operator has created a deployment which created three hello-world pods. This example is a bit misfitting, cause the hello-world pods are running only for a second and then are completed. The deployment leads to, that the pods getting restarted over and over again. But the Operator does what we wanted him to do.
So there’s only one thing missing for now. To undeploy the hello-world-operator again. This can be done by the make command also.
make undeploy
/usr/local/bin/kustomize build config/default | kubectl delete -f -
namespace "hello-world-operator-system" deleted
customresourcedefinition.apiextensions.k8s.io "helloworlds.thedatabase.me" deleted
serviceaccount "hello-world-operator-controller-manager" deleted
role.rbac.authorization.k8s.io "hello-world-operator-leader-election-role" deleted
clusterrole.rbac.authorization.k8s.io "hello-world-operator-manager-role" deleted
clusterrole.rbac.authorization.k8s.io "hello-world-operator-metrics-reader" deleted
clusterrole.rbac.authorization.k8s.io "hello-world-operator-proxy-role" deleted
rolebinding.rbac.authorization.k8s.io "hello-world-operator-leader-election-rolebinding" deleted
clusterrolebinding.rbac.authorization.k8s.io "hello-world-operator-manager-rolebinding" deleted
clusterrolebinding.rbac.authorization.k8s.io "hello-world-operator-proxy-rolebinding" deleted
configmap "hello-world-operator-manager-config" deleted
service "hello-world-operator-controller-manager-metrics-service" deleted
deployment.apps "hello-world-operator-controller-manager" deletedI hope you look forward for the second part of the series.
Philip
