Kubernetes Operator vs. Controller
Introduction
The terms Kubernetes Operator and Kubernetes Controller are often used interchangeably or misunderstood, leading to confusion among developers and operators alike. While both are integral to Kubernetes' extensibility and automation capabilities, they serve distinct purposes.
In this post, I’ll clarify the differences between these two patterns, highlight their use cases, and provide updated examples to help you better understand their roles in modern Kubernetes environments.
Kubernetes Controllers
Kubernetes Controllers are the backbone of its declarative model. They implement a control loop, a concept borrowed from industrial control systems, where they continuously monitor the cluster's state and reconcile it with the desired state defined in resource specifications.
Tip
A control loop is the fundamental building block of industrial control systems.
It consists of all the physical components and control functions necessary to automatically adjust the value of a measured process variable (PV) to equal the value of a desired set-point (SP).
A Kubernetes Controller is a control loop that monitors the state of cluster resources and takes action to bring them to a desired state. According to the Kubernetes documentation, controllers follow this pattern:
- Observe the current state of cluster resources.
- Compare it with the desired state defined in a Kubernetes object’s spec.
- Act to reconcile the differences, if necessary.
Examples of Built-in Controllers
- ReplicaSet Controller: Ensures that the number of Pods matches the desired replicas specified in a ReplicaSet.
- StatefulSet Controller: Manages stateful applications by ensuring ordered deployment and scaling.
- DaemonSet Controller: Guarantees that a Pod runs on all (or specific) nodes in a cluster.
Custom use cases
Controllers aren't limited to built-in resources. Developers can create custom controllers for specific needs:
- Annotation Controller: Ensures that all Pods are annotated with a specific value.
The controller monitors Pod resources and automatically adds the annotation to all Pods upon creation or modification.
For example, the controller could automatically add the annotation
controlled-by: dumbto all Pods. - Service Enforcement Controller: Deletes any Service resources that are of type
NodePort. The controller monitors Service resources and deletes any Service created or modified to be ofNodePorttype.
These examples demonstrate that a Controller's primary focus is enforcing a desired state, regardless of whether it involves built-in or custom logic.
Kubernetes Operators
A Kubernetes Operator is a specialized controller that extends Kubernetes APIs using Custom Resource Definitions (CRDs) to manage complex applications. The term was coined by CoreOS and has since evolved into a core pattern in Kubernetes.
Operators essentially act as "human operators" codified into software. They automate tasks that would traditionally require manual intervention by an administrator.
Key characteristics of an Operator
- Uses the controller pattern to reconcile state.
- Extends the Kubernetes API with CRDs to introduce new resource types.
- Encodes operational knowledge to automate tasks such as backups, upgrades, and scaling.
- Manages a single application and its lifecycle. Unlike generic controllers, Operators are application-specific and focus on managing a single application or component.
Examples of Operators
- Strimzi Operator: Manages Apache Kafka clusters on Kubernetes.
- Prometheus Operator: Deploys and manages Prometheus monitoring instances.
- PostgreSQL Operator: Automates the deployment and management of PostgreSQL databases.
Operators are commonly built using the Operator SDK and deployed with the Operator Lifecycle Manager (OLM).
When to use a Controller vs. an Operator
A basic Controller may be sufficient if:
- Your automation only requires watching built-in Kubernetes resources.
- Your logic does not require a new API abstraction.
Use an Operator when:
- You need to introduce a new resource type specific to your application.
- The application requires domain-specific operational knowledge, such as upgrades, scaling, and failover handling.
- You want to provide users with a Kubernetes-native experience for managing the application.
Conclusion
All Operators are controllers, but not all controllers are Operators. Controllers focus on enforcing Kubernetes-native behaviors, while Operators extend Kubernetes to manage complex applications with domain-specific knowledge.
While controllers can be written in any language, using frameworks like the Operator SDK simplifies development and reduces boilerplate code. Understanding when to use a controller versus an Operator can help developers build better automation solutions for Kubernetes workloads.
References
