Inside kube-controller-manager and cloud-controller-manager: How Kubernetes Keeps Cluster State in Sync

A deep dive into how kube-controller-manager and cloud-controller-manager reconcile state, run dozens of control loops, and keep Kubernetes self-healing

Anish Kumar

Containers and Infrastructure Specialist

Published Oct 7, 2026

Last modified Oct 7, 2026

When a Pod managed by a Deployment disappears, Kubernetes creates another one.
When a node becomes unhealthy, Kubernetes detects it and reacts.
When a Deployment changes from three replicas to six, new Pods appear.
When an HPA decides an application needs additional capacity, the replica count changes.
When a Service of type LoadBalancer is created on a cloud provider, an actual load balancer gets provisioned for it.

We often describe all of this by saying that Kubernetes is self healing, but none of these actions happen automatically just because the desired state is stored in etcd.

Something has to continuously observe that state, detect the difference between what we asked for and what currently exists, and take corrective action. Much of this reconciliation responsibility is handled by two key control plane components:

kube-controller-manager, which reconciles core Kubernetes
cloud-controller-manager, which reconciles the parts of the cluster that depend on the underlying cloud provider.

Neither of these is a single controller. Each is a process that hosts multiple independent control loops, every one responsible for reconciling a particular part of the system.

In this article, we will dive deep into kube-controller-manager and cloud-controller-manager to understand how they actually work under the hood, and how their internal architecture allows them to manage so many controllers reliably at scale.

Where Controller Managers Fit in Kubernetes

At a high level, Kubernetes contains several major components:

kube-apiserver is the communication hub. Every watch and every write from kube-controller-manager passes through here. It does not access etcd directly. Its Kubernetes API interactions go through kube-apiserver.
etcd stores Kubernetes API state, including the desired configuration declared in resource specifications. kube-controller-manager never accesses etcd directly. It observes Kubernetes objects through kube-apiserver and its local informer caches.
kube-controller-manager watches objects through the API server, compares the observed state with the desired state represented by Kubernetes API objects and issues corrective writes back through the API server when it finds a difference. A Pod disappearing, a replica count changing, a node going unhealthy all get picked up and acted by this component.
cloud-controller-manager runs the same watch, compare, correct loop as kube-controller-manager, but its controllers also interact with the cloud provider’s API to manage resources such as load balancers, node metadata, and networking. It exists as a separate process so cloud specific logic can remain outside the core kube-controller-manager binary. This is not limited to managed Kubernetes services. A self managed Kubernetes cluster running on cloud infrastructure can also use an external cloud-controller-manager provided by the cloud provider.
kube-scheduler watches for unscheduled Pods and selects a suitable node for them. Those Pods may have been created by workload controllers such as the ReplicaSet controller or created directly by a user. Several kube-controller-manager controllers wait on the scheduler’s output before they can do their own reconciliation.
kubelet reports Node and Pod status to the API server. The Node lifecycle controller watches those updates and detects when a node becomes unhealthy or unreachable, updating its state and applying the appropriate taints. The separate taint eviction controller can then evict affected Pods based on their tolerations, while workload controllers create any required replacements and the scheduler places those replacement Pods on healthy nodes.

kube-controller-manager sits between these concepts and it is responsible for following:

1
2
3
4
5
6
7
8
9
Observe current state
        ↓
Compare with desired state
        ↓
Detect a difference
        ↓
Take corrective action
        ↓
Observe again

Kubernetes describes controllers as control loops that continuously move the current cluster state closer to the desired state. This simple loop is one of the most important architectural ideas in Kubernetes.

Desired State, Current State, and Reconciliation

Every Kubernetes controller works off the same two pieces of information:

what is requested
what actually exists, currently

The desired state lives in etcd as the spec you wrote. The current state is whatever kube-controller-manager observes in the cluster at this moment. Reconciliation is just the ongoing act of comparing the two and closing the gap whenever they don’t match.

Consider this Deployment:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:1.29

The important part is the following configuration, which represents part of the desired state.

1
replicas: 3

This configuration tells Kubernetes:

1
I want three replicas of this application.

Now imagine one Pod disappears. The state becomes:

1
2
3
Desired Pods: 3
Current Pods: 2
Difference:   1

Kubernetes does not execute a predefined recovery script. Instead, one of its controllers (the ReplicaSet controller, in this case) notices the difference and performs another reconciliation.

1
2
3
4
5
6
7
Desired = 3
Current = 2
     ↓
Create 1 Pod
     ↓
Desired = 3
Current = 3

The controller does not finish permanently at this point. It continues observing the system because the state can change again. This is why Kubernetes controllers are called reconciliation loops.

Inside kube-controller-manager: One Process, Many Controllers

The name can be slightly misleading. kube-controller-manager is a single executable, but it hosts dozens of these independent controllers rather than being one itself. kube-controller-manager ships with 49 built-in controllers as of the Kubernetes 1.37 command reference, including:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
deployment-controller
replicaset-controller
statefulset-controller
daemonset-controller
job-controller
cronjob-controller
node-lifecycle-controller
horizontal-pod-autoscaler-controller
namespace-controller
resourcequota-controller
garbage-collector-controller
endpointslice-controller
persistentvolume-binder-controller
persistentvolume-attach-detach-controller
resourceclaim-controller
podgroup-protection-controller
storage-version-migrator-controller
...

The --controllers flag decides which of these are actually enabled, and passing * turns on everything that’s enabled by default.

The architectural point that matters is that these controllers are largely independent of each other. They happen to run inside the same process because doing so lets them share infrastructure such as API clients, caches, and informers, while keeping deployment and operations simpler than running 49 separate binaries.

Inside cloud-controller-manager

cloud-controller-manager follows the same pattern as kube-controller-manager, just with a smaller, cloud-focused set of built-in controllers. These reconcile the parts of the cluster that depend on the underlying cloud provider’s API, and include:

1
2
3
service-controller
node-controller
route-controller

The service controller watches Services of type LoadBalancer and provisions or updates the cloud provider’s load balancer to match.

The node controller enriches Node objects with cloud-specific metadata, such as instance type and zone, and removes Node objects for instances that no longer exist in the cloud.

The route controller configures the provider’s networking so Pod traffic can route correctly between nodes.

Some cloud provider implementations split Node handling into a separate Node controller and Node lifecycle controller, so the exact set of controllers can vary between providers.

Each of these still follows the same observe, compare, correct loop as kube-controller-manager’s controllers. A key difference is that these controllers reconcile Kubernetes objects with state obtained from the cloud provider API, such as instances, routes, and load balancers.

From Deployment to Running Pods: Reconciliation in Action

Let’s see the kube-controller-manager in action. Suppose we run:

1
kubectl create deployment web --image=nginx --replicas=3

The request reaches kube-apiserver, where it goes through authentication, authorization, admission processing, and validation. Once it passes all of that, the Deployment object gets persisted in etcd. No Pod exists yet at this point, storing the Deployment doesn’t create one.

Everything from here happens through a chain of independent controllers, each reacting to what the previous one produced.

Here’s how that plays out, one control loop at a time:

The Deployment controller notices the new Deployment and creates a matching ReplicaSet for it.
The ReplicaSet controller notices the ReplicaSet wants three replicas but has zero Pods, and creates three Pod objects to close the gap.
Those Pods exist but have no node assigned yet.
The scheduler picks a suitable node for each one.
The kubelet on each selected node sees a Pod assigned to it and starts the containers.

There is no central workflow engine coordinating this entire sequence. Each control loop responds to the state it observes and contributes its part toward convergence.

This loose coupling is an important reason Kubernetes can recover from failures without needing to replay one giant orchestration workflow.

Inside the Controller Loop: Informers, Caches, and Work Queues

The basic loop behind every controller sounds almost too simple:

1
Watch → Compare → Act → Repeat

The hard part is making that loop work across potentially hundreds of thousands of objects without hammering kube-apiserver on every pass. Let’s dive deep to understand what sits behind that loop internally:

Controllers cannot efficiently perform something like this forever:

1
2
3
4
5
GET every Deployment
GET every ReplicaSet
GET every Pod
wait
repeat

This approach doesn’t scale on a large cluster because that would generate enormous API traffic. Controllers avoid this with the list and watch model instead:

A Reflector obtains an initial snapshot of the resources it needs and then maintains a WATCH stream for subsequent changes. In current client-go versions, that initial state can be delivered through a streaming watch using sendInitialEvents, with a traditional LIST request available as a fallback. Conceptually, the model remains the same: build an initial view of the resource state, then continuously watch for changes.
RealFIFO sits behind the Reflector and queues the changes it receives. Each notification carries the type of change and the affected object. RealFIFO preserves these notifications in the order they arrive and does not deduplicate intermediate updates, ensuring that each change is passed on for processing in sequence.
The Shared Informer reads off that queue and does two things with each change:
- hands the change to whichever controller code is listening for it.
- updates the Indexer, a local, in-memory copy of every object it’s watching, so the cache always reflects what just happened

This cache is what makes reconciliation cheap: reading an object’s current state means reading it from memory, not making another round trip to the API server.

The word shared is important because if the Deployment controller, the ReplicaSet controller, and the Endpoint controller all need Pod data, kube-controller-manager doesn’t spin up three separate list/watch connections and three separate Pod caches. They register against the same Shared Informer for Pods and read from the same Indexer. This architecture keeps both memory use and API load far lower than one controller per resource type would produce.

Level Driven Reconciliation: Working from the Latest State

An informer event doesn’t trigger reconciliation logic directly inside the event handler. Instead, the handler extracts the object’s key, something like default/web (namespace/deployment object, and pushes just that key onto a work queue:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Object change
     │
     ▼
Informer event
     │
     ▼
default/web (namespace/ deployment object)
     │
     ▼
Work queue
     │
     ▼
Worker
     │
     ▼
Read latest object from cache
     │
     ▼
Reconcile

A worker later pulls the key off the queue and fetches the current version of the object from the Indexer, not whatever version triggered the event. That distinction is what makes the system resilient:

Multiple rapid changes to the same object collapse into a single key in the queue, so duplicate work doesn’t pile up.
A slow reconciliation doesn’t block the watch stream; the queue absorbs the backlog.
Failed reconciliations can be retried, with rate limiting and backoff applied per key.

Kubernetes controller is level driven because it asks for:

1
What should the system look like now?

rather than:

1
An UPDATE event happened, so execute operation X.

Imagine several updates happen quickly, for instance:

1
2
3
replicas: 3
replicas: 5
replicas: 8

A controller does not necessarily need to perform three independent scaling operations. A worker picks up the key and reads the object from cache, the spec already says 8, so it reconciles straight to 8 and the intermediate states never matter. The controller reconciles based on the latest known state, rather than treating each event as an instruction that must be executed literally.

This is also why Kubernetes tolerates duplicated, delayed, or coalesced events without falling apart. The controller was never executing instructions in the first place, it was only ever comparing desired state against current state and closing whatever gap it found.

This same reflector, informer, work queue pattern is exactly what client-go exposes to anyone building their own controller. client-go is the Go client library that Kubernetes’ own controller infrastructure is built on, so a custom controller written against it gets this same architecture as well.

How kube-controller-manager Runs in a Cluster

In a kubeadm based control plane, kube-controller-manager normally runs as a static Pod. The pod manifest can be found at:

1
/etc/kubernetes/manifests/kube-controller-manager.yaml

The kube-controller-manager pods run in kube-system namespace, you can see it using:

1
2
kubectl -n kube-system get pods \
  -l component=kube-controller-manager

The kubelet watches /etc/kubernetes/manifests and creates the control plane static Pods from those files. kubeadm also enables leader election for kube-controller-manager and kube-scheduler.

A typical manifest contains command line arguments similar to:

1
2
3
4
5
6
7
8
containers:
- command:
  - kube-controller-manager
  - --authentication-kubeconfig=/etc/kubernetes/controller-manager.conf
  - --authorization-kubeconfig=/etc/kubernetes/controller-manager.conf
  - --kubeconfig=/etc/kubernetes/controller-manager.conf
  - --leader-elect=true
  - --controllers=*

The exact arguments depend on cluster configuration and Kubernetes distribution.

Managed Kubernetes services may hide these control plane manifests entirely because the provider operates the control plane on your behalf.

Configuring kube-controller-manager

The kube-controller-manager binary has a large number of flags because individual controllers have their own tuning parameters. Generally, you should not tune most of them without a specific reason. However, several groups are worth understanding for production operations.

Selecting and Disabling Controllers

The most important flag is --controllers, and its default value is *. In Kubernetes 1.37, * means every controller that ships enabled by default and individual controllers can be toggled as shown below

1
2
--controllers=*,-horizontal-pod-autoscaler-controller   # disable one
--controllers=*,token-cleaner-controller                # explicitly enable one

Disabling built in controllers should be done carefully because other Kubernetes behavior may assume that those reconciliation loops are available.

Important kube-controller-manager Flags

These are the flags I would focus on rather than trying to document every available option.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|Flag                                   | Kubernetes 1.37 default| Why it matters                                    |
|---------------------------------------|------------------------|---------------------------------------------------|
|--controllers                          | *                      |Enables or disables individual controllers         |
|--leader-elect                         | true                   |Enables HA leader election                         |
|--leader-elect-lease-duration          | 15s                    |How long a leader lease stays valid without renewal|
|--leader-elect-renew-deadline          | 10s                    |How long the leader tries to renew before giving up|
|--leader-elect-retry-period            | 2s                     |Interval between election attempts                 |
|--leader-elect-resource-lock           | leases                 |Resource type used for leader locking              |
|--kube-api-qps                         | 20                     |Steady request rate toward kube-apiserver          |
|--kube-api-burst                       | 30                     |Short request bursts allowed toward kube-apiserver |
|--min-resync-period                    | 12h                    |Minimum base period for informer resync            |
|--horizontal-pod-autoscaler-sync-period| 15s                    |HPA reconciliation frequency                       |
|--node-monitor-period                  | 5s                     |Node lifecycle controller status sync interval     |
|--node-monitor-grace-period            | 50s                    |How long an unresponsive node is tolerated         |
|--terminated-pod-gc-threshold          | 12500                  |Threshold for terminated Pod garbage collection    |

There are also controller specific concurrency parameters such as:

1
2
--concurrent-deployment-syncs=5
--concurrent-gc-syncs=20

Increasing concurrency can make a controller more responsive, but it also increases CPU consumption and API traffic. Therefore, tuning kube-controller-manager is a balance between three things pulling against each other:

1
2
3
4
5
Reconciliation speed
        ↕
CPU consumption
        ↕
API server load

Pod CIDR and Node IPAM Configuration

kube-controller-manager can also hand out Pod CIDR ranges to nodes, controlled using the following options:

1
2
3
4
--allocate-node-cidrs
--cluster-cidr
--node-cidr-mask-size-ipv4
--node-cidr-mask-size-ipv6

When node CIDR allocation is enabled, the Node IPAM controller allocates Pod CIDR ranges to nodes. The exact behavior depends on the networking model and cluster configuration. These settings should always be designed together with the CNI configuration rather than changed independently on a running cluster.

Leader Election in Highly Available Control Planes

Leader election becomes important when the control plane is highly available and it has more than one node.

Suppose there are three control plane nodes, each running its own kube-controller-manager:

1
2
3
4
5
6
7
8
Control Plane 1
└── kube-controller-manager

Control Plane 2
└── kube-controller-manager

Control Plane 3
└── kube-controller-manager

Running three copies improves availability, but it also creates a coordination problem. If every kube-controller-manager instance ran the same control loops at the same time, multiple instances could try to reconcile the same resources concurrently, resulting in duplicate work and competing updates.

To avoid this, kube-controller-manager uses leader election. Only the elected leader runs the controller-manager reconciliation loops, while the other instances remain running and participate in leader election but do not execute the main controller-manager loop until they gain leadership.

They arrive at this arrangement by competing for a shared Lease object in kube-system, and only the winner starts reconciling:

HA kube-controller-manager Leader Election

The instances compete for a Kubernetes Lease object. Only the elected leader starts the normal controller execution path, while the other instances remain available to participate in future elections. You can inspect the current leader using:

1
kubectl -n kube-system get lease kube-controller-manager -o yaml

Look for the following fields in the output:

1
2
3
4
spec:
  holderIdentity: ...
  leaseDurationSeconds: 15
  renewTime: ...

If the active leader fails to renew the Lease in time, another controller-manager instance can acquire leadership and continue reconciliation.

Security, Identity, and RBAC

kube-controller-manager is a highly privileged control plane component. It needs access to many Kubernetes API resources because the individual controllers create, update and delete resources throughout the cluster.

For example, controllers may need to:

1
2
3
4
5
6
7
8
Create Pods
Update Deployments
Delete dependent resources
Approve or sign CSRs
Manage ServiceAccounts
Update Node related state
Bind PersistentVolumes
Update resource status

The component authenticates to kube-apiserver using its kubeconfig:

1
--kubeconfig=/etc/kubernetes/controller-manager.conf

It also exposes its own HTTPS endpoint, for metrics and other component-level access, secured behind authentication and authorization. In Kubernetes 1.37, the default secure port for this endpoint is 10257.

kube-controller-manager can use its shared controller-manager identity, but Kubernetes also supports giving individual controllers their own credentials through:

1
--use-service-account-credentials

When this option is enabled, individual controllers use controller specific ServiceAccounts when communicating with the API server. Kubernetes provides RBAC roles for these controller identities so each controller can operate with permissions appropriate to its responsibilities.

In kubeadm based clusters, this option is enabled by default, so controllers already use their individual ServiceAccount credentials rather than relying entirely on the shared kube-controller-manager identity.

The principle is straightforward:

1
2
3
4
5
Different controller
        ↓
Different API identity
        ↓
Permissions required for its role

This provides better separation of API permissions between controllers. The controllers still run inside the same kube-controller-manager process, so this is primarily RBAC and API identity isolation, not process isolation.

Troubleshooting kube-controller-manager

When kube-controller-manager has a problem, it rarely announces itself directly. Instead, it shows up as an application problem that just doesn’t resolve on its own:

1
2
Deployment exists
Pods are not being created

1
2
Node is NotReady
Pods are not being replaced

1
2
HPA exists
Replica count never changes

These symptoms can indicate that the relevant controller loop is not progressing, although API server, scheduler, or kubelet problems can sometimes produce similar behavior. A sensible troubleshooting order is to first verify that kube-controller-manager and the relevant controller loop are healthy.

Start by checking the kube-controller-manager Pods. If the pods are not running, nothing downstream will reconcile.

1
2
kubectl -n kube-system get pods \
  -l component=kube-controller-manager

Inspect pod logs, which is usually where the actual error surfaces.

1
2
kubectl -n kube-system logs \
  kube-controller-manager-<control-plane-node>

For an HA cluster, check the leader information and then inspect its logs.

1
2
kubectl -n kube-system get lease \
  kube-controller-manager -o yaml

For kubeadm clusters, inspect the static Pod configuration directly on the control plane node:

1
cat /etc/kubernetes/manifests/kube-controller-manager.yaml

kube-controller-manager also exposes its own component endpoints over authenticated HTTPS on its secure port, which defaults to 10257 .

Also, older articles may refer to insecure port 10252 . This port doesn’t reflect current Kubernetes architecture, so it shouldn’t be used as a reference for how the component works today.

Kubernetes 1.37: Controller and Leader Election Improvements

Kubernetes 1.37 continues to evolve several areas around controller management rather than introducing all of them from scratch. Some capabilities remain Beta or Alpha, while others are extended or graduate to Stable. Together, these changes improve leader coordination, failover behavior, cache consistency, and API storage maintenance.

Coordinated Leader Election

Traditional leader election only answers one question, which candidate currently owns the Lease.

Coordinated Leader Election adds LeaseCandidate objects and allows Kubernetes to make leader selection more deterministic, which is particularly useful during control plane upgrades where multiple component versions can temporarily coexist. This capability remains Beta and is disabled by default.

The following components can participate once the feature and its required API are enabled:

1
2
kube-controller-manager
kube-scheduler

The built-in strategy picks the candidate with the oldest emulation version first, then falls back to binary version and creation time. This lets the control plane satisfy version skew requirements mid-upgrade, rather than leaving leadership to chance.

Graceful Leader Lock Release

Another controller-manager feature worth watching in Kubernetes 1.37 is ControllerManagerReleaseLeaderElectionLockOnExit. It remains Alpha and disabled by default.

The standard behavior may require another controller-manager instance to wait until the existing Lease expires.

When this feature is enabled, kube-controller-manager can actively release its leader election lock during a controlled transition instead of waiting for the Lease TTL. The payoff is shorter failover time on graceful restarts, exactly the kind of restart that happens routinely during upgrades or maintenance.

Mitigating Stale Controller Caches

Local informer caches are what make controllers scalable, but that scalability comes with a catch. The cache can briefly lag behind kube-apiserver.

Consider a controller that performs a write:

1
2
3
4
5
Controller
    │
    │ UPDATE
    ▼
API Server

1
2
3
Controller cache
    │
    └── has not observed update yet

If a controller reconciles again immediately using that stale cache, it risks making a decision based on information that’s already outdated.

kube-controller-manager includes mitigation for the stale cache. The mechanism allows selected controllers to track the resource version of their own writes and wait until the informer cache has caught up to that version before reconciling further.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Controller writes object
        │
        ▼
Remember resourceVersion
        │
        ▼
Reconcile triggered again
        │
        ▼
Has local cache observed that version?
       / \
     No   Yes
     │      │
 Requeue   Continue

Kubernetes has been working on mechanisms to reduce problems caused by stale controller caches. In Kubernetes 1.37, this work expands further, including mitigation for the HorizontalPodAutoscaler controller along with additional mechanisms and metrics for detecting cache staleness.

This feature is particularly interesting because it exposes a detail that is normally invisible to users. Kubernetes controllers are not continuously reading perfectly current information directly from etcd. They usually operate against eventually consistent local caches.

Storage Version Migration Becomes Stable

Kubernetes resources stored in etcd have a particular storage API version. When the preferred storage version changes, existing objects can remain in the old encoding until something rewrites them.

A StorageVersionMigration object triggers the built-in:

1
storage-version-migrator-controller

This reads and rewrites existing resources into the current storage version.

Conceptually, the flow looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Existing objects
stored as old version
        │
        ▼
StorageVersionMigration
        │
        ▼
Storage Version Migrator Controller
        │
        ▼
Read and rewrite objects
        │
        ▼
Current storage version

This capability has graduated to Stable and ships enabled by default, reducing the need for manual rewrite scripts and the previous out of tree storage version migrator.

Final Thoughts: The Control Loop Behind Kubernetes

kube-controller-manager and cloud-controller-manager are some of the important components to study if you want to move beyond simply using Kubernetes and start understanding how Kubernetes actually works.

The API server gives Kubernetes its declarative interface, but declarations alone do not make anything happen. Controllers are what continuously turn those declarations into actions, whether that’s recreating a Pod, evicting Pods off an unhealthy node, or provisioning a cloud load balancer for a Service.

Understanding desired versus current state, reconciliation, shared informers, local caches, work queues, and leader election also makes the rest of kube-controller-manager’s behavior easier to reason about. It explains why it tolerates duplicate or delayed events, and why only one instance reconciles at a time in an HA cluster.

So when Kubernetes replaces a failed Pod, reacts to an unhealthy node, scales an application, or provisions a cloud resource on your behalf, it is worth remembering what’s happening underneath:

1
2
3
4
Observe
Compare
Reconcile
Repeat

That continuous feedback loop, running inside kube-controller-manager and cloud-controller-manager alike, is at the heart of Kubernetes.

If you find my work valuable and would like to support it, you are welcome to sponsor me.

Happy Learning!!!

If you found this post helpful and enjoy reading about AWS architecture, DevOps, Containers and infrastructure automation, feel free to connect with me and follow my work on other platforms — LinkedIn , Medium , Substack .

# kubernetes # devops # sre # containers

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Inside kube-controller-manager and cloud-controller-manager: How Kubernetes Keeps Cluster State in Sync

Anish Kumar

Where Controller Managers Fit in Kubernetes

Desired State, Current State, and Reconciliation

Inside kube-controller-manager: One Process, Many Controllers

Inside cloud-controller-manager

From Deployment to Running Pods: Reconciliation in Action

Inside the Controller Loop: Informers, Caches, and Work Queues

Level Driven Reconciliation: Working from the Latest State

How kube-controller-manager Runs in a Cluster

Configuring kube-controller-manager

Selecting and Disabling Controllers

Important kube-controller-manager Flags

Pod CIDR and Node IPAM Configuration

Leader Election in Highly Available Control Planes

Security, Identity, and RBAC

Troubleshooting kube-controller-manager

Kubernetes 1.37: Controller and Leader Election Improvements

Coordinated Leader Election

Graceful Leader Lock Release

Mitigating Stale Controller Caches

This feature is particularly interesting because it exposes a detail that is normally invisible to users. Kubernetes controllers are not continuously reading perfectly current information directly from etcd. They usually operate against eventually consistent local caches.

Storage Version Migration Becomes Stable

Final Thoughts: The Control Loop Behind Kubernetes