
Monitoring Kubernetes with CloudWatch Container Insights on Amazon EKS
Pod-level metrics, container logs, and a live EKS operations dashboard
Running a Kubernetes cluster without proper observability is like driving with your eyes closed. You know pods are running, but you have no idea which ones are consuming the most memory, which nodes are under pressure, or where that intermittent crash is coming from.
CloudWatch Container Insights solves this for Amazon EKS — it gives you pod-level metrics, container logs, and a live performance dashboard, all integrated into the same CloudWatch you already use for your other AWS resources.
In this demo, I set up Container Insights on the EKS cluster from the previous demos and explored the data it collects.
What Container Insights Provides
Standard CloudWatch: EC2 metrics → CPU, Memory at instance level
Container Insights adds: Cluster level → Node count, overall CPU/Memory Node level → Per-node resource utilization Pod level → CPU and Memory per pod Container → Individual container metrics Logs → Pod application logs Node host logs Kubernetes dataplane logs
Architecture
EKS Cluster (aiops-eks) │ ├── CloudWatch Agent (DaemonSet) │ Collects: CPU, Memory, Network, Disk metrics │ Destination: /aws/containerinsights/aiops-eks/performance │ └── FluentBit (DaemonSet) Collects: Application logs, Host logs, Dataplane logs Destinations: /aws/containerinsights/aiops-eks/application /aws/containerinsights/aiops-eks/host /aws/containerinsights/aiops-eks/dataplaneTwo DaemonSets run on every node — CloudWatch Agent for metrics, FluentBit for logs. A DaemonSet ensures exactly one pod per node regardless of how many nodes exist.
Step 1 — IAM Permissions
The node group IAM role needs permission to write metrics and logs to CloudWatch:
IAM → Roles → (node group role) → Attach policy: CloudWatchAgentServerPolicyWithout this, the CloudWatch Agent pods will run but silently fail to publish any data.
Step 2 — CloudWatch Agent (Metrics)
The CloudWatch Agent was deployed as a DaemonSet using AWS’s official manifest. A ConfigMap defines what to collect:
{ "logs": { "metrics_collected": { "kubernetes": { "cluster_name": "aiops-eks", "metrics_collection_interval": 60 } }, "force_flush_interval": 5 }}kubectl get daemonset -n amazon-cloudwatch# NAME DESIRED CURRENT READY# cloudwatch-agent 1 1 1 ✅Step 3 — FluentBit (Logs)
FluentBit requires a ConfigMap named
fluent-bit-cluster-info with cluster details before it will start. This is a common setup mistake — deploying the DaemonSet before creating the ConfigMap causes CrashLoopBackOff:Error: configmap "fluent-bit-cluster-info" not foundThe fix is straightforward — create the ConfigMap first:
kubectl create configmap fluent-bit-cluster-info \ --from-literal=cluster.name=aiops-eks \ --from-literal=http.server=On \ --from-literal=http.port=2020 \ --from-literal=read.head=Off \ --from-literal=read.tail=On \ --from-literal=logs.region=eu-central-1 \ -n amazon-cloudwatchAfter the ConfigMap was created and the DaemonSet restarted, both pods came up healthy:
kubectl get pods -n amazon-cloudwatch# NAME READY STATUS# cloudwatch-agent-xxxxx 1/1 Running ✅# fluent-bit-xxxxx 1/1 Running ✅Step 4 — Container Insights Dashboard
Once both DaemonSets were running, metrics started flowing into CloudWatch within 2–3 minutes.
CloudWatch → Container Insights → Performance monitoring → aiops-eks
The dashboard provides multiple views:
Cluster view:
Nodes status : 2 ReadyNode CPU util : 4.84%Node Memory util : 27.8%Container restarts : 0Pod CPU and Memory over pod limit: Both charts show live utilization relative to the resource limits defined in the pod spec — this is what makes Container Insights more useful than raw CloudWatch metrics. You see utilization in context of what the pod is actually allowed to use.
Additional views available:
- Namespaces — resource breakdown per namespace
- Nodes — per-node CPU and memory
- Services — network metrics per service
- Workloads — deployment-level view
- Pods — individual pod metrics
- Containers — container-level granularity



Step 5 — Log Groups Created
After FluentBit started, four log groups appeared automatically:
/aws/containerinsights/aiops-eks/application ← Nginx and app container logs/aws/containerinsights/aiops-eks/host ← Node OS logs/aws/containerinsights/aiops-eks/performance ← CloudWatch Agent metrics/aws/containerinsights/aiops-eks/dataplane ← Kubernetes control plane logsStep 6 — Querying Logs with CloudWatch Logs Insights
CloudWatch → Logs Insights → /aws/containerinsights/aiops-eks/application
Query:fields @timestamp, kubernetes.pod_name, log| filter kubernetes.namespace_name = "aiops-demo"| sort @timestamp desc| limit 20This returns the last 20 log lines from every container running in the
aiops-demo namespace — no SSH, no kubectl exec required.
Key Lessons
The FluentBit ConfigMap must exist before the DaemonSet is deployed. If you deploy the DaemonSet first, it enters CrashLoopBackOff and keeps pulling the image every 30 seconds. Create the ConfigMap first, then apply the DaemonSet — or restart it after the ConfigMap is created.
IAM permissions are always the first thing to verify. The CloudWatch Agent ran without errors but published no data until
CloudWatchAgentServerPolicy was attached to the node role. If your metrics are missing, check IAM before debugging the agent configuration.Container Insights gives you context that raw metrics cannot. Knowing a pod uses 27.8% memory is useful — knowing it uses 27.8% of its 256Mi limit is actionable. Container Insights shows utilization relative to resource limits, which is what actually matters for capacity planning.
DaemonSets automatically scale with your cluster. When a new node is added by Cluster Autoscaler, the CloudWatch Agent and FluentBit pods are automatically scheduled on it. You do not need to do anything — monitoring scales with your cluster.
Container Insights vs Standard CloudWatch
Capability Standard CloudWatch Container Insights
EC2 instance metrics ✅ ✅
Pod-level metrics ❌ ✅
Container-level metrics ❌ ✅
Application logs❌ ✅ (FluentBit)
K8s-aware dashboard ❌ ✅
Auto-scales with cluster N/A ✅ (DaemonSet)
Full Source Code
All manifests and setup scripts used in this demo are available on GitHub:
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article