Where’s All My Memory Gone? Mapping K8s Memory Metrics To Physical Resources
Talkat KubeCon + CloudNativeCon Europe 2025 · on Wednesday, 2025-04-02· at 16:15· Level 1 | Hall Entrance S10 | Room C·
summary
Prometheus reads container_memory_working_set_bytes from cAdvisor (exposed by kubelet), which computes it from cgroup stats: on v2 memory.current minus inactive_file (v1 uses usage minus total_inactive_file). The pipeline involves node_exporter (not containers), libcontainer/runC, and kubelet; minikube and kubectl proxy are useful for inspection. The metric is a heuristic tied to Linux OOM scoring and QoS classes, and a small cgroup-monitoring tool demonstrates it reliably but not perfectly.
AI-written summary · about
chapters (14)
- 0:00Kubernetes memory metrics intro
- 1:08Container memory working set
- 3:19Metrics pipeline sources
- 6:34CAdvisor memory collection
- 8:19CAdvisor memory working set
- 10:03Linux cgroups accounting
- 12:52Kubernetes quality of service classes
- 15:00Cgroups memory stats
- 17:32Physical vs virtual memory
- 20:03Memory overcommit page faults
- 22:12Shared memory challenges
- 23:16File cache reclaimable memory
- 24:20Mapping to Prometheus metrics
- 26:07Analysis tools resources
related talks
29mMonitoring: Follow the Data from the Darkest Corners
20mMonitoring Energy Consumption and Building Energy Efficient Systems the Cloud Native Way
31mMonitor Your Cloud Costs Using Open Source Tooling
38mInside Kubernetes Resource Management (QoS) – Mechanics and Lessons from the Field
26mBuilding Cost Visibility For Your Kubernetes Clusters
27mWhat’s in the Box? Resource Management in Kubernetes
20mEnergy Efficient Placement of Edge Workloads
06mLearnings from Running Prometheus on Google Cloud Scale