DevOps & cloud-native field notes from production.
Kubernetes, Redpanda, Elasticsearch, and the unglamorous work of keeping stateful workloads alive — with real numbers.
- Monitoring Kubernetes Volume Usage with kubelet_volume_stats_used_bytes (and Its Gotchas)
The kubelet already exposes per-volume usage metrics to Prometheus. Here's the PromQL we use for dashboards and alerts, plus the five gotchas that will mislead you if you trust the numbers blindly.
- Redpanda Operator 26.2.1 Recreated Our PVCs: Why We Removed the Decommission Controller
A routine operator upgrade changed a StatefulSet template label. The new StatefulSetDecommissioner treated it as a decommission event and started recreating PVCs. Here's the incident, the root cause, and the fix.
- How We Cut ~$1,600/Month on EBS by Rightsizing Redpanda Volumes on Kubernetes
We were paying for 20 TiB of EBS gp3 for two Redpanda clusters while using less than half of it. Here's the exact measurement and migration process we used to rightsize stateful volumes you can't shrink in place.