CNCF: Kubernetes debugger erases traces — a serious problem for security audits

The Kubernetes debugger that silently erases traces

kubectl is the standard CLI tool for managing Kubernetes clusters — a container orchestration platform. The kubectl debug tool allows the introduction of temporary ephemeral containers into live pods for diagnostics without modifying the production system.

CNCF (Cloud Native Computing Foundation), the organization behind Kubernetes, has just published a concerning finding: when a kubectl debug session ends, Kubernetes deletes all data about it. Exit codes, session duration and the identity of the targeted container vanish — without a trace.

Why does the kubectl debug problem affect incident response?

Imagine the scenario: an on-call engineer is investigating an incident, notes “exit 42 — connection pool exhausted” and hands off to the next shift. The next engineer wants to verify this through the Kubernetes API — and gets a container not found error. The data only exists in notes written under stress.

The technical cause: unlike regular containers, which have lastState with a termination record, ephemeral containers have no equivalent in EphemeralContainerStatus. CNCF confirms this is a design gap in the Kubernetes specification.

Are PCI DSS, SOC 2 and HIPAA at risk?

PCI DSS requirement 10.3 mandates a detailed audit trail of every access to systems that process card data. SOC 2 access activity and HIPAA requirements point in the same direction. Organizations using kubectl debug within regulated Kubernetes clusters cannot prove to an auditor who accessed which container.

CNCF SIG Node proposes a minimal fix: adding a lastState field to EphemeralContainerStatus without a breaking change. Temporary workarounds include logging to shared volumes, monitoring via the Kubernetes watch API, and forwarding data to an external SIEM system.