podbench#

A development seat — editor, gdb, a Python inner loop — inside a Kubernetes pod, reached over nothing but your kubeconfig.

podbench launches a debug container into a running pod (or into a sacrificial clone of one), tunnels ssh through kubectl exec, and hands you the config stanza that VS Code Remote-SSH needs. There is no port-forward to babysit, no pod IP to reach, no file sync and no traffic interception: the editor is already inside the cluster.

              laptop
     VS Code · Remote-SSH · kubectl
                 |
                 |  ssh over kubectl exec (API server only)
                 |
  +--------------+--------------------------------------+
  |  pod                                                 |
  |                                                      |
  |  +----------------------+  +----------------------+  |
  |  | app container        |  | podbench             |  |
  |  |                      |  |                      |  |
  |  | workload process     |  | sshd · vscode-server |  |
  |  |   (maybe distroless) |  | gdb · git · uv       |  |
  |  | fs visible at        |  |                      |  |
  |  |   /proc/<pid>/root   |  |                      |  |
  |  +----------------------+  +----------------------+  |
  |                                                      |
  |  shared: PID namespace (via the target)              |
  |          network namespace (always)                  |
  +------------------------------------------------------+

Two artefacts: a debug container image (ghcr.io/gilesknap/podbench) and a launcher you never have to install — uvx podbench runs it straight from PyPI, uses your kubeconfig, and shells out to kubectl for everything it does to a cluster.

	Observe mode	Iterate mode	Hotfix mode
Command	`uvx podbench attach pod/foo`	`uvx podbench dev pod/foo`	`uvx podbench hotfix init pod/foo`
What it does	adds an ephemeral container to the live pod	authors a sacrificial clone with the app idled and podbench as a real sidecar	puts the app’s venv on a claim, so an edit survives the restart — and the restart is the relaunch
Built for	distroless targets with no shell of their own; gdb against the running workload	edit → relaunch → see the change through the Service	an emergency fix that has to outlive the session, with provenance
Resources	shares the workload’s limits, cannot reserve its own	has its own cpu, memory and ephemeral-storage requests	a ReadWriteOnce claim, mounted at the same path on both sides
Needs	nothing at deploy time	nothing at deploy time	deploy-time chart cooperation, and it is Python-only and single-replica-only
Risk to the workload	real — see below	none by default; the origin pod is never touched	it rolls the workload: that is how the fix takes
Explained in	What `attach` does	What `dev` does	What `hotfix` does

Try it#

$ uvx podbench doctor
$ uvx podbench attach pod/web-7d9f8c5b4-x2k9p -n demo

doctor checks the prerequisites, the cluster-side RBAC and the one-time ssh Include line — the only thing podbench leaves behind — and names whatever is missing; --fix adds the Include. attach then walks a capability ladder, lands the best seat the cluster will admit, and runs its probe inside the container it just created, so the report is measured rather than inferred from the spec it asked for:

rung        full - uid 0, gid 0, CapEff 00000000a80c25fb
supports
  [x] live attach (gdb -p <pid>)
  [x] read-only inspect (/proc/<pid>/root, maps, environ)
      root, maps and environ readable
  [x] debug launched processes (podbench dbg --launch ./prog)
  [ ] iterate (edit, relaunch, verify through the Service)
  [x] ssh seat (Remote-SSH: editor, shell, git, sftp)
measured
  verdict     live attach available
  blocker     none

ssh config written to ~/.podbench/config.d/demo-web-7d9f8c5b4-x2k9p.conf
then:  ssh podbench-demo-web-7d9f8c5b4-x2k9p

Connect that alias with Remote-SSH: Connect to Host…, or let podbench vscode do the whole thing — seat, pod sizing, debugpy and window — in one command.

The other two modes are a table row away. For the inner loop, podbench dev authors the dev pod and podbench run relaunches the app inside it: measured end to end, 1.18 s per edit → relaunch → verified-through-the-Service cycle. When the fix has to survive the pod, podbench hotfix moves the venv onto a claim so a restart no longer restores the image’s code, and records where the change came from.

Only prereleases are published so far, so uvx podbench resolves the newest beta today. To run an unreleased checkout instead, put --from git+https://github.com/gilesknap/podbench in front of the verb.

Read this before you attach to a live pod#

Not fine print — each of these has bitten a spike on a real cluster.

A seat shares the pod’s limits and cannot reserve its own — and it is a VS Code session, not the seat, that spends them. An ephemeral container may not declare resources at all, so the seat lives in the pod’s cgroup: exceed memory and the kernel OOM-kills something in it, exceed ephemeral storage and the kubelet evicts the whole pod. How much that matters is now measured rather than assumed. Ten live seats on a Diamond beamline (2026-08-19) cost 13–23 MiB each, against 170–3858 MiB of headroom per pod, three seats to a pod, no OOM anywhere — so attach reads this pod’s headroom, prints it on the report’s memory row, and warns only when it is genuinely thin. A vscode-server is the case that still bites: 1215 MiB live with a single extension, which does not fit in most of those pods, so podbench vscode is checked against the same number — and raises the target’s limit in place to cover the shortfall, since it is about to spend it. attach --resize MEMORY is the same lever by hand, and podbench dev gives the seat limits of its own. One beamline at one moment: that falsifies “always warn”, it does not prove no cluster is tight.
Being refused SYS_PTRACE is normal, not an error. It is outside both the baseline and the restricted Pod Security Standards, so podbench walks a ladder with two valid rungs and lands the better one the cluster admits. Four unrelated subsystems deny attach with the same EPERM, so it probes and names the blocker instead of leaving you an errno.
Ephemeral containers are permanent. They cannot be removed, restarted or edited, and a name once used is burnt for the life of the pod. So attach reconnects by default, nothing may live only in the writable layer, and --new is for when you mean it.
A dev pod never joins a Service by accident. podbench dev drops the origin’s selector labels unless you pass --take-traffic, and --cutover records the original selector for an exact restore.

The docs carry the reasoning and the measurements behind all four, and What is proven, and what is not is candid about the gaps — every memory figure is still a lower bound, and Hotfix mode has never been run against a cluster.

What	Where
Source	gilesknap/podbench
Launcher	https://pypi.org/project/podbench (prereleases only)
Image	`ghcr.io/gilesknap/podbench`
Chart	`oci://ghcr.io/gilesknap/charts/podbench`
Documentation	https://gilesknap.github.io/podbench
Releases	gilesknap/podbench

Where to start#

Never used it — Setup, then Your first session.
Here to open an editor on a live pod — VS Code Remote-SSH.
Here to debug a crash — Debug with gdb.
Here to change code in the cluster — Iterate on Python.
Here to decide whether to allow it — Security model.
Looking for a flag — Command-line reference.
Met a word you do not know — Glossary.
Wondering how finished it is — What is proven, and what is not.

How the documentation is structured#

Documentation is split into four categories, also accessible from links in the top bar.

Tutorials
- Setup
- Your first session

Tutorials for setting up and typical usage. New users start here.

How-to Guides

Practical step-by-step guides for the more experienced user.

Explanations

Explanations of how it works and why it works that way.

Reference

Technical reference material including APIs and release notes.