ADR-0005 Architecture decision record

The Sandbox pod's containers: Harness, Custodian, an optional User Container, and an optional Sandbox Effector

A Sandbox pod composes up to four containers around one shared worktree volume (/work):

• Harness container — the CR's primary container: the Sandbox Image (user-built or user-brought, ADR-0037) with jr2's runtime mounted at /opt/jr2 and the command overridden to start the Harness. The Agent's toolchain lives here because the Working tools (read/write/edit/bash) execute here — the tools the Agent can reach are this image's. It is also where a human kubectl execs to inspect a run with the agent's exact tools, worktrees, and filesystem.
• Custodian container — jr2-composed sidecar, upstream Envoy configured at provision by the Orchestrator (ADR-0013, ADR-0059): the pod's credential holder. It holds the Sandbox token and the held secrets its pod names — model API keys above all — as files, and carries the Harness's traffic out: its Menu and the ask to the Orchestrator, and its outbound HTTPS. It exists as a separate container precisely because the Working tools give the Agent code execution in the Harness container — the credentials live where the Agent cannot read them, and the Harness container's env holds only their Stand-ins.
• User Container — opt-in third seat, composed only when the workspace() names its image (a file: docker context the Machine ships or a registry ref, the same resolution as the Sandbox Image, no default — ADR-0049). The authoring surface is that string — user?: string among workspace()'s options, beside image — or the object it widens to, { image, resources?: { limits: { memory?, cpu? } } }, whose only other key is the seat's split of the Workspace's Size (ADR-0060): a budget is the pod's business, not the process's, so it puts nothing in the container. Env and ports are not forwarded, because every key jr2 forwarded into the process would be a crack in "jr2 puts nothing in it"; a routable port (the follow-up below) is the known candidate for the next key. In the pod it is the container named user (kubectl exec -c user). It runs its own entrypoint, untouched: jr2 injects nothing into its process — no env, no command, no probe — and overrides nothing; it mounts only what the checkouts need to be checkouts (below). The zero-contract seat, which is exactly why it exists. Two jobs no other seat can do:
  • Unattended services. A container's one command belongs to the Harness in the primary seat (ADR-0037), so an image's own services — an sshd for managed access, an IDE server, a metrics agent — need a container whose command jr2 deliberately does not own. A system built on jr2 that hands people access to their Sandboxes automates through this seat.
  • Credential visibility. A session's secrets — a forwarded ssh agent socket above all — live in this container's private mount namespace, where the Agent executes nothing. The human sshs in with agent forwarding, works, and the socket is gone on disconnect; at no point does it share a filesystem with code the Agent runs.
• Sandbox Effector — a jr2-composed container, added at provision only when an Effect below the workspace() is placed there — a push, an http whose body is a Worktree file, or a credentialed command (ADR-0067, ADR-0068), which jr2 up reads off the Machines and declares in the pod's manifest beside its Held one. In the pod it is the container named effector. Its image is the Sandbox Image unless the workspace() names effector: { image } — a file: context or a registry ref, resolved like the User Container's — for the tools and the credentials kept out of the Agent's image. Its command is jr2's binary on the runtime volume (/opt/jr2/bin/effector); it runs as its image's own USER (ADR-0037's 1000 fallback included), hardened like the Harness. The Agent executes nothing in it: it holds the secrets its Effects name and the credentials its pushes spend, which the Agent never reads. With the default image it runs as the Agent's uid, and that is the rule, not a gap: the containers share no mount namespace and no /proc, so the barrier is the container and the read-only mount, not the uid. Inside it, a credentialed command's child — the author's argv, often running the Agent's code — shares the Effector's uid, so the Effector confines it with Landlock to the image's system paths, its own scratch (its TMPDIR included; never /tmp) and the secret files it names, and ends every process it left before its Effect settles (ADR-0068). No Custodian, no harness.env, and no env beyond where its mounts are and its pod's name, address and node.

The Harness and User containers mount /work read-write, /repos read-only — the node's Repo cache, one bare checkout per Repo the Sandbox names (ADR-0004, ADR-0051) — and /opt/jr2 read-only, the runtime volume — so human and agent see identical files. Neither /repos nor /opt/jr2 is a second exception; each is half of the first. The worktrees are --shared clones whose alternates resolve objects from /repos/<key>, so a seat holding /work alone holds checkouts whose every borrowed object is missing — git in the User Container dies on "unable to normalize alternate object path". And their origin fetches through a program on /opt/jr2 (ADR-0053), so a seat without that volume holds checkouts whose git fetch dies. The Custodian mounts none of the three — the pod's credential holder has no business in the working tree.

What each container mounts of the credentials, stated once:

Container	Mounts
harness	jr2-ca at /etc/jr2/ca (the trust bundles). None of the Custodian's volumes.
custodian	its values (the Sandbox token, its pod's jr2-held-secrets-<pod>, each user secretKeyRef its pod names) at /etc/jr2/custodian/values; its leaves (jr2-held-tls); its pod's config (its own <sandbox>-custodian, and program.lua from jr2-held); upstream.crt.
user	the checkouts' three volumes, and nothing else.
effector	/work, /repos/<key> and /opt/jr2 read-only; at /run/jr2/secrets (0400) the secrets its Effects name — its pod's jr2-effector-<pod> literals, each user secretKeyRef by reference — and at @git/<name>/ each git.credentials entry's push Secret the Sandbox CR names (gitCredentials), which the operator projects from that entry's Secret by reference into this container alone (@ is no character of a secret's name, so no secret's file takes that directory); at /run/jr2/effector its key (its own <sandbox>-effector) and effector.json; an emptyDir at /run/jr2/scratch as $HOME; at /etc/jr2/effector-ca, read-only, the instance's caBundle ConfigMap when the instance ships one (ADR-0020).

One git wall stays the image's own: git's dubious-ownership guard fires in the User Container whenever its uid differs from the Harness's (the attach created the trees), and safe.directory is honored only from system/global config — files jr2 does not own in this seat. Env-form config (GIT_CONFIG_* on the sidecar) was considered and rejected: it would crack "jr2 puts nothing in it", and it does not even cover the seat's flagship access path — ssh login sessions scrub container env — so the image would still owe a line for sshd while carrying jr2's env for exec sessions, two mechanisms where one honest one serves. A User Container image whose sessions run git carries its own git config --global --add safe.directory '*' (or ships it in /etc/gitconfig).

The worktrees' remotes encode which hop each seat can make: git fetch origin asks the node cache, and the cache asks the remote with the cluster's credential (ADR-0053 — the hop the pod can make, from any seat, with no credential of its own), while origin's push url is the real remote in the Binding's own spelling — the attach sets it from the slot the Machine bound (ADR-0051), so a Machine that bound over ssh pushes over ssh even when the cache was cloned over https, and nothing else plumbs it. A push from the Harness or the User Container therefore succeeds exactly when a caller supplies the credential — a human's forwarded agent in the User Container — and never for the Agent, because no container it executes in holds one (the credential-visibility story above, unchanged). An UNATTENDED publish is the Machine's push() Effect (ADR-0067): it runs in the Sandbox Effector, which holds the push credential of the git.credentials entry the Repo Slot's url falls under — the provision names that entry's Secret on the Sandbox CR and the operator projects it (ADR-0051) — and pushes to that url, never to origin's push url from the Agent's config. A git.credentials entry's credentials never enter the Orchestrator: jr2 up writes the push credential — a push: { token } or push: { sshKey }, or the clone credential where the entry has no push field — into that entry's own push Secret (jr2-git-push-<hash of match>), which only the Effector is given, and the clone token into the clone Secret jr2-git-<hash of match>, which only the cache agent reads (ADR-0051, ADR-0068). The Orchestrator names both and holds neither. Opening a pull request is an http Effect the author writes; jr2 has no forge client.

Sharing /work across uids

jr2 sets runAsUser nowhere — each image's own USER decides its seat's uid — so the two writing seats may disagree, and plain POSIX modes would then make the other seat's files read-only. jr2 closes that with two mechanisms it happens to own, unconditionally (both are inert when the uids already match):

• The pod carries fsGroup — the work group, default 2000. The default is convention; the one override lives on the WorkspaceSpec (workGroup?: number, beside image and user — pod composition is the spec's business, ADR-0037), for the image whose session users already hold a gid of their own. Never a config key. Any value is safe for the jr2 seats: Kubernetes grants the fsGroup as a supplemental group to every container process, so the Harness writes /work whatever the number, and no image's /etc/group needs to know it. fsGroup puts a setgid group on /work's root, and setgid propagates down, so everything either seat creates is group-owned by the work group.
• The attach stamps a default ACL on every repo root it creates — u::rwx,g::rwx,o::r-x, set by work-acl, a static helper on the runtime volume (part of ADR-0037's published /opt/jr2 surface, static for the same reason rg is: it executes on a libc jr2 does not control). A default ACL is inherited by everything created beneath it, and POSIX ignores the process umask where one is present — so every file either seat writes under a repo tree lands group-writable (664/775), with zero lines in any image: no umask in a login shell, no jr2 knowledge in a brought User Container. Ownership without writability was the trap — fsGroup alone leaves the other seat group-read. On a filesystem without POSIX ACL support the helper warns and changes nothing, and sharing degrades to the umask fallback below.

Group membership is the one thing an image can still owe. kubectl exec sessions and a non-root sshd's logins hold the work group automatically — both inherit the container's supplemental groups, where Kubernetes put the fsGroup. A root sshd is the exception: the logins it setuids rebuild their groups from /etc/group, so that image puts its session user in the work group itself (groupadd -g 2000 work && usermod -aG work <user>) — or the spec points workGroup at a gid its users already hold.

The promise is scoped to repo trees. An ad-hoc path at /work's top level — either seat's mkdir /work/scratch — carries no default ACL, so files there fall back to the writer's umask. The Harness runs at umask 002 (itself and every tool child) and the attach script opens with the same line, so jr2's own writes are group-writable even there; what a User Container session writes outside a repo tree is its own business. A Workspace with no Repo Slot (ADR-0051) has no repo tree: its steps share /work itself, every Agent and command of it in the one Harness container as one uid, so they need no ACL; a User Container's writes there have this paragraph's limit, until a user of that seat needs more.

Constraints on the seat

The seat is zero-contract, not zero-physics. What an image brought here must live with:

• The entrypoint must block. The pod's restart policy restarts an exited container, so a CMD that finishes crash-loops. Run a long-lived process — an sshd, an IDE server, sleep infinity at minimum.
• The ports of jr2's seats are taken — one network namespace: :8080 (the Harness), the Custodian's 127.0.0.1:8081 (control), 127.0.0.1:15001 (egress) and :15021 (health), and :8082 (the Sandbox Effector, in a pod that has one). Everything else is the image's. A process that binds :8082 while the Sandbox Effector is down gets signed requests it can neither forge nor answer (ADR-0068): the Effect fails closed.
• Root is allowed. The hardened security context (non-root, all capabilities dropped, no escalation) applies to the seats jr2 owns — Harness, Custodian and Sandbox Effector, per container — and deliberately not here: the seat's identity is "what jr2 does not own", and hardening it is an opinion. A root sshd that binds :22 and setuids sessions down to its login user — the standard shape for managed access — runs unmodified; a platform that wants this seat hardened hardens its own image or its namespace's Pod Security profile. The credential-visibility boundary never depended on this seat being unprivileged, only on the Agent executing nothing in it.

Why separate containers, not one image

• Image ownership — the Sandbox Image is the user's (ADR-0037), the Custodian is a pinned upstream image, and the User Container is whatever its owner wants, contract-free because jr2 puts nothing in it. None bloats or constrains the others.
• Independent resource limits — the Workspace's Size is one ceiling for the pod, and a Machine may split it so the User Container gets a limit of its own (ADR-0060); then a runaway build there can't starve the agent (and vice-versa). Without a split the User Container shares the pod budget.
• Lifecycle independence — a Harness crash restarts only the Harness; an interactive session in the User Container survives, and vice-versa.

What the split is — and is not — a security boundary

The pod is the isolation unit (one per Workspace, the k8s north-star): its containers share a kernel, network namespace, node, and the worktree volume, and are one trust domain in every isolation sense. The one property a container edge does provide is credential visibility — the asymmetry ADR-0013 builds on: the Agent executes code in the Harness container and none in the others, so a secret that exists only in another container's mount namespace is out of its reach. That is why the Custodian holds the Orchestrator credential and the model keys, and it is what the User Container offers a human session — with its edges stated plainly:

• It holds only for what stays in the private filesystem or session. Anything on /work or listening on localhost is shared with the Agent — the pod has one network namespace. The Custodian's listeners are on localhost too; what they hand out is a key's USE toward its bound host, never the key (ADR-0059).
• A live forwarded agent socket is usable by anything that executes as that session's user in that namespace — so a session holding one should not blindly execute what it finds on /work (the worktree is agent-authored; sourcing its direnv, running its scripts, or trusting its justfile volunteers the boundary away).
• It is not port isolation. The User Container's sshd and the Agent's dev server contend for the same ports.
• It is not a seat for parallel authorship. Two writers on one checkout collide as files, whatever the container layout; parallel work belongs on a second checkout — fetch the branch out (git fetch "ext::kubectl exec -i <pod> -c harness -- git upload-pack /work/<slot>/default" needs no credential in the pod at all) and push from where the keys already live.

Considered options

• No third seat; start services by hand (kubectl exec -c harness -- sshd). Works for a person, rejected as the mechanism: a service a managed system relies on cannot depend on a human's exec, and it plants the session's credentials in the Agent's own mount namespace.
• Ephemeral containers as the attach path. Right for one-off, creds-free debugging (kubectl debug --target=harness with a plain image), rejected as the standing mechanism: they cannot run from pod birth, accumulate per attach, and are spent on exit.
• A truly minimal Harness (agent loop only, tools executed in a sibling container) — would need a cross-container execution transport jr2 does not have. Not blocking: the agent toolchain lives in the Sandbox Image; revisit only if that ever becomes a real cost (ADR-0037 records the same rejection from the image side).
• umask discipline as the sharing mechanism (fsGroup + umask 002 in every writer). Rejected as what the promise rests on: the Harness's half is one startup line jr2 controls, but the User Container's half is a line every image must remember ([ -d /work ] && umask 002 in a login shell) — a gotcha, and a silent one: forgetting it costs nothing until the Agent cannot edit a human's file, and the failure reads as the Agent's. It also cuts the other way — a session's deliberate umask 077 under a repo tree would break the sharing the pod exists for; under a default ACL, /work's repo trees are definitionally shared. The umask survives as defence in depth, not contract.
• A root init container stamping the ACL on /work's own root — complete coverage (no repo-tree scoping), owner rights with every capability dropped, kit code only, the standard volume-permissions idiom. Rejected: it puts a runAsNonRoot: false container in every pod, so no jr2 pod could satisfy the Pod Security "restricted" profile — today that is forfeited only by a User Container that chooses root, and that choice belongs to the user, not to the composition. The attach already owns the only place repo roots are born, so the narrower stamp costs one script line.
• Unifying uids across seats (document "make both images run the same USER"). Rejected: it makes the uid a cross-image contract — every pairing of Sandbox Image and User Container must agree forever, which is exactly the coupling the zero-contract seat exists to remove.

Consequences

• The generic Sandbox CRD composes all of this as plain container specs (ADR-0001 holds — the operator stays agent-agnostic); the Orchestrator's kubectlSandbox supplies the images. The pod runs two containers, plus user when the spec names one and effector when an Effect is placed there.
• Every container but the Sandbox Effector carries a limit from the Workspace's Size (ADR-0060): the Custodian a fixed kit slice, the User Container its split if stated, the Harness the rest. The Sandbox Effector carries no limit of its own: like a User Container with no split, it shares the pod's Size ceiling — the whole Size, as pod-level resources — with the containers beside it, and what share of the Size it gets is not decided. The Harness container alone mounts a memory-backed /dev/shm (sizeLimit 25% of its share); the User Container keeps the pod's default.
• jr2 does not gate readiness on the User Container and never restarts the pod for it; it lives and dies by the pod's own policy.
• Operator: runAsNonRoot moves from the pod level to the jr2-owned containers — the Harness, the Custodian and the Sandbox Effector (the Custodian states its own context: non-root, read-only root filesystem, no capabilities); the hardened-by-default rule for sidecar specs exempts the user container; the pod gains fsGroup (spec.workGroup ?? 2000). Orchestrator: the attach script stamps the default ACL on each repo root before the clone that fills it — inheritance happens at creation, never retroactively. Harness image: vendors the static work-acl into /opt/jr2/bin (ADR-0037's surface). Harness: sets umask 002 at startup. shareProcessNamespace stays off — ever — because the credential-visibility boundary depends on it.
• The Harness container's /work writes being group-writable widens nothing: every process that could abuse group access already runs in the same trust domain, and the work group exists only inside this pod.
• Routable access is a follow-up, deliberately untaken: the operator's Service serves the Harness at :8080, and a managed system wanting cluster-routed ssh to the User Container needs a declared port there. kubectl port-forward reaches any listener in the pod without one, which covers the human case day one.
• GLOSSARY.md: User Container is a glossary term.