Appearance

Architecture ​

Most parts of an Instance run in one Kubernetes namespace. The Orchestrator runs your Machines. A Harness runs each Agent's Turn. A Custodian holds the credentials of a Harness pod. An Effector runs each http and push Effect, each command that names an Effect secret, and each dispatch to a different Orchestrator. The operator makes the pod of each Sandbox. One operator serves each cluster. The Repo cache keeps a copy of each Repo on each node that needs it. This page tells where each part runs and what it does.

Why the parts are separate ​

An Agent that works on a Repo runs code. Its Working tools give it a shell. With the shell, it can read each file and each variable in its container. It can also connect to each address that its pod can reach. Thus jr2 puts each thing that the Agent must not touch in a different place:

  • The control of a run is in the Orchestrator. An Agent cannot change its own Machine. It can only pick from its Menu, and the Orchestrator decides what each pick does.
  • Each Workspace has its own Sandbox: a pod with its own filesystem, network and processes. Agents in two Workspaces share no files and no ports. Workspaces tells more.
  • The credentials are in containers in which the Agent executes nothing: the Custodian, the Effector and the Repo cache. Secrets and trust tells how.

What runs where ​

Youyour computer, where no part of a run executes
jr2 CLI
Consolein a browser
Assistantwrites your Machines
↓ the HTTP API, through a port forward
the cluster
Instancein the namespace of the Instance, one of each
Orchestratorruns your Machines, and keeps each run
Instance Harnessruns the Turns of the Menu-only Agents
Harness
Custodian
Instance Effectorruns the Effects that read no Worktree file
Repo cacheone pod on each Sandbox node
↓ a Turn, an Effect↑ a pick, through the Custodian↓ the Repo cache gives each Repo
Sandboxone pod for each Workspace, in the same namespace
Harnessruns the Turns of the Agents that work on a Repo
Custodianholds the Held secrets of the pod
Sandbox Effectorruns the Effects that work on the Worktree
User Containerwhen the Workspace names one
↑ the operator makes the pod of each Sandbox
jr2-systemone for each cluster
operatormakes each Sandbox pod, and removes it when its Lease lapses
↓ the Custodians call the model provider↓ the Repo cache fetches from the git host
Outsidewhat the work uses
model provider
git host
What runs where, from your computer to the world outside the cluster

The Instance Harness, the Instance Effector, the Repo cache and the Sandbox Effector exist only when a Machine of the Instance needs them. The sections below tell when.

A Sandbox pod can also have a User Container, when its workspace() names one. It runs its own image, and it mounts the Worktree read-write. Workspaces tells more.

On your computer ​

The jr2 CLI runs on your computer, in the Instance folder. The Assistant works there too, and writes Machines and config in that folder. Nothing of a run executes on your computer.

jr2 up reads the folder and converges the cluster of the current kubectl context. Converge tells what it makes. By default, each other command that talks to the Orchestrator, such as jr2 run, reads the Instance token from the cluster. Then it opens a kubectl port-forward to the Orchestrator's Service, for the life of the command. The global options tell how to give a URL and a token instead. The CLI keeps no local state about the cluster.

The Console is a page that the Orchestrator serves. You open it in a browser, through a port forward that you start. Open the Console gives the commands.

The Orchestrator ​

The Orchestrator is one pod for each Instance (replicas: 1), with the Service jr2-orchestrator on port 4000. It hosts all the Workflows of the Instance and many runs of each Workflow. It keeps the snapshot of each run on its own volume, so a run continues after a restart. Durable runs tells how.

The pod runs the jr2-orchestrator Kit image, which holds no code of yours. The Orchestrator loads your Workflows and your config from the Assembly that jr2 up publishes to it, and it keeps each Assembly on its volume. Converge tells how.

The Orchestrator does these things:

  • It runs each Machine as an xstate actor, and holds the state of each run.
  • It serves the HTTP API that the CLI, the Console, a webhook and the Custodians call. See the HTTP API.
  • It sends each Turn to a Harness, and judges each pick that comes back.
  • It writes a Sandbox resource when a run enters a Workspace. Then it watches the Sandbox resource until the Workspace ends.
  • It sends each Effect to the place where it runs. It runs a dispatch with no target itself: the Effect starts a run of a Workflow of the same Instance.

The Orchestrator does no work of an Agent. It runs no model and no Working tools. That work runs in the pods that the cluster places.

To isolate two sets of Workflows, use two Instances. Each Instance has its own namespace and its own Orchestrator.

The Harness ​

The Harness is jr2's own server for the Turns of Agents. It speaks to the model provider. It gives the model the Agent's Menu and its Working tools. It executes each Working tool that the model calls, and sends each pick through the Custodian. It holds no Agents of its own. Each Turn brings the Agent's definition with it. Agents and Turns tells what happens in a Turn.

The Harness has two placements:

  • In each Sandbox. The Harness is the primary container of the Sandbox pod, named harness. It runs in the Sandbox Image, and jr2 mounts the Harness into the pod at /opt/jr2. Thus the Working tools of the Agent reach the tools of that image. An Agent that works on a Repo has its Turns here.
  • As the Instance Harness. The Instance Harness is one StatefulSet for each Instance, jr2-instance-harness. Each Turn of a Menu-only Agent (workspace: "none") runs here. Its pod has a Harness and a Custodian, and no Worktree. It keeps each conversation on its volume while a run can continue it. jr2 up converges it when a Machine of the Instance has a Menu-only Agent.

A Harness answers only the Orchestrator. The Orchestrator sends a bearer token on each call, and the Harness accepts no call without it, except its health check. jr2 up also converges NetworkPolicies that let only the Orchestrator's pods connect to a Harness pod.

The Custodian ​

Each Harness pod has a Custodian: each Sandbox, and the Instance Harness. The Custodian is a container beside the Harness. The Agent executes no code in it. It does two things:

  • It carries four kinds of call from the pod to the Orchestrator: the Menu reads, the picks, the polls for the answer to a pick, and the fetch asks. It sends each one with the pod's Sandbox token. These four routes are all that it forwards. It refuses all other routes.
  • It holds the Held secrets of its pod, such as a model API key. It puts the value into the requests that go to the hosts of each Held secret.

The Custodian runs Envoy, from a Pinned image. The Orchestrator writes its config for a Sandbox, and jr2 up writes it for the Instance Harness. Secrets and trust tells how the Custodian holds a credential.

The Effector ​

The Effector runs each http and push Effect, each command that names an Effect secret, and each dispatch that names a target. Thus a credential that an Effect uses stays out of the Harness container. The Effector is jr2's own program, and it runs in a container in which the Agent executes nothing. Effects tells which Effect runs where. It has two placements:

  • The Sandbox Effector is a container in a Sandbox pod, named effector. It mounts the Worktree read-only. It runs a push, an http Effect whose body is a Worktree file, and a command that names an Effect secret. Its image is the Sandbox Image, unless the workspace() names a different image.
  • The Instance Effector is one pod for each Instance, jr2-instance-effector. It runs each http Effect that reads no Worktree file, and each dispatch that starts a run on a different Orchestrator. It holds the Instance token of that Orchestrator as an Effect secret. It runs in the jr2-harness Kit image.

Each placement exists only when a Machine of the Instance puts an Effect there. A command that names no Effect secret does not use an Effector. It runs in the Harness container of its Sandbox, with the same tools and access as the Agent.

The Orchestrator signs each request that it sends to an Effector, and the Effector signs each answer. The Orchestrator only asks, and the Effector holds the credentials.

The operator ​

The operator is a Kubernetes operator. One copy serves the whole cluster: it runs in the namespace jr2-system, and each Instance on the cluster shares it. By default, jr2 up converges it. With operator.manage: false, you run it yourself.

The operator reconciles two custom resources:

  • A Sandbox resource becomes one pod and a Service. The Orchestrator writes the Sandbox resource, and the operator makes the pod and reports its state. The operator never makes a second pod for a Sandbox. When the pod ends, the Sandbox is lost.
  • A Repo names one git repository that the Repo cache keeps.

The operator knows nothing about Agents, Turns or Worktrees. The Orchestrator does that work after the Sandbox is ready. The operator also removes a Sandbox resource when its Lease lapses, for example after the Orchestrator stopped for longer than the Lease allows. Workspaces tells how the Lease works.

The Repo cache ​

The Repo cache keeps one bare clone of each Repo on each node that needs it. It is a DaemonSet, jr2-repo-cache, in the namespace of the Instance. It has one pod on each Sandbox node. jr2 up converges it when a Machine of the Instance uses a Sandbox.

The operator mounts the cache of each Repo into a Sandbox, read-only, at /repos. The clone in the Sandbox gets its objects from that cache. A git fetch origin in the Sandbox asks the Repo cache for a fetch. Then the Repo cache fetches from the host of the Repo, with the clone credential of the cluster. Thus the Harness container needs no git credential to fetch.

Rules ​

  • An Instance has exactly one Orchestrator, and the Orchestrator always runs in the cluster.
  • A namespace holds one Instance. jr2 up refuses a namespace that a different Instance owns.
  • By default, the CLI reaches the cluster of the current kubectl context. Kubernetes RBAC decides who can read the Instance token.
  • A Harness accepts a call only from the Orchestrator, with the bearer of its own pod. Only its health check needs no bearer.
  • The Agent executes code only in the Harness container. It executes nothing in the Custodian or in the Effector.
  • The Orchestrator watches the Sandbox resources and never reads a pod. The operator puts the facts of the pod on the status of the Sandbox resource.
  • A Sandbox has one pod for its life. A lost pod is never made again.

The docs of Kit version 0.5.1. Each page is also at its .md URL.