Maintain
This page is for the box in day-to-day use. It covers what runs, where the state lives, updates, backups, disk space, logs, and what each error message means.
What runs
The stack is three containers.
| Service | Role | Published |
|---|---|---|
orchestrator | The server and the operator app | 127.0.0.1:8080 (ORCHESTRATOR_PORT) |
postgres | The database | No |
opensandbox | Starts and stops machines' containers through the Docker socket | No |
Only the orchestrator is reachable from outside Docker, and only on this host. Agents run on their organisation's machines: one sandbox container per machine, shared by every agent placed on it (Machines). Sandboxes can reach the public internet.
The agent image
Every machine runs agent-runtime:0.4 (SANDBOX_IMAGE). It has a browser and the coding toolchains. ./setup.sh builds it from runtime/ with two build arguments at their defaults:
| Argument | Default | Other value |
|---|---|---|
BROWSER | chrome: Google Chrome on amd64, Debian's Chromium on arm64 | chromium: Debian's Chromium on every platform |
HARNESSES | claude-code cursor antigravity | Any subset of those three. Codex is always installed. Empty leaves Codex as the one harness |
To build it another way, run docker build yourself:
docker build --build-arg BROWSER=chromium --build-arg HARNESSES= --tag agent-runtime:0.4 runtime/./setup.sh rebuilds agent-runtime:0.4 with the defaults on every run, the update included. After it, build your own image again, or give it another tag and name that tag in SANDBOX_IMAGE.
A coding run of an agent on a harness the image lacks fails at start with an error that names the missing harness. Harnesses explains which harnesses an image carries.
Where the state lives
STATE_ROOT in .env is an absolute path on this host. The stack keeps everything under it. Each directory is mounted into the container that uses it. The stack creates no named Docker volume.
| Tree | Holds |
|---|---|
postgres/ | The PostgreSQL data directory: every table |
opensandbox/ | The sandbox server's SQLite lifecycle store |
workspaces/ | One directory per machine, mounted at /home in the machine's container, with one home per agent on it: workspaces/<machine id>/<agent id>, /home/<agent id> inside. Scripts, the browser profile, and a coding agent's repositories live there |
notes/ | The notes vault |
attachments/ | Chat images, one file per attachment |
connectors/ | Connector packages installed from a registry |
The path must be absolute because the sandbox server mounts each machine's directory by that same path.
Everything the box knows is this tree plus .env.
Update
Read the changelog. Each version names its effect (Versions).
| Effect | Do |
|---|---|
no impact | Nothing |
rebuild | The commands below |
breaking | Back up, then follow the steps in that version's entry |
To rebuild:
git pull
./setup.sh
docker compose up --detach --wait --build./setup.sh keeps every value already in .env. It rebuilds the agent image, so a change under runtime/ reaches the next machine container. --build rebuilds the orchestrator image from the checkout.
The server applies new database migrations at boot, before it listens (Orchestrator).
Back up
PostgreSQL holds notes, records, memory, scripts, runs, and every credential. Credentials are encrypted with the key in .env. workspaces/ holds what agents wrote to disk. attachments/ holds uploaded images.
Stop the stack, then copy STATE_ROOT:
docker compose stopOr dump the database while the stack runs:
docker compose exec -T postgres pg_dump -U agent_platform -Fc agent_platform > guilds.dumpKeep .env with the backup. Without it, the credentials in the backup cannot be decrypted.
Restore
Stop the stack. Put the copy of STATE_ROOT back in place, with the .env it was made under. Then start the stack:
docker compose up --detach --waitTo load a dump instead, stop the orchestrator, restore into the running database, then start the orchestrator again:
docker compose stop orchestrator
docker compose exec -T postgres pg_restore -U agent_platform -d agent_platform --clean --if-exists < guilds.dump
docker compose start orchestratorRun setup.sh again
./setup.sh is safe to run again. It keeps every value .env already has, and it rebuilds the agent image.
It refuses to generate a new secret when STATE_ROOT/postgres already holds a database. That database was created with values only its owner has. Put those values into .env, or point STATE_ROOT at an empty directory.
./setup.sh --env-only writes .env and the state directories. It builds no image.
The stack does not start without .env. Each missing value says run ./setup.sh first, which writes it into .env.
Free disk space
A deleted machine or agent can leave its directory under workspaces/. The sweep lists the directories no live machine or agent owns, and that nothing changed in for a number of days:
docker compose exec orchestrator pnpm sweep:workspaces --age-days 7It prints each path it would remove. Add --apply to remove them.
An agent's home is also bounded by its max_workspace_mib, and a machine's directory by the machine's cap (Agent home).
Start over
To go back to an empty box, stop the stack, remove the state directories, and start the stack again:
docker compose down
rm -rf .state/{postgres,opensandbox,workspaces,notes,attachments,connectors}
docker compose up --detach --wait.state is the default STATE_ROOT. Use the path .env names when it differs.
The server applies its migrations to the empty database and creates the first operator and the Home organisation again. .env and the images stay. On a development checkout, make reset does the same (Development).
Logs and metrics
The orchestrator writes one JSON line per event to stdout. Docker keeps them as json-file logs, five files of 10 MB each. Follow them with:
docker compose logs --follow orchestratorEach line carries timestamp, level, run_id, guild_id, agent_id, operation, error_type, message, and traceback. It also carries source, request_id, session_id, path, method, and status when they apply. Warnings and errors are also written to stderr as one readable line.
Secrets never reach the log. Every value of an environment variable whose name ends in API_KEY, SECRET, TOKEN, PASSWORD, or PASSWD is replaced by [REDACTED:<name>] before a line is written.
Every API call from the operator app carries that tab's X-Request-Id. It appears in the log as request_id. The app also reports its own uncaught errors and failed calls to the server, as source: frontend. One id lines up what the operator saw with what the server did.
At boot the server logs listening once it accepts requests, and sentry enabled or sentry disabled.
The event loop counts as stalled when it is held 1 second or more. A stall logs event loop stalled for <ms> ms at warn, with operation: runtime.stall. A pooled database connection lost while idle logs idle database connection lost: <reason> at warn, with operation: db.pool. The pool drops it and opens another on the next query.
With SENTRY_DSN set, Sentry receives, under SENTRY_ENVIRONMENT (Configure):
- errors, as issues
- traces, sampled at
SENTRY_TRACES_SAMPLE_RATE - CPU profiles while a sampled trace is open, for a process sampled at
SENTRY_PROFILE_SESSION_SAMPLE_RATE - every
info,warn, anderrorlog line, as a Sentry log whose attributes are the line's fields, already redacted - an
EventLoopBlockedevent with the main thread's stack for each stall, at most 10 an hour - runtime metrics every 10 seconds, under Explore, Metrics:
| Metric | Unit | Measures |
|---|---|---|
node.event_loop.delay_max, node.event_loop.delay_p99 | ms | Event loop delay |
node.cpu | percent of one core | Server CPU. Two busy cores read 200 |
node.memory.heap_used, node.memory.rss | bytes | Server memory |
host.load_1m | Host load average | |
host.memory.used | percent | Host memory |
host.disk.free, host.disk.used | bytes, percent | The filesystem under / (mount: root) and under WORKSPACE_ROOT (mount: workspace) |
db.pool.total, db.pool.idle, db.pool.waiting | The connection pool | |
db.server.connections, db.server.max_connections | PostgreSQL client connections | |
node.event_loop.stall | count | Each 10 seconds with a stall |
db.pool.error | count | Each idle connection lost |
Sentry never receives user information, cookies, request bodies, model inputs and outputs, or query data.
GET /health answers {"status":"ok"} after a database ping. A monitor outside the box can poll it.
When it fails
| Message | Cause |
|---|---|
run ./setup.sh first, which writes it into .env | docker compose found no .env, or one without STATE_ROOT, SECRETS_MASTER_KEY, POSTGRES_PASSWORD, or OPENSANDBOX_API_KEY |
STATE_ROOT must be an absolute path | ./setup.sh found a relative path in .env. The sandbox server mounts machine directories by the absolute one |
… already holds a database, and .env has no … | ./setup.sh would generate a secret over a database created with another one. Put that database's values into .env, or point STATE_ROOT at an empty directory |
SECRETS_MASTER_KEY is the all-zero development key: set 32 random bytes | The server refuses the development key outside the development stack (Security register, SEC-14) |
SECRETS_MASTER_KEY must be 32 bytes (base64 or hex) | The key is not 32 bytes |
AUTH_MODE must be compat-cookie, trusted-header | .env names a mode core does not have |
TRUSTED_PROXY_SECRET is required when AUTH_MODE=trusted-header, and the same for PUBLIC_BASE_URL and OPERATOR_EMAIL | The proxy mode is missing one of its three settings (Remote access) |
OPERATOR_EMAIL is another user's email | A user other than the box's first already holds that address |
Create agent, Studio, the Builder, or recipe apply shows Connect a model provider first | The organisation has no provider connected. Connect one under Settings, Model keys (Model keys) |
A run ends with connect a … key on Model keys | The organisation has no key for that agent's model provider, or removed it. Connect one under Settings, Model keys (Model keys) |
Model keys answers <Provider> rejected the key or did not accept the key | The provider refused GET /models with that key. Nothing was saved (Model keys) |
The picker lists a model under Not offered | The agent's stored model is not among the organisation's chosen models. It still runs while the provider's key is connected (Model keys) |
| Model keys cannot browse a provider's catalog, or lists stale models | The box could not fetch models.dev. It serves its last catalog or the snapshot it ships with (Configure) |
A run fails at start with workspace.full | The agent's home is over its max_workspace_mib (or WORKSPACE_MAX_MIB), or its machine's directory is over the machine's cap. The message names which. Free space from the CLI tab, or raise the cap in the agent's Settings or under Settings, Machines |
machine_id is required: the organisation has more than one machine | An agent create named no machine while the organisation has several |
machine … is full | The machine already holds 32 agents, or the policy's agents-per-machine cap |
| Stored credentials fail to decrypt after a restore | The restore runs under a different SECRETS_MASTER_KEY than the one the data was written with |
The orchestrator container restarts on its own (restart: unless-stopped). A setting it refuses shows in its log on every attempt.