agents-exe

Iterative Development Workflow

2026-10-04T15:53:54Z

Generated from documentation/ITERATIVE-WORKFLOW.md, the repository is the canonical source and may be ahead of this page.

Iterative Development Workflow

How to handle multi-step, recursive, or iterative invocations on this codebase.

Invocation Patterns

Pattern 1: Fresh Start (No Previous State)

When: First time working on this codebase, or no knowledge base.

Steps:

Initialize knowledge base tables
Scan all source files → code_index
Read existing docs → docs_index
Identify documentation gaps
Create initial documentation plan

-- Initialize if empty
CREATE TABLE IF NOT EXISTS project_overview (...);
CREATE TABLE IF NOT EXISTS code_index (...);
CREATE TABLE IF NOT EXISTS docs_index (...);
CREATE TABLE IF NOT EXISTS commits_seen (...);

-- Populate code_index
-- (scan all .hs files with bash_list-files and bash_grep-files)

Pattern 2: Continue Previous Work

When: Knowledge base exists, continuing documentation task.

Steps:

Query what was completed
Check for new commits since last work
Review current docs_index state
Pick up where left off

-- What was documented?
SELECT module_name, last_updated 
FROM code_index 
WHERE documented_in IS NOT NULL
ORDER BY last_updated DESC;

-- What's missing?
SELECT filepath, module_name 
FROM code_index 
WHERE documented_in IS NULL;

-- Any new commits?
SELECT * FROM commits_seen 
ORDER BY commit_date DESC LIMIT 10;

Pattern 3: Handle New Code Changes

When: Code has changed since last documentation.

Steps:

Check commits_seen vs git log
Read new commit details
Identify affected modules
Update documentation
Record commits as processed

# Get recent commits
bash_get_git_log

# Read specific commit
bash_get_git_commit abc123

# Find affected files
bash_grep-files "changed_pattern"

Pattern 4: Deep Dive on Subsystem

When: Need to document one specific subsystem thoroughly.

Steps:

Identify all modules in subsystem
Read each module source
Extract types and functions
Create/update doc file
Link all modules in code_index

-- Find subsystem modules
SELECT * FROM code_index 
WHERE module_name LIKE 'System.Agents.Subsystem.%';

Chunking Strategy

Large documentation tasks should be chunked:

By Subsystem

Chunk 1: Core (Base, Runtime, AgentTree)
Chunk 2: Tools (Bash, MCP, OpenAPI, IO)
Chunk 3: Sessions (Types, Loop, Store)
Chunk 4: Interfaces (CLI, TUI, MCP Server)
Chunk 5: Utilities (FileLoader, ExportImport)

By Document

Iteration 1: README.md (overview)
Iteration 2: architecture.md (core)
Iteration 3: tools.md (tools)
Iteration 4: cli-commands.md (CLI)
Iteration 5: Remaining docs

State Tracking Between Iterations

Always update the knowledge base at the end of each iteration:

-- Record what's been documented
UPDATE code_index 
SET documented_in = 'documentation/architecture.md',
    last_updated = datetime('now')
WHERE module_name IN ('System.Agents.Base', 'System.Agents.Runtime');

-- If new doc created
INSERT INTO docs_index (doc_path, title, description, related_modules)
VALUES ('documentation/new-feature.md', 'New Feature', 'Description', 'Module1, Module2');

Recursive Documentation

When Documenting Recursion Features

The codebase has recursion (agents calling agents). When documenting:

Acknowledge the recursion - Note that this is a recursive feature
Show depth tracking - Document ctxMaxDepth, CallStackEntry
Explain limits - Document how infinite loops are prevented
Cross-reference - Link to related recursive patterns

## Agent Recursion

Agents can call other agents, creating a call stack:

```haskell
data ToolExecutionContext = ToolExecutionContext
    { ctxCallStack :: [CallStackEntry]  -- Tracks recursion
    , ctxMaxDepth :: Maybe Int          -- Prevents infinite loops
    }

Decision Tree

Invoked on codebase
       │
       ▼
Knowledge base exists?
       │
   ┌───┴───┐
   │       │
  Yes     No
   │       │
   ▼       ▼
Check     Initialize
state     tables
   │       │
   └───┬───┘
       ▼
Task specified?
       │
   ┌───┴───┐
   │       │
  Yes     No
   │       │
   ▼       ▼
Do task   Identify
          gaps
       │
       ▼
Multiple subtasks?
       │
   ┌───┴───┐
   │       │
  Yes     No
   │       │
   ▼       ▼
Chunk    Complete
work     task
   │
   ▼
Update KB
   │
   ▼
Done (or next iteration)

Handoff Notes

When work will continue in next invocation, include:

What’s done: List completed documentation
What’s next: Specific next steps
Blockers: Any issues to resolve
Context: Relevant state from knowledge base

Example handoff:

## Handoff Summary

Completed:
- documentation/architecture.md (core types, runtime, agent tree)
- documentation/tools.md (bash tools, partial MCP)

Next:
- Complete MCP section in tools.md
- Document OpenAPI tools
- Create mcp.md for protocol details

Blockers: None

Context:
- 12 of 79 modules documented
- Runtime and AgentTree fully covered
- Tools subsystem 60% complete

Common Iteration Sequences

Sequence: Full Documentation Pass

Iteration 1: README + architecture
Iteration 2: Tools system
Iteration 3: Sessions + TUI
Iteration 4: CLI + Export/Import
Iteration 5: File loading + cleanup
Iteration 6: Review + cross-references

Sequence: New Feature Documentation

Iteration 1: Understand feature (read code)
Iteration 2: Document core types
Iteration 3: Document API/functions
Iteration 4: Add examples
Iteration 5: Update related docs
Iteration 6: Add to README/index

Sequence: Maintenance Update

Iteration 1: Check new commits
Iteration 2: Identify affected docs
Iteration 3: Update affected sections
Iteration 4: Verify consistency
Iteration 5: Record commits processed

Tool Usage by Phase

Discovery Phase

bash_list-files - Find all source files
bash_grep-files - Search for patterns
bash_get_git_log - See recent changes

Analysis Phase

bash_get_git_commit - Read specific changes
Read source files with grep results
Query code_index for relationships

Documentation Phase

bash_write-file - Create/update docs
Update docs_index with new entries
Update code_index with document links

Verification Phase

Query for consistency
Check cross-references
Verify all modules covered

Anti-Patterns to Avoid

Don’t document without updating knowledge base
Don’t create docs without adding to docs_index
Don’t leave code_index modules unlinked
Don’t assume previous state - always query
Don’t document too much in one iteration (fatigue/errors)
Don’t forget the summary line (first line of response)

Success Metrics

Good iterative documentation:

Each iteration has clear scope
Knowledge base stays consistent
No lost work between invocations
Progress is measurable (X of Y modules done)
Final docs are comprehensive and cross-referenced

Quick Reference Card

STARTING:
  SELECT * FROM project_overview;
  SELECT * FROM docs_index;
  SELECT * FROM code_index WHERE documented_in IS NULL;

DURING:
  Read source → Extract types → Write doc → Update KB

ENDING:
  UPDATE code_index SET documented_in = 'documentation/X.md' ...
  INSERT INTO docs_index ... (if new doc)
  SELECT consistency checks

HANDOFF:
  List done, next steps, blockers, context
  Include summary line as first line

Use this workflow for multi-step documentation tasks.

Spec: bash tools v2, long-running tool processes

2026-10-04T15:53:54Z

Generated from todos/bash-tools-v2.md, the repository is the canonical source and may be ahead of this page.

Spec: bash tools v2, long-running tool processes

Status: proposal, 2026-09-25, from issue #539 and a robot-chat conversation the same day. Nothing implemented. The v1 protocol (documentation/binary-tool.md) stays valid unchanged: a tool that does not say otherwise is a v1 tool.

Goal

A bash tool today is one process per call: describe once, then run with the arguments, stdin written once and closed, stdout collected until exit, and that stdout is the tool result. Progress exists (tool.progressed events carry each line as it arrives, see runProcessReportingOutput), but the model only ever sees the final stdout, and a tool cannot be talked to while it runs.

v2 lets a tool declare that it is a process rather than a function:

it may keep running after answering, streaming information out in chunks the runtime understands (a declared event shape);
its stdin may stay open, the runtime feeding it JSON events shaped as the tool declares, so the model (or another tool, or mail) can keep talking to it;
from the model’s perspective such a tool is stateful: starting it returns a handle (start it twice and there are two handles, each addressable on its own); the tool family gets one typed send tool taking a handle and an event, and one shared system tool (processes: list, status, tail, stop) serves every family, so process management never multiplies the session’s tools;
the tool declares how it wants to be run, in the spirit of a systemd unit or a daemontools service directory: the user to run as, an envdir, a run directory, timeouts, what to do when it exits.

The runtime change is large (D2 below says how it lands in slices); the protocol change is small, and this document is mostly the protocol.

1. The `describe` document, v2

A v2 tool adds "protocol": 2 to its describe output. Everything v1 defines keeps its meaning (slug, description, args, empty-result). Three optional objects and one flag (check, §1.5) are new.

{
  "protocol": 2,
  "slug": "log_watcher",
  "check": true,
  "description": "Follows a log file and reports matching lines",
  "args": [ { "name": "path", "type": "string", "backing_type": "string",
              "arity": "single", "mode": "dashdashspace",
              "description": "the file to follow" } ],
  "process": {
    "mode": "stream",
    "output": { "framing": "jsonl", "event": { "type": "object",
                "properties": { "line": {"type": "string"}, "level": {"type": "string"} },
                "required": ["line"] } },
    "input":  { "framing": "jsonl", "event": { "type": "object",
                "properties": { "op": {"enum": ["pattern", "pause", "resume"]},
                                "value": {"type": "string"} },
                "required": ["op"] } },
    "ready": { "kind": "line", "match": "^ready$" }
  },
  "runtime": {
    "run_as": { "user": "logs", "group": "logs" },
    "sandbox": { "fs": { "read": ["/var/log"], "write": ["state"] }, "net": "none" },
    "envdir": "env",
    "environment": { "LC_ALL": "C" },
    "rundir": "state",
    "timeout": { "start": 10, "stop": 5, "idle": null },
    "on_exit": "report",
    "cancel": "SIGTERM"
  }
}

1.1 `process`: what kind of process this is

Field	Values	Meaning
`mode`	`oneshot` (default), `stream`, `service`	`oneshot` is v1: run to completion, stdout is the result. `stream`: the result is the sequence of output events until exit; the call completes when the process exits. `service`: the process is expected to outlive the call; starting it returns a handle and the model works with it through the process tool (§3).
`output.framing`	`jsonl` (default for v2), `lines`, `chunks`	How stdout is cut into events: one JSON object per line, one text line per event, or raw chunks as they arrive (binary-safe, for tools that produce no lines).
`output.event`	a JSON Schema	The shape of one output event under `jsonl`. Validated by the runtime; an event that does not match is delivered as an `invalid` event carrying the raw line, never dropped silently. Optional under `lines`/`chunks`.
`input.framing`	`jsonl` (default), `lines`	How the runtime writes events to stdin. Absent `input`: stdin gets the v1 `mode: stdin` arguments and is closed, as today.
`input.event`	a JSON Schema	What the runtime accepts to send to this process; also the schema of the `event` argument of the family's typed `send` tool (§2.4).
`digest`	`{"per": "round"}`, `{"per": "events", "n": N}`, `{"per": "seconds", "n": N}` (default `round`)	For a `service`: how chatty output is folded into mail (§2.3). Declared by the tool, per tool, because its author knows what an event is worth.
`ready`	`{"kind": "line", "match": REGEX}`, `{"kind": "exit"}`, `{"kind": "immediate"}`	When the process counts as started: a stdout line matching (its first events are still delivered), its exit (a `oneshot`), or right away. Bounded by `runtime.timeout.start`.

stderr is never part of the result. It is captured into the process’s bounded output ring (§2.3) and traced, as today.

1.2 `runtime`: how to run it

Every field is optional; an absent field keeps v1 behaviour (run as the agents-exe process, its environment plus the session context variables, its working directory, no timeouts beyond the call’s, cancellation kills the process group).

Field	Meaning
`run_as`	`{"user", "group"}`: run the process as this user and group. The runtime uses one mechanism it was configured with (`setpriv`, `sudo -u`, `runuser`, or a refusal when none is configured); the tool never names the mechanism. Same shape and reasoning as salmon's `RunAs`/`Mechanism` (`multi-user-privilege-separation.md` there).
`sandbox`	`{"fs": {"read": [...], "write": [...]}, "net": "host"\\|"none", "tmp": "shared"\\|"private"}`: what the process may touch (§1.4). Enforced by one operator-configured mechanism, `bwrap` or `landlock`; never images or containers.
`envdir`	A subdirectory name under the operator's explicit `--envdir-root`, read the daemontools way: one file per variable, file name is the name, file content is the value. Read at start; absent files are absent variables. Lets an operator provision secrets without touching the tool or the checkout. There is no implicit default location: a tool that declares `envdir` while no `--envdir-root` was given fails to load with a `LoadingError` naming the field.
`environment`	Literal variables, added last (they win over `envdir`).
`rundir`	A directory, relative to the tool's directory, created if missing and made the process's working directory; where a `service` keeps its state.
`timeout`	`start`: seconds to reach `ready` or fail the start; `stop`: seconds between the cancel signal and `SIGKILL`; `idle`: seconds without any output or input event after which a `service` is stopped (`null`: never).
`on_exit`	For a `service`: `report` (default: the model is told, the handle becomes final), `restart` (bring it back, bounded by a small backoff; every restart is an event), `fail` (the call that started it fails if it has not completed yet).
`cancel`	The signal sent first on cancel/stop: `SIGTERM` (default), `SIGINT`, or a JSON event `{"send": {...}}` written to stdin before the signal, for tools that prefer a polite request.

1.3 `commands` and `help`: several actions in one binary (from #539)

Out of scope of the process story but part of the same describe bump: "commands": [ {"name", "description", "args"} ] lets one binary expose several tools (slug.name each), and help (./tool help [command]) gives the runtime a longer text to disclose on demand rather than in every system prompt. A setup step (./tool setup) that provisions the tool’s own dependencies is left for a later spec: it is an operator concern, not a runtime one.

1.4 `sandbox`: what the process may touch, without images

A small declaration, no more: fs.read and fs.write are lists of paths (absolute, or relative to the tool’s directory), net is host (default) or none, tmp is shared (default) or private. Everything not listed is not granted; the tool’s own directory is always readable so it can run. Like run_as, the tool names what it needs and the operator names how it is enforced (--sandbox bwrap, --sandbox landlock or --sandbox auto); a tool that declares sandbox with no mechanism configured fails to load with a LoadingError, it never runs unconfined by silence. There is no silent default: the two mechanisms give different guarantees (bubblewrap hides paths and really cuts the network, Landlock denies access and covers TCP only), so picking one for the operator could quietly weaken what a tool asked for.

--sandbox auto is the explicit opt-in to “per tool, the strongest mechanism that can honour every declared field”: bubblewrap first, then Landlock, and a tool declaring net: none or tmp: private skips Landlock rather than being weakened, failing to load when no mechanism qualifies. It is the ordered-backends idea of §1.5 applied to mechanisms, and agents-exe check --probe prints what auto resolved to for each tool. sandbox composes with run_as (the sandbox is set up for the process that runs as that user).

bubblewrap builds a mount namespace: only the declared paths (plus the minimum to exec the tool) exist, net: none is a network namespace, tmp: private a fresh tmpfs, and the process dies with agents-exe. It needs unprivileged user namespaces, which many distributions restrict: a plain bwrap on the Ubuntu machine this was written on failed with setting up uid map: Permission denied (kernel.apparmor_restrict_unprivileged_userns=1; not conclusive, that shell may itself have been inside another sandbox).
Landlock is a kernel security module needing no privilege and no namespaces: the runtime applies a ruleset in the child before exec. Paths outside the ruleset are denied (not hidden), and on kernels with network rules it can deny TCP bind and connect, which is not the same as no network (to verify in the slice, then state precisely in check).

agents-exe check probes the configured mechanism for real, the way it would run a tool, and reports what it can and cannot enforce (for example “net: none best effort under landlock”). Where a mechanism cannot honour a declared field the tool fails to load naming the field; it is never silently weakened.

Non-goal: image or container isolation. agents-exe has no notion of an image, and this spec adds none. A tool that wants podman or docker calls it from its own script, like any other command, and agents-exe sees an ordinary process. (Reviewed against juhp/encapsule, an interactive podman dev-shell: useful vocabulary for named capabilities, the wrong shape for a non-interactive tool call.)

These are declarations, not privileges: an agent’s configuration (and the operator’s --set/--pin parameters, tool-partial-application.md) may refuse or override them. run_as in particular is only honoured when the operator configured a mechanism; otherwise the tool loads with a LoadingError naming the field, the same way a with binding that names an undeclared parameter does today.

1.5 Health: a real probe, and ordered backends per capability

The pattern comes from Agent-Reach (a “capability layer” that selects and health-checks the upstream tools an agent uses, MIT): the same capability can be served by several tools, and whether one works here, now is found by running a cheap command, never by looking the binary up on PATH (a stale shim passes which and cannot execute).

check, a third verb. A v2 tool may say "check": true in describe and then answers ./tool check: a cheap, read-only self-test (no writes, no login, no remote mutation, no long-lived process; a service is not started) that prints one JSON document and exits 0 when the tool is usable:

{ "ok": false, "detail": "jina reader answered 429", "fix": "set JINA_API_KEY in the envdir" }

detail and fix are for the operator, fix is a prescription. The runtime runs check exactly as it would run the tool (same runtime: run_as, sandbox, envdir, timeouts), so a probe that passes proves the sandbox and the user work too. A tool that cannot be checked without side effects omits check; it is then unverified (it loaded and describe answered, nothing more) and is never reported ok.

A capability is an ordered list of backends. The model sees one tool, web_read; the agent configuration says which tool binaries can serve it, preferred first:

{ "capability": "web_read", "backends": ["tools/web/read-jina", "tools/web/read-curl"] }

All backends of a capability must declare the same args (checked at load, a LoadingError otherwise) so the model’s schema does not depend on which one is active. Switching backends is reordering the list, or an operator override --backend web_read=read-curl that moves the named backend first (an unknown name is ignored, so a stale override never hides a working backend). The active backend is the first one whose probe is ok, else the first unverified one, else none (the capability fails to load when required, is omitted with a warning otherwise). Every tool result carries the backend that served it, so the model can say what it used.

A failed call does not silently retry the next backend. The model sees the failure and which backend produced it; automatic fallback is left out of the first slice (open question).

agents-exe check --probe [--json] runs every probe and reports, per capability, the active backend and every candidate:

{ "capability": "web_read", "active": "read-jina",
  "backends": [ { "name": "read-jina", "status": "ok", "ms": 310 },
                { "name": "read-curl", "status": "unverified" } ] }

status is ok, failed (with detail and fix), unverified or skipped (a sandbox mechanism that cannot enforce a declared field, §1.4). Plain check keeps its meaning (configuration loads: exit 0 / 1); with --probe it exits 2 when the configuration loads but a required capability has no active backend. Do not confuse it with ready (§1.1): ready says a process that was started is up, check says a tool could work before anything is started.

2. The runtime side

2.1 A `oneshot` v2 tool

Identical to v1 except that output.framing/output.event apply: the result delivered to the model is the events, rendered as a JSON array (or the joined lines), and each event is also a tool.progressed payload as it arrives. Nothing else changes; this is the cheap slice that makes the new describe real.

2.2 A `stream` tool

The call runs as an asynchronous call (RunAsync, async-tool-calls.md): the model gets the running placeholder, events arrive as progress, the call completes when the process exits, and the result is the full event sequence (bounded: the last N events, N from the agent’s config, the count of elided ones stated). wait, get-tool-call-status and cancel-tool-call work unchanged; get-tool-call-status shows the latest events, which is the “tail” the model needs mid-run.

2.3 A `service` tool: the process handle

Starting a service tool is a call that completes at ready: its result is a process handle {"process": "<id>", "tool": "log_watcher", "status": "running", "since": ...}, an entity in the OS world like a tool call is today (OS.Conversation.ToolCalls), owned by the session. Each start is a new process with its own handle: an agent that starts log_watcher on two files holds two handles and talks to either by its handle; nothing is deduplicated or shared between starts. A handle has:

the declared input.event schema (what send accepts);
a bounded ring of its output events and of its stderr lines (same shape as a node’s output ring in salmon’s serve: small, per process, what an operator wants when it fails);
its lifecycle: starting, running, stopping, exited (code), failed (why), restarting (n);
the runtime it was started with, resolved (which user, which envdir).

A process belongs to the session that started it. It survives across the session’s runs (it is exactly the “background call” case, evictIdle and hasRunningCalls already keep such a session resident) but not a restart of agents-exe: recoverOnStartup marks it failed: lost on restart, as it does for orphaned async calls, and the model learns it from the process tool. Durable services are a node of their own in whatever supervises agents-exe, not this spec (D3).

Output events of a running service are delivered to the model as mail (session-mailbox.md): each event, or a digest of them as the tool’s digest declares (default: per round) when they are chatty, is an envelope from process:<id>, so the model’s turn is woken the same way any other asynchronous news wakes it, with the same wakeOn controls. This is the notify progress level that spec deferred (D9 there), given a producer.

2.4 Process tools: one typed `send` per family, one shared `processes`

Two layers, split by what varies:

Per family, one typed tool. For each service tool T in the toolbox the runtime generates, at load, one companion tool T.send (slug.name, the same naming as commands, §1.3). Its arguments are process (the handle) and event, whose schema is the declared input.event verbatim, so the model gets a typed function per family instead of an untyped argument, and a family with two running processes is addressed by handle: log_watcher.send {process: p1, event: ...} and {process: p2, ...}. The answer is the events the process produced in reply within a short window, or accepted. The companion exists from load, not from start, so the tool list (and the tool cache) does not change while processes come and go.

Shared across families, one tool. Everything that does not depend on a family’s schema is one system capability with sub-commands (the shape of get-tool-call-status and friends), so a session holding ten service families still has one management tool:

Sub-command	Arguments	Answer
`list`		every process of the session (and, for a root agent, of its sub-sessions), with tool, status and age
`status`	`process`	the handle with its counters, last events, last stderr lines, and the accepted input schema
`tail`	`process`, `n`	the last `n` output events, oldest first
`stop`	`process`, `force?`	the declared cancel, then `SIGKILL` after `timeout.stop`; the exit becomes the answer

The model discovers which processes exist through list and status; a tool’s description should say it is a service and name the events it accepts, in a sentence.

3. What the model sees, end to end

The tool list holds log_watcher (start it), log_watcher.send (talk to one of its processes) and processes (list, status, tail, stop, for anything started). Starting is an ordinary tool call with the declared args, and may be done twice: each start returns its own handle.
The result is the handle. The model continues; when the watcher reports a line, mail arrives: from process:p1 {"line": "...", "level": "error"}.
The model calls log_watcher.send {process: p1, event: {op: "pattern", value: "ERROR"}}; the answer is accepted or the watcher’s reply.
processes stop p1 when done; or the session ends and every process of it is stopped with its declared cancel (a session delete stops them the way it cancels runs).

Decisions (proposed)

D1. The protocol number, not new commands. "protocol": 2 in describe is the whole opt-in; run keeps its name and argument passing, so a v2 tool can be tried as a v1 tool by ignoring the new fields.

D2. Independent features, each shippable (the feature 9914ff73 is split as needed). (1) describe v2 parsing, oneshot with framing and validation; (2) runtime for oneshot tools (envdir with --envdir-root, environment, rundir, timeout.stop, cancel) and run_as and sandbox behind operator-configured mechanisms (bwrap, landlock), which ship ahead of the process modes because a low-privilege oneshot tool is valuable alone; (3) stream on the existing async engine; (4) service: the handle entity, mail delivery and digest, the typed T.send companions, the shared processes capability, and the runtime fields only a service uses (timeout.idle, on_exit). #539’s commands/help are a fifth, independent slice, and (§1.5) the check verb with check --probe --json is a sixth, small one that lands with (1); capabilities with ordered backends and the --backend override are a seventh, depending on it.

D3. Not durable across an agents-exe restart. A service is lost with the process that started it and reported as such; supervising long-lived services belongs to systemd/salmon, which can restart agents-exe itself. Revisit only with a concrete need (os-as-standalone-server.md G11 says the same of watches and run handles).

D4. Events, not bytes, at the model boundary. The model never sees a raw stream; it sees validated events (or invalid ones with the raw text). chunks framing exists for tools, not for the model: the runtime delivers chunks to progress consumers and to the ring only.

D5. Mail, not polling, for a service’s output. The existing mailbox is how asynchronous news reaches a session; a second channel would duplicate its wake rules. Chatty services get digests, bounded by the ring, and the digest policy is the tool’s own declaration.

D6. A typed send per family, a shared processes for the rest. One handle per start, several handles per family; the family’s input schema lives in one generated T.send tool present from load (an agent configuration may omit it, saving its tokens), while list, status, tail and stop are shared so the session’s tool list does not grow with the number of families’ management surfaces.

D7. Explicit envdir root. Secrets are provisioned where the operator says (--envdir-root), never found by convention next to the tool.

D8. Process isolation, not image isolation. sandbox is enforced by bubblewrap or Landlock, chosen by the operator (explicitly, or with the opt-in --sandbox auto, never by a silent default); agents-exe never builds, pulls or names an image. It composes with run_as, refuses rather than weakens, and is probed by check.

D9. Probe, don’t look up. Whether a tool works is found by running it (check, under its real runtime), read-only, reported as ok, failed, unverified or skipped; a tool that cannot be probed safely is unverified, never ok. Backends are an ordered list per capability, the first ok one is active, and the tool result names the backend used. Pattern from Agent-Reach (§1.5).

Resolved in review (2026-09-25, PR #574 comments)

send: one tool per process family with a handle and a typed event; a second start of the same tool gives a second handle, addressable on its own; process management (list, status, tail, stop) is shared so it does not pollute the session’s tools (§2.4, D6).
run_as for oneshot: yes, split into its own feature ahead of the process modes (D2).
Digest policy: per tool, declared in process.digest (§1.1, §2.3).
envdir: explicit, an operator --envdir-root with a per-tool subdirectory, no implicit location (§1.2, D7).
The generated T.send companion is optional per agent: an agent configuration may omit it for a service the model may start but never talk to, which saves the tool’s tokens (owner’s answer, 2026-09-25; D6).
Isolation backend: bubblewrap and Landlock, no images; whoever wants podman calls it from the tool’s own script (§1.4, D8).
Sandbox mechanism selection: no silent default; the operator picks, or opts in to --sandbox auto (strongest mechanism that honours every declared field), shown by check --probe (owner, 2026-09-25; §1.4, D8).
Health: adopt Agent-Reach’s doctor pattern (ordered backends, a real probe, --json), but not Agent-Reach itself, which is deferred as heavy (owner, 2026-09-25; §1.5, D9).

Open questions

Where is a capability’s backend list declared: in the agent configuration (as sketched), or in a manifest next to the tool directories so several agents share it?
When are probes run and how long are they cached: once at load, lazily on the first call, or with a TTL? A probe that costs a network call should not run on every session start.
Automatic fallback to the next backend on a failed call: worth it for transient failures if the tool declares which exit codes mean “try another” (fallback_on), or is the model seeing the failure enough?

File Loader

2026-10-04T15:53:54Z

Generated from documentation/file-loader.md, the repository is the canonical source and may be ahead of this page.

File Loader

The file loader utilities provide JSON file loading, section-based file parsing, and directory scanning for agent configurations and tools.

Overview

┌─────────────────────────────────────────────────────────────────┐
│                      File Loader System                          │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐          │
│  │ FileLoader   │  │ FileLoader.  │  │ FileLoader.  │          │
│  │ (main API)   │  │ JSON         │  │ Section      │          │
│  │              │  │ (parsing)    │  │ (extract)    │          │
│  └──────────────┘  └──────────────┘  └──────────────┘          │
│                                                                  │
│  Functions:                                                      │
│  - listJsonDirectory  - readJsonDescriptionFile                  │
│  - loadJsonFile       - parseAgentJson                           │
│  - findSection        - extractSection                           │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Core Types

FileLoader (`System.Agents.FileLoader`)

module System.Agents.FileLoader
    ( module System.Agents.Base
    , module System.Agents.FileLoader.JSON
    , Trace (..)
    , Agents (..)
    , listJsonDirectory
    , loadJsonFile
    , InvalidAgentError (..)
    ) where

data Trace
    = LoadJsonFile !FilePath
    | LoadJsonFileFailure !FilePath String
    deriving (Show)

data Agents = Agents
    { dir :: FilePath
    , agents :: [AgentDescription]
    }

data InvalidAgentError
    = LoadFailure FilePath String
    deriving (Show)

FileLoader.JSON (`System.Agents.FileLoader.JSON`)

module System.Agents.FileLoader.JSON where

readJsonDescriptionFile :: 
    FilePath -> 
    IO (Either String AgentDescription)

parseAgentJson :: 
    ByteString -> 
    Either String AgentDescription

FileLoader.Section (`System.Agents.FileLoader.Section`)

module System.Agents.FileLoader.Section where

-- Extract a delimited section from file content
extractSection :: 
    Text ->           -- Start delimiter (e.g., "----BEGIN----")
    Text ->           -- End delimiter (e.g., "----END----")
    Text ->           -- File content
    Maybe Text        -- Extracted section

-- Find section with default delimiters
findSection :: 
    Text ->           -- Section name
    Text ->           -- File content  
    Maybe Text        -- Extracted content

Directory Scanning

listJsonDirectory

Recursively finds all .json files in a directory:

listJsonDirectory :: FilePath -> IO [FilePath]
listJsonDirectory path = do
    entries <- listDirectory path
    files <- forM entries $ \entry -> do
        let fullPath = path </> entry
        isDir <- doesDirectoryExist fullPath
        if isDir
            then listJsonDirectory fullPath
            else return [fullPath | isJson fullPath]
    return $ concat files
  where
    isJson p = takeExtension p == ".json"

Example:

-- Directory structure:
-- agents/
--   ├── main.json
--   ├── helpers/
--   │   ├── helper1.json
--   │   └── helper2.json
--   └── utils.json

listJsonDirectory "agents"
-- Returns: ["agents/main.json", "agents/helpers/helper1.json", 
--           "agents/helpers/helper2.json", "agents/utils.json"]

With Tracing

loadJsonFile :: 
    Tracer IO Trace -> 
    FilePath -> 
    IO (Either InvalidAgentError AgentDescription)
loadJsonFile tracer path = do
    runTracer tracer (LoadJsonFile path)
    result <- readJsonDescriptionFile path
    case result of
        Right desc -> return $ Right desc
        Left err -> do
            runTracer tracer (LoadJsonFileFailure path err)
            return $ Left $ LoadFailure path err

JSON Parsing

Agent Description Parsing

readJsonDescriptionFile :: FilePath -> IO (Either String AgentDescription)
readJsonDescriptionFile path = do
    contents <- ByteString.readFile path
    return $ parseAgentJson contents

parseAgentJson :: ByteString -> Either String AgentDescription
parseAgentJson = eitherDecodeStrict

JSON Schema

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "my-agent",
    "apiKeyId": "openai",
    "flavor": "openai",
    "modelUrl": "https://api.openai.com/v1",
    "modelName": "gpt-4",
    "announce": "A helpful assistant",
    "systemPrompt": ["You are helpful."],
    "toolDirectory": "tools",
    "mcpServers": [
      {
        "tag": "McpSimpleBinary",
        "contents": {
          "name": "filesystem",
          "executable": "/usr/bin/mcp-fs",
          "args": ["--root", "/home/user"]
        }
      }
    ],
    "openApiToolboxes": [
      {
        "tag": "OpenAPIServer",
        "contents": {
          "SpecUrl": "https://api.example.com/openapi.json",
          "BaseUrl": "https://api.example.com",
          "Headers": null,
          "Token": "${API_TOKEN}"
        }
      }
    ],
    "extraAgents": [
      {"slug": "helper", "path": "./helper.json"}
    ]
  }
}

Section Extraction

Use Case: Multi-part Prompt Files

Section extraction allows embedding multiple content blocks in a single file:

# My Prompt File

Some instructions here.

----BEGIN SYSTEM----
You are a code reviewer. Focus on:
- Security issues
- Performance
- Maintainability
----END SYSTEM----

----BEGIN EXAMPLE----
Example review output format...
----END EXAMPLE----

Rest of file...

Extracting Sections

import System.Agents.FileLoader.Section

content <- Text.readFile "prompt.md"

let systemPrompt = extractSection 
        "----BEGIN SYSTEM----" 
        "----END SYSTEM----" 
        content
        
let exampleOutput = extractSection
        "----BEGIN EXAMPLE----"
        "----END EXAMPLE----"
        content

Named Sections

-- Using findSection with default delimiters
findSection :: Text -> Text -> Maybe Text
findSection name content = 
    extractSection start end content
  where
    start = "----BEGIN " <> name <> "----"
    end   = "----END " <> name <> "----"

-- Usage
findSection "PROMPT" content
findSection "CONTEXT" content
findSection "EXAMPLES" content

Configuration Loading

agents-exe.cfg.json

Project configuration discovery:

locateAgentsExeConfig :: IO (Maybe FilePath)
locateAgentsExeConfig = do
    go =<< getCurrentDirectory
  where
    go "" = return Nothing
    go "/" = return Nothing
    go path = do
        let tentative = path </> "agents-exe.cfg.json"
        exists <- doesFileExist tentative
        if exists
            then return $ Just tentative
            else go (takeDirectory path)

Search order:

Start at current directory
Check for agents-exe.cfg.json
If not found, move to parent
Repeat until root

Configuration Schema

data AgentsExeConfig = AgentsExeConfig
    { agentsConfigDir :: Maybe FilePath
    , agentsDirectories :: [FilePath]
    , agentsFiles :: [FilePath]
    , agentsLogs :: Maybe AgentsExeLogConfig
    } deriving (Show, Generic)

data AgentsExeLogConfig = AgentsExeLogConfig
    { logJsonHttpEndpoint :: Maybe String
    , logJsonPath :: Maybe FilePath
    , logRawPath :: Maybe FilePath
    , logSessionsJsonPrefix :: Maybe FilePath
    } deriving (Show, Generic)

instance FromJSON AgentsExeConfig
instance FromJSON AgentsExeLogConfig

Error Handling

JSON Parse Errors

data InvalidAgentError
    = LoadFailure FilePath String
    deriving (Show)

-- Example error messages:
-- "LoadFailure ./agent.json: Error in $.modelName: expected Text, got Number"
-- "LoadFailure ./agent.json: Missing required field 'systemPrompt'"

File Not Found

handleFileError :: FilePath -> IOException -> IO (Either InvalidAgentError a)
handleFileError path e = return $ Left $ LoadFailure path (show e)

loadJsonFileSafe :: FilePath -> IO (Either InvalidAgentError AgentDescription)
loadJsonFileSafe path = 
    (loadJsonFile silentTracer path) `catch` handleFileError path

Validation

validateAgent :: Agent -> Either ValidationError ()
validateAgent agent = do
    when (T.null $ slug agent) $ 
        Left $ MissingField "slug"
    when (T.null $ apiKeyId agent) $ 
        Left $ MissingField "apiKeyId"
    when (T.null $ modelUrl agent) $ 
        Left $ MissingField "modelUrl"
    -- ... more validation

File Watching

Hot Reload Support

The bash toolbox uses file watching for hot reloading:

-- From BashToolbox module
initializeBackroundToolbox :: 
    Tracer IO Trace -> 
    FilePath -> 
    IO (Either ToolboxError Toolbox)
initializeBackroundToolbox tracer dir = do
    -- Initial tool discovery
    tools <- discoverTools dir
    
    -- Create background thread for watching
    backgroundVal <- Background.backgroundVal 
        tracer
        (BashToolsLoadingTrace . map toolName)
        (loadTools dir)
        
    return $ Right Toolbox
        { tools = backgroundVal
        , triggerReload = Background.triggerReload backgroundVal
        }

File Notification

-- From FileNotification module
watchDirectory :: 
    FilePath -> 
    (FilePath -> IO ()) ->  -- Callback on change
    IO WatchHandle

Best Practices

Configuration Files

Use .json extension: Consistent with loader expectations
Validate on load: Check required fields
Use UTF-8 encoding: Standard for JSON
Pretty print: Human-readable config files
Comments in separate file: JSON doesn’t support comments

Section Extraction

Clear delimiters: Use distinctive markers
Nesting awareness: Don’t overlap sections
Graceful degradation: Handle missing sections
Document format: Explain section usage

Directory Structure

project/
├── agents-exe.cfg.json      # Project config
├── agents/
│   ├── main.json            # Main agent
│   ├── helpers/
│   │   ├── code-helper.json
│   │   └── doc-helper.json
│   └── shared-tools/        # Shared tool library
├── sessions/                # Session storage
│   └── session-*.json
└── prompts/                 # Reusable prompts
    ├── code-review.md
    └── pr-description.md

Examples

Loading Multiple Agents

import System.Agents.FileLoader
import Prod.Tracer (silentTracer)

loadAllAgents :: FilePath -> IO [AgentDescription]
loadAllAgents dir = do
    jsonFiles <- listJsonDirectory dir
    results <- mapM (loadJsonFile silentTracer) jsonFiles
    return [agent | Right agent <- results]

-- With error reporting
loadAllAgentsWithErrors :: FilePath -> IO ([AgentDescription], [InvalidAgentError])
loadAllAgentsWithErrors dir = do
    jsonFiles <- listJsonDirectory dir
    results <- mapM (loadJsonFile silentTracer) jsonFiles
    let (errors, agents) = partitionEithers results
    return (agents, errors)

Extracting Code from Markdown

import System.Agents.FileLoader.Section

extractHaskellCode :: Text -> Maybe Text
extractHaskellCode content = 
    extractSection "```haskell" "```" content
    <|> extractSection "```hs" "```" content

extractAllSections :: Text -> [Text]
extractAllSections content =
    catMaybes $ map (`findSection` content) sectionNames
  where
    sectionNames = ["PROMPT", "CONTEXT", "EXAMPLES", "INSTRUCTIONS"]

Custom Config Loading

loadProjectConfig :: IO (Maybe AgentsExeConfig)
loadProjectConfig = do
    mPath <- locateAgentsExeConfig
    case mPath of
        Nothing -> return Nothing
        Just path -> do
            result <- eitherDecodeFileStrict' path
            case result of
                Left err -> do
                    putStrLn $ "Config error: " ++ err
                    return Nothing
                Right config -> return $ Just config

Integration with Other Modules

AgentTree Integration

-- AgentTree uses FileLoader to discover agents
loadAgentTree :: FilePath -> IO (Either AgentTreeError AgentTree)
loadAgentTree rootFile = do
    -- Load root agent
    rootResult <- readJsonDescriptionFile rootFile
    case rootResult of
        Left err -> return $ Left $ LoadError err
        Right rootDesc -> do
            -- Discover child agents
            let toolDir = toolDirectory (agent rootDesc)
            childFiles <- listJsonDirectory toolDir
            -- ... build tree

CLI Integration

-- Main.hs uses FileLoader for agent discovery
initArgParserArgs :: IO ArgParserArgs
initArgParserArgs = do
    agentsExecConfig <- locateAgentsExeConfig
    case agentsExecConfig of
        Just path -> loadFromConfig path
        Nothing -> loadDefaults
  where
    loadFromConfig path = do
        config <- eitherDecodeFileStrict' path
        jsonPaths <- traverse listJsonDirectory (agentsDirectories config)
        return $ ArgParserArgs
            { defaultAgentFiles = agentsFiles config <> concat jsonPaths
            , ...
            }

Agent templates

2026-10-04T15:53:54Z

Generated from documentation/agent-templates.md, the repository is the canonical source and may be ahead of this page.

Agent templates

An agent file can be a tramaj program instead of JSON. The program is evaluated once, when the agent is loaded, and must produce the JSON an agent file holds. From there on nothing changes: the value goes through the same parser as a .json file, and check, run, tui, serve and the rest see an ordinary agent.

Use a template when several agents should share a sandbox or a toolbox, or when one agent definition should vary with a value the operator picks.

A first template
Where templates are found
Parameters: $ctx
Libraries
The agents library
Seeing the result
Limits

A first template

agents/coder.tramaj:

@a=import("agents", {}).vals

@code=$a.sandbox({
  name: "code",
  allow: $a.under($ctx.workspace, ["src", "test"]),
  deny: [$a.pattern("*.key")]
})

$a.agent({
  slug: "coder",
  apiKeyId: "main-key",
  flavor: "OpenAIv1",
  modelUrl: "https://api.openai.com/v1",
  modelName: "gpt-4o",
  announce: "edits code under the workspace",
  systemPrompt: ["You edit code under `$ctx.workspace`."],
  parameters: [{name: "workspace", description: "root of the checkout"}],
  builtinToolboxes: [
    $a.developer-toolbox({
      name: "dev",
      capabilities: ["read-file-range", "write-file-range", "patch-file"],
      sandbox: $code
    }),
    $a.lua-toolbox({name: "lua", sandbox: $code})
  ]
})

agents-exe --set workspace=/srv/app check --agent-file agents/coder.tramaj

@name=expr binds a name, $name reads it, and the last expression is the result. $code is written once and used by two toolboxes. The language itself (objects, arrays, lambdas, map, branch, string interpolation with backticks) is described in tramaj’s reference.

One program is one agent. The result is the whole agent file, envelope included ({"tag": "OpenAIAgentDescription", "contents": {...}}), which is what $a.agent({...}) builds.

Where templates are found

Wherever a .json agent file is, under the extension .tramaj:

--agent-file path/to/agent.tramaj;
agentsFiles and agentsDirectories in agents-exe.cfg.json;
the default directory, ~/.config/agents-exe/default;
an agent’s tool directories, where sub-agents are discovered;
the path of an extraAgents entry.

--agent SLUG evaluates templates to learn their slugs. A template that does not evaluate (a parameter it needs has no value, say) is set aside there, so that it does not prevent selecting another agent.

Parameters: `$ctx`

A template’s $ctx holds the process parameters: the values given with --set, --set-json, --pin, --pin-json and --params-file (see Parameters, bindings, and narrowing sub-agents). $ctx.workspace is the value of --set workspace=....

A template declares what it reads, in the parameters of the agent it produces, as an agent declares the parameters its bindings use. Loading fails when the template:

reads	error
a parameter with no value	names the parameter and the `--set` flag to give
a parameter absent from `parameters`	undeclared parameter
a parameter declared with scope `session` or `message`	a template is evaluated once, at load, and those have no value then
a parameter declared `secret`	the value would be written into the configuration; bind it to a tool argument instead
`$ctx` as a whole (`lookup($ctx, ...)`, `has($ctx, ...)`)	each parameter is read by name, so that the reads can be checked

Evaluation happens at load and only then. A value that changes per session or per message stays a parameter binding; it is not something a template can read.

A declared default is not seen by the template: only values the operator supplied are in $ctx.

Libraries

A library is a tramaj file whose top-level bindings other programs read:

libs/team.tramaj:

@a=import("agents", {}).vals
@code=$a.sandbox({name: "code", allow: $a.under($ctx.workspace, ["src", "test"])})
@dev=$a.developer-toolbox({name: "dev", capabilities: ["read-file-range"], sandbox: $code})
null

An agent template imports it by file name, passing what the library reads from its own $ctx:

@a=import("agents", {}).vals
@team=import("team", {workspace: ctx(workspace)}).vals

$a.agent({
  slug: "reviewer",
  ...
  parameters: [{name: "workspace"}],
  builtinToolboxes: [$team.dev]
})

ctx(workspace) hands the template’s own $ctx.workspace to the library, and counts as a read of workspace.

Library directories are listed in agents-exe.cfg.json:

{
  "agentsDirectories": ["./agents"],
  "tramajLibraries": ["./libs"]
}

Every .tramaj file of a listed directory is a library named after the file (libs/team.tramaj is team). When two directories hold the same name, the earlier one wins, and a directory wins over the built-in agents library. A library that does not parse stops startup, with its file name.

Keep library directories apart from agent directories: in an agent directory, every .tramaj file is loaded as an agent.

The `agents` library

Always available, as import("agents", {}). Every function takes one object; fields marked optional may be left out.

Path predicates

function	result
`dir(path)`	the directory and everything below it
`dir-shallow(path)`	the directory's own files
`dir-exactly(path)`	the directory itself
`file(path)`	one file
`pattern(glob)`	file names matching a glob
`extensions([ext])`	file extensions, without the dot
`any-of([p])`, `all-of([p])`, `none-of([p])`	combinations
`allow-all`, `deny-all`	constants
`under(root, [path])`	a list: `dir` of each path below `root`

Sandboxes

sandbox({allow, deny, name, maxFileSize}): allow is a list of predicates and is required; deny, name and maxFileSize are optional. A path is allowed when an allow entry matches and no deny entry does.

sandbox-ref(name): a reference to a sandbox the agent declares in its fileSandboxes map (see Named Sandboxes), usable wherever a toolbox takes sandbox:

$a.agent({
  ...,
  fileSandboxes: {code: $a.sandbox({allow: [$a.dir("./src")]})},
  builtinToolboxes: [
    $a.developer-toolbox({name: "dev", capabilities: ["read-file-range"], sandbox: $a.sandbox-ref("code")}),
    $a.lua-toolbox({name: "lua", sandbox: $a.sandbox-ref("code")})
  ]
})

Toolboxes

function	fields
`developer-toolbox`	`name`, `capabilities`; optional `description`, `sandbox`, `activation`, `buildCommand`
`system-toolbox`	`name`, `capabilities`; optional `description`, `sandbox`, `activation`, `envVarFilter`, `commandFilter`
`lua-toolbox`	`name`; optional `description`, `maxMemoryMB`, `maxExecutionTimeSeconds`, `allowedTools`, `allowedHosts`, `sandbox`, `activation`
`tools-dir`	`path`; optional `filter`, `activation`, `bindings` (a `bashToolboxes` entry)
`tool`	`path`; optional `activation`, `bindings` (a `bashToolboxes` entry)
`mcp`	`name`, `executable`; optional `args`, `activation`, `env` (an `mcpServers` entry)
`helper`	`slug`, `path`; optional `with`, `narrowable` (an `extraAgents` entry)

Agent

agent({...}) wraps an agent’s fields in the file’s envelope.

Anything the library has no function for is written as the plain JSON object it is: a template’s objects are JSON objects.

Seeing the result

agents-exe --set workspace=/srv/app check --show-config --agent-file agents/coder.tramaj

prints the JSON the template evaluated to, then the usual check line. --show-config prints a .json agent file as it is.

Limits

mcp-server, tool-call and session take no process parameters, so a template that reads $ctx does not load there. session also sees the built-in library only.
The developer toolbox’s validate-agent and create-agent tools read JSON agent files only.
Agents stored through agents-server’s API are JSON; a stored agent cannot be a template yet, and there are no stored libraries.
Plain agents-server has the built-in library only. agents-exe serve adds the tramajLibraries of agents-exe.cfg.json.

Advanced Configuration

2026-10-04T15:53:54Z

Generated from documentation/advanced-configuration.md, the repository is the canonical source and may be ahead of this page.

Advanced Configuration

This guide covers advanced configuration patterns for agents-exe, including recursive agent references, mutual recursion, and runtime safety controls.

Recursive Agent References
Recursion Safety
- Depth Limits
- Call Stack Tracking
Configuration Examples
Best Practices
Troubleshooting

Recursive Agent References

By default, agents can only reference sub-agents located in their toolDirectory. This creates a strict tree hierarchy. However, some use cases require:

Self-references: An agent that can call itself for iterative refinement
Mutual recursion: Agent A calls Agent B, which calls Agent A
Cross-tree references: Agents in different branches need to collaborate

The extraAgents configuration field enables these patterns by allowing explicit declarations of agent references outside the normal directory hierarchy.

How It Works

When an agent defines extraAgents, the system uses a two-phase initialization:

Discovery Phase: All agent configurations are loaded via BFS traversal
Validation Phase: References are validated (all slugs must exist)
Runtime Shell Phase: Runtime shells are created with empty tool lists
Wiring Phase: Tool references are resolved via the RuntimeRegistry
Tree Building Phase: The final AgentTree is constructed

This approach allows cycles in the agent reference graph while preventing runtime failures from missing references.

Self-References

An agent can reference itself to implement iterative refinement patterns:

┌─────────────────┐
│  refine-code    │◄────────────────┐
│  (self-ref)     │                 │
└────────┬────────┘                 │
         │                          │
         │ tool call                │
         ▼                          │
┌─────────────────┐                 │
│  refine-code    │─────────────────┘
│  (same agent)   │
└─────────────────┘

Use cases:

Iterative code improvement (“make it faster”, “add tests”)
Progressive document refinement
Recursive problem decomposition with depth limits

Mutual Recursion

Two or more agents can reference each other:

┌─────────────┐      ┌─────────────┐
│  planner    │─────►│  executor   │
│             │◄─────│             │
└─────────────┘      └─────────────┘

Use cases:

Planning and execution loops
Code generation and review cycles
Question-answering with fact-checking

Configuration via extraAgents

The extraAgents field in an agent’s JSON configuration declares references to agents outside the toolDirectory hierarchy.

Field Structure:

data ExtraAgentRef = ExtraAgentRef
    { extraAgentSlug :: AgentSlug   -- ^ How to reference this agent
    , extraAgentPath :: FilePath    -- ^ Path to agent JSON file
    }

Important: The slug field in extraAgents must match the target agent’s actual slug as defined in its JSON file. It is NOT an alias. The system uses this slug to look up the agent in the registry after all configurations are loaded.

For example, if you want to reference an agent defined in helper.json that has "slug": "my-helper", your extraAgents entry must use "slug": "my-helper", not any other name.

JSON Format:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "my-agent",
    "apiKeyId": "my-key",
    "flavor": "OpenAI",
    "modelUrl": "https://api.openai.com",
    "modelName": "gpt-4",
    "toolDirectory": "tools",
    "extraAgents": [
      {
        "slug": "helper-agent",
        "path": "../helpers/helper-agent.json"
      },
      {
        "slug": "my-agent",
        "path": "./agent.json"
      }
    ],
    "announce": "A recursive agent that can call itself.",
    "systemPrompt": [
      "You are a helpful assistant.",
      "You can call yourself recursively for complex tasks."
    ]
  }
}

Field Descriptions:

Field	Type	Description
`slug`	string	Must match the target agent's actual `slug` as defined in its JSON file
`path`	string	Relative or absolute path to the agent's JSON file

Warning: A common mistake is to use an arbitrary name for the slug field in extraAgents. This will cause a MissingAgentReference error because the system cannot find an agent with that slug. Always ensure the slug in extraAgents matches the slug field in the target agent’s configuration file.

Recursion Safety

Unbounded recursion can lead to infinite loops and excessive API costs. The system provides multiple safety mechanisms.

Depth Limits

The ToolExecutionContext tracks recursion depth via a call stack:

data ToolExecutionContext = ToolExecutionContext
    { ctxCallStack :: [CallStackEntry]  -- ^ Call chain tracking
    , ctxMaxDepth  :: Maybe Int         -- ^ Optional depth limit
    -- ... other fields
    }

data CallStackEntry = CallStackEntry
    { callAgentSlug      :: Text
    , callConversationId :: ConversationId
    , callDepth          :: Int
    }

When an agent calls another agent, the system:

Creates a new CallStackEntry with incremented depth
Checks against ctxMaxDepth if set
Returns Left (MaxRecursionDepthExceeded callStack) if limit exceeded
Otherwise, continues with the new context

Important: Currently, depth limits are enforced at runtime via the context, not at configuration time. Future versions may support maxRecursionDepth in agent configuration.

Call Stack Tracking

Every nested agent call appends to the call stack:

Depth 0: [root]                    -- Initial context
Depth 1: [agent-a, root]           -- After first call
Depth 2: [agent-b, agent-a, root]  -- After second call

The call stack enables:

Debugging: Full call chain in error messages
Cycle detection: Check if an agent is already in the stack
Context-aware behavior: Different logic at different depths

Helper Functions:

-- Check current recursion depth
currentRecursionDepth :: ToolExecutionContext -> Int

-- Check if at or beyond specific depth
isAtDepth :: Int -> ToolExecutionContext -> Bool

-- Get full call chain (root first)
callChain :: ToolExecutionContext -> [CallStackEntry]

-- Check for cycles (agent already in stack)
isAgentInCallStack :: Text -> ToolExecutionContext -> Bool

Configuration Examples

Basic Recursive Agent

An agent that can call itself for iterative code refinement:

Directory Structure:

refine-agent/
├── agent.json
└── tools/
    └── some-tool.sh

agent.json:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "refine-code",
    "apiKeyId": "openai-key",
    "flavor": "OpenAI",
    "modelUrl": "https://api.openai.com",
    "modelName": "gpt-4o",
    "toolDirectory": "tools",
    "extraAgents": [
      {
        "slug": "refine-code",
        "path": "./agent.json"
      }
    ],
    "announce": "Code refinement agent with self-recursion.",
    "systemPrompt": [
      "You are a code refinement specialist.",
      "Analyze the provided code and suggest improvements.",
      "If the task is complex, break it into steps and call",
      "the 'refine-code' tool iteratively, up to 3 levels deep.",
      "Track your progress and stop when improvements diminish."
    ]
  }
}

Note that the slug in extraAgents ("refine-code") matches the agent’s own slug field. This is required for self-references to work correctly.

Usage:

agents-exe run -p "Refactor this Python function for performance:" \
               -f myscript.py \
               --agent-file refine-agent/agent.json

Mutually Recursive Agents

A planner and executor that collaborate in a loop:

Directory Structure:

plan-exec/
├── planner.json
├── executor.json
└── tools/
    └── validate.sh

planner.json:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "task-planner",
    "apiKeyId": "openai-key",
    "flavor": "OpenAI",
    "modelUrl": "https://api.openai.com",
    "modelName": "gpt-4o",
    "toolDirectory": "tools",
    "extraAgents": [
      {
        "slug": "task-executor",
        "path": "./executor.json"
      }
    ],
    "announce": "Task planner that delegates to executor.",
    "systemPrompt": [
      "You are a strategic planner.",
      "Break down complex tasks into executable steps.",
      "Use the 'task-executor' tool to execute each step.",
      "Review results and iterate up to 5 times if needed."
    ]
  }
}

executor.json:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "task-executor",
    "apiKeyId": "openai-key",
    "flavor": "OpenAI",
    "modelUrl": "https://api.openai.com",
    "modelName": "gpt-4o",
    "toolDirectory": "tools",
    "extraAgents": [
      {
        "slug": "task-planner",
        "path": "./planner.json"
      }
    ],
    "announce": "Task executor that can request replanning.",
    "systemPrompt": [
      "You are an execution specialist.",
      "Implement the planned steps precisely.",
      "If execution fails or needs adjustment,",
      "call 'task-planner' with context for revised planning.",
      "Stop after 5 iterations to prevent infinite loops."
    ]
  }
}

Note that in planner.json, the extraAgents slug is "task-executor" which must match the slug field in executor.json. Similarly, in executor.json, the extraAgents slug is "task-planner" which must match the slug field in planner.json.

Usage:

agents-exe run -p "Set up a Django project with user auth:" \
               --agent-file plan-exec/planner.json

Hierarchical Agent with Recursion

A boss agent with recursive worker agents at different levels:

Directory Structure:

hierarchy/
├── boss.json
├── senior-dev.json
├── junior-dev.json
└── tools/
    ├── git.sh
    └── test.sh

boss.json:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "boss",
    "apiKeyId": "openai-key",
    "flavor": "OpenAI",
    "modelUrl": "https://api.openai.com",
    "modelName": "gpt-4o",
    "toolDirectory": "tools",
    "extraAgents": [
      {
        "slug": "senior-dev",
        "path": "./senior-dev.json"
      }
    ],
    "announce": "Project manager coordinating development.",
    "systemPrompt": [
      "You are a technical project manager.",
      "Delegate architecture decisions to senior-dev.",
      "Track overall progress and ensure quality."
    ]
  }
}

senior-dev.json:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "senior-dev",
    "apiKeyId": "openai-key",
    "flavor": "OpenAI",
    "modelUrl": "https://api.openai.com",
    "modelName": "gpt-4o-mini",
    "toolDirectory": "tools",
    "extraAgents": [
      {
        "slug": "junior-dev",
        "path": "./junior-dev.json"
      },
      {
        "slug": "senior-dev",
        "path": "./senior-dev.json"
      }
    ],
    "announce": "Senior developer for architecture and review.",
    "systemPrompt": [
      "You are a senior software developer.",
      "Design architecture and review implementations.",
      "Delegate implementation tasks to junior-dev.",
      "Use self-recursion for complex code reviews (max 2 levels)."
    ]
  }
}

junior-dev.json:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "junior-dev",
    "apiKeyId": "openai-key",
    "flavor": "OpenAI",
    "modelUrl": "https://api.openai.com",
    "modelName": "gpt-4o-mini",
    "toolDirectory": "tools",
    "extraAgents": [],
    "announce": "Junior developer for implementation.",
    "systemPrompt": [
      "You are a junior software developer.",
      "Implement features following senior-dev specifications.",
      "Write tests and documentation for all code."
    ]
  }
}

Best Practices

1. Always Set Depth Limits

When implementing recursive patterns, explicitly track depth and set limits:

-- In tool implementation
case pushAgentContext "my-agent" newConvId parentCtx of
    Left (MaxRecursionDepthExceeded stack) -> 
        return $ "Max depth reached. Current chain: " ++ show (map callAgentSlug stack)
    Right newCtx -> continueWithSubAgent newCtx

2. Document Recursion Patterns

Add clear documentation in system prompts:

{
  "systemPrompt": [
    "You can call yourself recursively using the 'my-agent' tool.",
    "IMPORTANT: Stop after 3 levels of recursion.",
    "Track recursion depth in your reasoning."
  ]
}

3. Use Appropriate Models

Different recursion depths benefit from different models:

Depth	Recommendation
1-2 (root + 1 level)	Use strongest model (gpt-4o)
3-5	Can use lighter model (gpt-4o-mini)
5+	Review architecture; may indicate wrong approach

4. Validate Configuration

Use the check command to validate recursive configurations:

agents-exe check --agent-file my-recursive-agent.json

This will:

Verify all extraAgents paths resolve
Check for duplicate slugs
Warn about detected cycles (cycles are allowed but warned)

5. Monitor Costs

Recursive agents can incur unexpected API costs:

Log recursion depth in your traces
Set up budget alerts
Consider using lighter models for deep recursion
Test with depth limits before removing them

6. Prefer Explicit Over Implicit

Be explicit about which agents can recurse:

{
  "extraAgents": [
    {"slug": "explicit-self", "path": "./agent.json"}
  ]
}

Rather than relying on implicit discovery.

7. Handle Cycle Warnings

When the system detects cycles, it logs warnings but continues:

CyclicReferencesWarning [["agent-a", "agent-b", "agent-a"]]

Review these warnings to ensure cycles are intentional.

Troubleshooting

“MissingAgentReference” Error

Problem: An agent references a slug that doesn’t exist.

ReferenceError: Missing agent reference 'self-reference'
  Referrer: 'local:lrm' in file: localdev-agents/kimi-07.err1.json
  
  Hint: The slug 'self-reference' was not found in any loaded agent configuration.
  
  Did you mean to reference one of these agents?
    - 'local:lrm' (defined in localdev-agents/kimi-07.err1.json)
  
  Remember: The 'slug' in extraAgents must match the target agent's actual slug,
  not an arbitrary name. Check that your extraAgents configuration uses the 
  correct slug from the target file.

Common Causes:

The slug in extraAgents doesn’t match the target agent’s actual slug
The referenced agent file doesn’t exist at the specified path
The referenced agent file has a different slug than expected

Solutions:

Check that the slug in extraAgents matches the target agent’s slug field exactly
Verify the path in extraAgents points to the correct file
Open the target agent’s JSON file and confirm its slug field
Remember: The slug in extraAgents is NOT an alias - it must match the target’s actual slug

Example Fix:

If you have an agent file helper.json:

{
  "contents": {
    "slug": "my-helper",
    ...
  }
}

Your extraAgents entry must use "slug": "my-helper":

{
  "extraAgents": [
    {
      "slug": "my-helper",
      "path": "./helper.json"
    }
  ]
}

Using "slug": "helper" or any other name will cause the error.

“DuplicateAgentSlug” Error

Problem: The same slug is defined in multiple files.

ReferenceError (DuplicateAgentSlug "helper" ["./a/helper.json", "./b/helper.json"])

Solutions:

Rename one of the agents
Use different slugs for different purposes
Consolidate duplicate definitions

Infinite Recursion

Problem: Agents call each other indefinitely.

Symptoms:

High API costs
Sessions don’t terminate
Repeated tool calls with similar parameters

Solutions:

Add explicit depth tracking to system prompts
Use pushAgentContext with ctxMaxDepth in tool implementations
Add termination conditions (“stop if no improvements”)
Review and simplify agent collaboration patterns

Performance Issues

Problem: Recursive agents are slow.

Solutions:

Use lighter models for deep recursion levels
Reduce context window by summarizing earlier results
Cache intermediate results
Consider flattening the recursion into iteration

Summary

Recursive agent references enable powerful patterns like iterative refinement and collaborative problem-solving. The key points are:

Use extraAgents to declare references outside toolDirectory
Important: The slug in extraAgents must match the target agent’s actual slug
The system validates all references before runtime
Implement depth limits via ToolExecutionContext
Monitor for cycles and unintended infinite recursion
Document recursion patterns in system prompts

For more information, see:

System.Agents.AgentTree - Loading and initialization
System.Agents.Tools.Context - Recursion tracking
System.Agents.Base - ExtraAgentRef type

Parameters, bindings, and narrowing sub-agents

2026-10-04T15:53:54Z

Generated from documentation/parameters-and-bindings.md, the repository is the canonical source and may be ahead of this page.

Parameters, bindings, and narrowing sub-agents

Most agent frameworks give a tool’s arguments to the model in full, or hide them entirely behind a wrapper script the author has to hand-maintain. This feature gives you a middle ground with no code: an agent’s author declares a handful of named parameters — a tenant id, an API token, a base URL — and then binds them straight into any tool’s arguments. The model never sees the bound value, never has to be told it, and never gets a chance to mistype, leak, or override it. A secret parameter never even enters the conversation, the session file, or a trace.

The same mechanism reaches down the agent tree: one agent can partially apply a sub-agent’s tools before handing it a task — “call the diff reviewer, but its own git diff tool is pinned to this repo and this branch” — without changing the sub-agent’s file at all, and without ever seeing the values the sub-agent’s own author chose to keep hidden.

If you only need one binding once, Bindings is all you need. The rest of this document exists for when you need more: per-session values from agents-server, sub-agents that narrow other sub-agents, or a narrowing worth reusing under a name.

Quick example
Declaring parameters
Supplying values
Bindings
Secrets
Passing parameters to sub-agents (with)
Narrowing sub-agents at call time
Naming a narrowing (derive_agent)
Design notes

Quick example

An agent that declares a tenant parameter and binds it into a bash tool’s tenant_id argument, so the model can call query-invoices without ever knowing — or choosing — which tenant it is querying:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "invoices-agent",
    "apiKeyId": "openai",
    "flavor": "openai",
    "modelUrl": "https://api.openai.com/v1",
    "modelName": "gpt-4",
    "announce": "answers questions about a tenant's invoices",
    "systemPrompt": ["You are an invoices assistant."],
    "parameters": [
      { "name": "tenant", "scope": "session", "required": true }
    ],
    "bashToolboxes": [
      {
        "tag": "FileSystemDirectory",
        "contents": {
          "Path": "./tools",
          "Bindings": [
            { "arg": "tenant_id", "value": { "tag": "Param", "contents": "tenant" } }
          ]
        }
      }
    ]
  }
}

query-invoices’ schema, as the model sees it, has no tenant_id argument at all — it is indistinguishable from a tool that never had one. Run it with a value for tenant:

agents-exe run --agent-file agent.json --set tenant=acme-corp --prompt "how many invoices are overdue?"

Declaring parameters

An agent’s parameters field is a list of named holes its author declares; a caller supplies the value. Nothing here is visible to the model — a parameter is plumbing, not a tool argument the LLM fills in.

{
  "parameters": [
    {
      "name": "tenant",
      "description": "which tenant's data to operate on",
      "scope": "session",
      "required": true
    },
    {
      "name": "github_token",
      "secret": true,
      "scope": "process",
      "default": { "tag": "EnvVar", "contents": "GITHUB_TOKEN" }
    }
  ]
}

Field	Default	Meaning
`name`	—	Matched by `Param` bindings, `--set`/`--pin`, `with`, and the HTTP API.
`description`	(none)	For the operator's own documentation; never shown to the model.
`secret`	`false`	The value is a string, and is never persisted, traced, or returned by any API — see [Secrets](#secrets).
`scope`	`"process"`	Who may supply it: `"process"` (only the operator, at startup), `"session"` or `"message"` (a caller of `agents-server`, see [Supplying values](#supplying-values)).
`required`	`true`	An unbound required parameter is an error before any LLM call — at startup for a process-scope parameter, at session creation or message post for the others.
`default`	(none)	Resolved once, at tree load, if nothing else supplies a value first. Reuses the same source shapes as a toolbox secret: `{"tag": "Given", "contents": "literal"}`, `{"tag": "EnvVar", "contents": "NAME"}`, `{"tag": "FileSystem", "contents": "/path"}`, `{"tag": "Command", "contents": ["cmd", "arg1", "arg2"]}`, or `{"tag": "ApiKey", "contents": "key-id"}` (an entry of the `--api-keys` file).

Resolution order for one parameter, first hit wins: a pinned process value (--pin) short-circuits everything else; then a message-scope value; then a session-scope value; then a process value (--set/--set-json/ --params-file); then default; otherwise it is unbound.

Supplying values

`agents-exe` (process scope)

agents-exe run --agent-file agent.json \
    --set tenant=acme-corp \
    --set-json max_results=10 \
    --pin region=eu-west-1 \
    --params-file ./params.json \
    --prompt "..."

Flag	Meaning
`--set NAME=VALUE`	Sets a parameter to a string value. Repeatable.
`--set-json NAME=JSON`	Sets a parameter to any JSON value (number, bool, object...). Repeatable.
`--pin NAME=VALUE`	Like `--set`, but the value cannot be overridden by a session- or message-scope value from `agents-server` — useful to lock a container to one tenant. Repeatable.
`--pin-json NAME=JSON`	The JSON form of `--pin`. Repeatable.
`--params-file FILE`	A JSON file of `{"name": value, ...}`. Flags win over a file's values.

These flags work the same way on agents-server (see agents-server.md), where they set values every session shares.

`agents-server` (session and message scope)

A session- or message-scope parameter is supplied per request, in the same params object used by POST /v1/sessions, .../messages, .../resume, and POST /v1/continuations/:token:

curl -sS localhost:8080/v1/sessions \
    -d '{"agent": "invoices-agent", "message": {"text": "how many invoices are overdue?"}, "params": {"tenant": "acme-corp"}}'

A session-scope value is kept in memory for the life of the session (a client must resupply it after a restart or eviction); a message-scope value lasts for that one run only. Neither is ever written to the session file — see agents-server.md for the full request shape and error responses, and for setting parameters over MCP-over-HTTP via Agents-Param-<name> headers.

Bindings

A Bindings list on a toolbox description ties one tool’s argument to a fixed value or to a parameter. It is supported on bash toolboxes (both FileSystemDirectory and SingleTool), OpenAPI toolboxes, and PostgREST toolboxes:

{
  "tag": "FileSystemDirectory",
  "contents": {
    "Path": "./tools",
    "Bindings": [
      { "arg": "tenant_id", "value": { "tag": "Param", "contents": "tenant" } },
      { "arg": "token", "value": { "tag": "Param", "contents": "github_token" } },
      { "tool": "export-*", "arg": "format", "value": { "tag": "Literal", "contents": "csv" } }
    ]
  }
}

Field	Default	Meaning
`tool`	(all tools)	A glob (`*` matches any run of characters) against the tool's name, to bind the same argument differently across several tools.
`arg`	—	The argument name, as the underlying tool declares it.
`value`	—	`{"tag": "Literal", "contents": <json>}` for a fixed value chosen by the agent's author, or `{"tag": "Param", "contents": "<parameter name>"}`, resolved fresh at call time.
`whenUnbound`	`"fail"`	What happens if a `Param` binding has no value at call time: `"fail"` (the call fails with a message an operator can act on; the model just sees a generic "tool not found"), `"omit"` (the argument is left out — only valid for an optional argument), or `"expose"` (the argument reappears in the tool's schema for the model to fill in itself, for as long as it is unbound — see below).

A bound argument is removed from the tool’s schema entirely — indistinguishable from a tool that never had it. If the model sends the argument’s name anyway, the bound value wins.

An agent itself can also carry a top-level bindings list, applied after every toolbox’s own bindings and matched by the tool’s final, LLM-visible name — the one way to bind an argument on a tool from a toolbox that has no per-tool tool glob of its own (a bash tool, for instance):

{
  "bindings": [
    { "tool": "bash_query_invoices", "arg": "tenant_id", "value": { "tag": "Param", "contents": "tenant" } }
  ]
}

`whenUnbound: "expose"`

Most bindings should fail or omit when unbound: a tool the model can only half-use is worse than one it cannot see. "expose" is the deliberate exception, for an argument that is usually supplied for the model but should fall back to letting the model choose when it is not: the tool’s schema is a live function of whether the session currently has a value for that parameter, updating as an agents-server session’s parameters are set — no agent rebuild, no restart. A secret parameter cannot use "expose": it would just let the model retype the secret in plain text, defeating the reason it was marked secret.

Secrets

A secret: true parameter’s value is a string, and:

is redacted ("<secret>") everywhere it might otherwise be printed, logged, or traced;
is never persisted — a session’s stored parameters only ever include non-secret ones; a secret session-scope value lives only in agents-server’s own memory, and is lost on restart or eviction;
is never returned by any agents-server API response;
never appears in the conversation itself: a bound value is merged into the tool call just before dispatch, after the model’s turn is already decided, so it is not part of the session’s own history.

A bash tool argument bound to a secret parameter must use the bash env calling mode ("mode": "env"), which passes it as an environment variable rather than on the command line — anything else would show up in ps and in traces regardless of how carefully the framework kept it from the model. agents-exe check refuses to load an agent that gets this wrong.

An MCP server gets a secret through its env map ({"tag": "Param", "contents": "name"}); see mcp.md. The server starts once per tree, so only process-scope parameters can be used there.

Passing parameters to sub-agents (`with`)

Parameter values never travel to a sub-agent implicitly — a name like tenant is not special, and a sub-agent that happens to declare the same parameter name should not silently inherit whatever the caller set. An extraAgents reference’s with field says explicitly which of the sub-agent’s own parameters the caller fills, and from what:

{
  "extraAgents": [
    {
      "slug": "diff-reviewer",
      "path": "./diff-reviewer.json",
      "with": {
        "repo_token": { "tag": "Param", "contents": "github_token" }
      }
    }
  ]
}

with is resolved against the caller’s own parameters when the sub-agent is actually prompted, and is checked at load time: every key must be a parameter the sub-agent declares, and every required session-scope parameter of the sub-agent must be covered — by with, or by the sub-agent’s own process value or default.

Narrowing sub-agents at call time

Every sub-agent reachable through extraAgents or a toolDirectory gets a prompt_agent_<slug> tool. Two more things come with it, letting an agent partially apply a sub-agent’s own tools — or one of its sub-agents’, arbitrarily deep — for one call, without touching any file:

describe_agent {"slug": "<helper>"} — registered whenever an agent has at least one helper. It shows, from the caller’s own position in the chain, what is still open: the helper’s declared parameters (and whether each is already bound), its tools’ still-open arguments, and the same view recursively for its own helpers. A helper marked narrowable: false (see below) shows only its announce.
bindings and with on the prompt_agent_<slug> call itself — the same shapes as a toolbox’s Bindings and a reference’s with, but chosen by the calling model, for this one call:
```
{
  "what": "review this diff for security issues",
  "bindings": [
    { "arg": "repo", "value": { "tag": "Literal", "contents": "backend" } },
    { "agent": "linter", "arg": "strict", "value": { "tag": "Literal", "contents": true } }
  ],
  "with": { "severity_threshold": { "tag": "Literal", "contents": "high" } }
}
```
Each binding’s optional agent field addresses where it applies: omitted for the helper itself, a slash-separated path ("linter", "a/b") for one of its own helpers, or "**" for the helper and everything below it. A binding not addressed at the helper itself travels down, re-rooted, for the helper to apply in turn when it prompts one of its own helpers — so a grandparent can bind an argument three levels down in one call. A call-time with cannot refill a parameter the reference’s own static with already fills.

A helper’s extraAgents reference can set "narrowable": false (default: true) to refuse all of this: describe_agent shows only its announce, and any bindings/with on a call into it — direct or inherited from above — is refused. Bound arguments stay bound either way: narrowable only controls whether more can be bound from outside, not whether an operator’s own bindings in the helper’s file are visible.

Errors the calling model can act on — an unknown helper, no tool matching a glob, no such open argument, a narrowable: false target — are reported before the helper runs, naming what is still open, so one retry usually suffices. What is actually bound never appears in the response either way, by the same rule as any other binding.

Naming a narrowing (`derive_agent`)

Repeating the same bindings on every call costs tokens and invites inconsistency. derive_agent saves one under a name for the rest of the session:

{
  "from": "diff-reviewer",
  "slug": "backend-strict",
  "bindings": [{ "arg": "repo", "value": { "tag": "Literal", "contents": "backend" } }],
  "with": { "severity_threshold": { "tag": "Literal", "contents": "high" } }
}

and any later prompt_agent_<from> call reuses it with "as": "<slug>", instead of repeating bindings/with. It is sugar, not a new mechanism — nothing is stored beyond that derive_agent call landing in the session’s own history, folded back into a table the next time it is needed. An unknown as name tells the model to call derive_agent first, and narrowable: false refuses derive_agent and as the same way it refuses an inline bindings/with.

Design notes

A handful of decisions shape all of the above, in case they explain a surprise:

The author decides what is bindable, not the caller. Otherwise any client of agents-server could pin any argument of any tool.
The schema is static, except for "expose": a bound argument is hidden whether or not a value is currently present, so the tool list stays identical across sessions — simpler prompt caching, simpler reasoning about what the model can do.
No templating. A parameter never appears inside a system prompt or a path string; it only ever fills a tool argument. Templating changes the agent’s definition per session and puts values in front of the model, which is a different feature with different risks.
Values live in the call context, never in the registration. A toolbox loads once and is shared across every session that uses it (hot-reload, one MCP connection, one SQLite handle); rebuilding it per session would mean re-describing every script and restarting every MCP server.

See todos/tool-partial-application.md in the repository for the full design rationale and the phased implementation history, if you want the “why” behind any of the above in more depth.

Session Management

2026-10-04T15:53:54Z

Generated from documentation/sessions.md, the repository is the canonical source and may be ahead of this page.

Session Management

Session management provides persistent storage and retrieval of agent conversations, enabling conversation resumption, history analysis, and multi-modal content support. Sessions can now be stored across multiple locations with a unified read view.

Overview

Sessions represent complete conversations between users and agents, including:

Session metadata: IDs, timestamps, agent references, version info
Turn history: Complete conversation turns
Tool calls: Records of tool invocations and results
Media attachments: Base64-encoded images, documents, audio, video
Context: Full conversation state for resumption

The SessionStore now supports multi-location storage:

Write location: Single directory where new sessions are written
Read locations: Multiple directories searched for existing sessions
Deduplication: First location wins when sessions exist in multiple places
Tilde expansion: Paths like ~/.config/agents-exe/sessions/ are resolved automatically

┌─────────────────────────────────────────────────────────────┐
│                       Session                                │
├─────────────────────────────────────────────────────────────┤
│  SessionId: "uuid"                                          │
│  ConversationId: "uuid"                                     │
│  AgentSlug: "my-agent"                                      │
│  SessionVersion: 1                                          │
├─────────────────────────────────────────────────────────────┤
│  Turns:                                                      │
│  ┌─────────────────────────────────────────────────────┐   │
│  │ Turn 1                                              │   │
│  │   User: "Hello!"                                    │   │
│  │   [📎 image.png]                                    │   │
│  │   Assistant: "Hi there!"                            │   │
│  └─────────────────────────────────────────────────────┘   │
│  ┌─────────────────────────────────────────────────────┐   │
│  │ Turn 2                                              │   │
│  │   User: "List files"                                │   │
│  │   Tool: list_files -> ["a.txt", "b.txt"]            │   │
│  │   [📎 screenshot.png]                               │   │
│  │   Assistant: "Found 2 files..."                     │   │
│  └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                  Multi-Location SessionStore                 │
├─────────────────────────────────────────────────────────────┤
│  Write Location: ./sessions/                                 │
│  Read Locations:                                             │
│    1. ./sessions/           (primary, write location)       │
│    2. ./.agents-sessions/   (project-local archive)         │
│    3. ~/.config/agents-exe/sessions/  (global storage)      │
├─────────────────────────────────────────────────────────────┤
│  Deduplication: First location wins by ConversationId       │
│  Priority: ./sessions/ > ./.agents-sessions/ > ~/.config/   │
└─────────────────────────────────────────────────────────────┘

Core Types

Session Types (`System.Agents.Session.Types`)

newtype SessionId = SessionId UUID
newtype TurnId = TurnId UUID

-- | User query - can include text and media attachments.
data UserQuery = UserQuery
    { queryText :: Text
    -- ^ The text query/prompt
    , queryMedia :: [MediaAttachment]
    -- ^ Optional media attachments (images, audio, video, etc.)
    }

-- | Tool response from the user/agent system to the LLM.
data UserToolResponse
    = TextResponse Text                    -- Plain UTF-8 text
    | JsonResponse Aeson.Value             -- Structured JSON data
    | MediaResponse MediaAttachment        -- Single binary media (base64-encoded)
    | MixedResponse [ContentPart]          -- Multi-modal: alternating text and media

-- | Individual part of a mixed multi-modal response.
data ContentPart
    = TextPart Text
    | MediaPart MediaAttachment

-- | A media attachment containing base64-encoded binary data.
data MediaAttachment = MediaAttachment
    { mediaMimeType :: Text        -- ^ MIME type (e.g., "image/png")
    , mediaBase64Data :: Text      -- ^ Base64-encoded content
    , mediaFilename :: Maybe Text  -- ^ Optional filename
    }

-- | Session with versioning for backwards compatibility.
data Session = Session
    { sessionId :: SessionId
    , forkedFromSessionId :: Maybe SessionId  -- ^ For conversation forking
    , turnId :: TurnId
    , turns :: [Turn]
    , sessionVersion :: Maybe Int
    -- ^ Optional session version:
    --   Nothing = legacy (pre-media)
    --   Just 1 = media support enabled
    }

-- | User turn content including possible media.
data UserTurnContent = UserTurnContent
    { userPrompt :: SystemPrompt
    , userTools :: [SystemTool]
    , userQuery :: Maybe UserQuery
    -- ^ Query with optional media attachments
    , userToolResponses :: [(LlmToolCall, UserToolResponse)]
    -- ^ Tool responses with multi-modal support
    }

data Turn
    = UserTurn UserTurnContent (Maybe StepByteUsage)
    | LlmTurn LlmTurnContent (Maybe StepByteUsage)

Media Types (`System.Agents.Media.Types`)

-- | High-level media type classification.
data MediaType
    = MediaImage ImageType
    | MediaAudio AudioType
    | MediaVideo VideoType
    | MediaApplication ApplicationType
    | MediaText TextSubtype

data ImageType = ImagePNG | ImageJPEG | ImageGIF | ImageWebP | ImageSVG
data AudioType = AudioMPEG | AudioWAV | AudioOGG | AudioMP3 | AudioAAC | AudioFLAC
data VideoType = VideoMP4 | VideoWebM | VideoOGG | VideoAVI | VideoMOV
data ApplicationType = AppPDF | AppJSON | AppXML | AppOctetStream | AppZip
data TextSubtype = TextPlain | TextHTML | TextCSS | TextCSV | TextMarkdown

Session Versioning

Sessions include a version field for backwards compatibility:

Version	Description
`Nothing` / missing	Legacy session (pre-media support)
`Just 1`	Full media support enabled

Backwards Compatibility

-- JSON serialization handles both old and new formats
instance FromJSON Session where
    parseJSON = Aeson.withObject "Session" $ \v -> do
        mVersion <- v .:? "sessionVersion"
        Session
            <$> v .: "turns"
            <*> v .: "sessionId"
            <*> v .:? "forkedFromSessionId"
            <*> v .: "turnId"
            <*> pure mVersion

instance ToJSON Session where
    toJSON s = Aeson.object $
        [ "turns" .= s.turns
        , "sessionId" .= s.sessionId
        , "forkedFromSessionId" .= s.forkedFromSessionId
        , "turnId" .= s.turnId
        ]
        ++ ["sessionVersion" .= v | Just v <- [s.sessionVersion]]

Media Attachments

Supported Media Types

Category	Extensions	MIME Types
Images	`.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.svg`	`image/png`, `image/jpeg`, `image/gif`, `image/webp`, `image/svg+xml`
Documents	`.pdf`, `.txt`, `.md`, `.json`, `.xml`	`application/pdf`, `text/plain`, `text/markdown`, `application/json`, `application/xml`
Audio	`.mp3`, `.wav`, `.ogg`, `.aac`, `.flac`	`audio/mp3`, `audio/wav`, `audio/ogg`, `audio/aac`, `audio/flac`
Video	`.mp4`, `.webm`, `.mov`, `.avi`	`video/mp4`, `video/webm`, `video/quicktime`, `video/avi`

Creating Media Attachments

import qualified Data.ByteString.Base64 as B64
import qualified Data.Text.Encoding as Text

-- Create a media attachment from file
createMediaAttachment :: FilePath -> IO (Either String MediaAttachment)
createMediaAttachment path = do
    content <- BS.readFile path
    let mimeType = detectMimeType path
    let base64Data = Text.decodeUtf8 $ B64.encode content
    return $ Right $ MediaAttachment
        { mediaMimeType = mimeType
        , mediaBase64Data = base64Data
        , mediaFilename = Just $ Text.pack $ takeFileName path
        }

-- Detect MIME type from file extension
detectMimeType :: FilePath -> Text
detectMimeType path = case takeExtension path of
    ".png" -> "image/png"
    ".jpg" -> "image/jpeg"
    ".jpeg" -> "image/jpeg"
    ".pdf" -> "application/pdf"
    -- ... etc

Using Media in Queries

-- Create a query with media attachments
let query = UserQuery
        { queryText = "Describe this image"
        , queryMedia = [imageAttachment, documentAttachment]
        }

-- The query is sent to the LLM with media included

Tool Responses with Media

Tools can return media responses:

-- Tool returns an image
let toolResponse = MediaResponse $ MediaAttachment
        { mediaMimeType = "image/png"
        , mediaBase64Data = base64EncodedImageData
        , mediaFilename = Just "chart.png"
        }

-- Tool returns mixed text and media
let toolResponse = MixedResponse
        [ TextPart "Here's the analysis chart:"
        , MediaPart chartAttachment
        , TextPart "As you can see, the trend is increasing."
        ]

Session Store

The SessionStore module provides persistent storage with multi-location support:

data SessionStore = SessionStore
    { sessionWritePrefix :: FilePath
    -- ^ Directory where new sessions are written
    , sessionReadPrefixes :: [FilePath]
    -- ^ Directories to search for existing sessions, in priority order
    }

Creating a SessionStore

import System.Agents.SessionStore

-- Create with separate write and read locations
store <- mkSessionStore
    "./sessions/"                              -- Write location
    ["./sessions/", "~/.config/agents-exe/sessions/"]  -- Read locations

-- Create simple single-location store (backwards compatible)
simpleStore <- mkSimpleSessionStore "./sessions/"

-- Resolve tilde paths automatically
resolvedPath <- resolveSessionPath "~/.config/agents-exe/sessions/"
-- Returns: "/home/user/.config/agents-exe/sessions/"

Key Behaviors

Write Location: All new sessions are written to sessionWritePrefix
Read Locations: When reading sessions, all sessionReadPrefixes are searched in order
Auto-prepending: If write location is not in read locations, it’s automatically prepended to ensure newly written sessions are immediately readable
Deduplication: When listing sessions, if the same session ID exists in multiple locations, the first occurrence (highest priority) is kept
Tilde Expansion: Paths starting with ~ are automatically expanded to the user’s home directory

Configuration

Configure multi-location session storage in agents-exe.cfg.json:

{
  "agentsConfigDir": "~/.config/agents-exe",
  "agentsFiles": [],
  "sessions": {
    "writeLocation": "./sessions/",
    "readLocations": [
      "./sessions/",
      "./.agents-sessions/",
      "./task-sessions/",
      "~/.config/agents-exe/sessions/"
    ]
  }
}

Configuration Fields:

Field	Type	Required	Description
`sessions.writeLocation`	`FilePath`	Yes	Directory where new sessions are written
`sessions.readLocations`	`[FilePath]`	Yes	Directories to search for existing sessions

Backwards Compatibility:

If sessions section is missing, falls back to agentsLogs.logSessionsJsonPrefix (deprecated)
If no prefix is configured, uses ~/.config/agents-exe/sessions/ as default

Use Cases

Personal + Project Sessions

{
  "sessions": {
    "writeLocation": "./sessions/",
    "readLocations": [
      "./sessions/",
      "~/.config/agents-exe/sessions/"
    ]
  }
}

New sessions go to ./sessions/
Can access personal history from home directory
Project sessions take precedence if ID conflicts

Task-Based Organization

{
  "sessions": {
    "writeLocation": "./task-sessions/current/",
    "readLocations": [
      "./task-sessions/current/",
      "./task-sessions/archive/",
      "./sessions/"
    ]
  }
}

Current task sessions in current/
Archived tasks still searchable
General project sessions as fallback

Team Shared Sessions

{
  "sessions": {
    "writeLocation": "./my-sessions/",
    "readLocations": [
      "./my-sessions/",
      "./shared-sessions/",
      "~/.config/agents-exe/sessions/"
    ]
  }
}

Personal work in my-sessions/
Team-shared sessions in shared-sessions/
Personal history available

File Naming

Sessions are stored with the pattern conv.<uuid>.json:

sessions/
├── conv.550e8400-e29b-41d4-a716-446655440000.json
├── conv.6ba7b810-9dad-11d1-80b4-00c04fd430c8.json
└── conv.6ba7b811-9dad-11d1-80b4-00c04fd430c8.json

Session Store Operations

-- Store a session (always writes to sessionWritePrefix)
storeSession :: SessionStore -> ConversationId -> Session -> IO ()

-- Read a session (searches all read locations in order)
readSession :: SessionStore -> ConversationId -> IO (Maybe Session)

-- List all sessions (aggregates from all read locations, deduplicated)
listSessions :: SessionStore -> IO [(FilePath, Maybe Session, ConversationId)]

-- Generate file path for a session
sessionFilePath :: SessionStore -> ConversationId -> FilePath
sessionWritePath :: SessionStore -> ConversationId -> FilePath

Session Lifecycle

Creating a Session

newSession :: AgentSlug -> IO Session
newSession slug = do
    sid <- SessionId <$> UUID.nextRandom
    cid <- ConversationId <$> UUID.nextRandom
    tid <- TurnId <$> UUID.nextRandom
    return Session
        { sessionId = sid
        , forkedFromSessionId = Nothing
        , turnId = tid
        , turns = []
        , sessionVersion = Just 1  -- New sessions use version 1
        }

Adding a Turn

addUserTurn :: Session -> UserQuery -> [ToolCallRecord] -> IO Session
addUserTurn session query calls = do
    let userContent = UserTurnContent
            { userPrompt = currentPrompt
            , userTools = availableTools
            , userQuery = Just query
            , userToolResponses = calls
            }
    let turn = UserTurn userContent Nothing
    return session { turns = turns session ++ [turn] }

Conversation Loop

conversationLoop :: 
    SessionStore -> 
    Runtime -> 
    Session -> 
    IO ()
conversationLoop store runtime session = do
    -- Get user input (possibly with media)
    (input, media) <- getUserInput
    let query = UserQuery input media
    
    -- Call LLM with session context and media
    let messages = sessionToMessages session
    response <- callLLM runtime messages query
    
    -- Execute any tool calls
    toolResults <- executeToolCalls runtime (toolCalls response)
    
    -- Update session
    newSession <- addTurn session query toolResults
    
    -- Persist (always writes to write location)
    storeSession store (conversationId newSession) newSession
    
    -- Continue
    conversationLoop store runtime newSession

Session Serialization

JSON Format

{
  "sessionId": "550e8400-e29b-41d4-a716-446655440000",
  "turnId": "6ba7b810-9dad-11d1-80b4-00c04fd430c8",
  "forkedFromSessionId": null,
  "turns": [
    {
      "tag": "UserTurn",
      "contents": {
        "userPrompt": {"systemPrompt": "You are helpful..."},
        "userTools": [...],
        "userQuery": {
          "text": "What's in this image?",
          "media": [
            {
              "mimeType": "image/png",
              "base64Data": "iVBORw0KGgoAAAANSUhEUgAA...",
              "filename": "screenshot.png"
            }
          ]
        },
        "userToolResponses": []
      },
      "byteUsage": {
        "inputBytes": 1234,
        "outputBytes": 567,
        "reasoningBytes": 0,
        "totalBytes": 1801
      }
    },
    {
      "tag": "LlmTurn",
      "contents": {
        "llmResponse": "I can see a chart showing...",
        "llmThinking": null,
        "llmToolCalls": []
      }
    }
  ],
  "sessionVersion": 1
}

UserToolResponse JSON Formats

// TextResponse
{
  "type": "text",
  "content": "The file contains..."
}

// JsonResponse
{
  "type": "json",
  "content": {"status": "ok", "count": 42}
}

// MediaResponse
{
  "type": "media",
  "mimeType": "image/png",
  "base64Data": "iVBORw0KGgoAAAANSUhEUgAA...",
  "filename": "chart.png"
}

// MixedResponse
{
  "type": "mixed",
  "parts": [
    {"type": "text", "text": "Here's the chart:"},
    {"type": "media", "mimeType": "image/png", "base64Data": "..."},
    {"type": "text", "text": "As you can see..."}
  ]
}

Session Operations

Resuming a Session

# Start with existing session
agents-exe run --agent-file agent.json --session-file session.json

mainOneShot :: 
    SessionStore ->
    Maybe FilePath ->  -- Session file path
    Maybe Session ->   -- Pre-loaded session
    Props -> 
    Text ->            -- Prompt
    [MediaAttachment] -> -- Media attachments
    IO ()
mainOneShot store mPath mSession props prompt media = do
    -- Load or create session
    session <- case mSession of
        Just s -> return s
        Nothing -> case mPath of
            Just path -> fromMaybe (newSession props.agentSlug) 
                                   <$> readSessionFromFile path
            Nothing -> newSession props.agentSlug
    
    -- Create query with media
    let query = UserQuery prompt media
    
    -- Run conversation
    ...
    
    -- Save session (to write location)
    writeSession store updatedSession

Session Printing (`System.Agents.SessionPrint`)

The SessionPrint module provides rich markdown formatting for session files, including statistics visualization and configurable content display with media support.

Session Print Types

-- | Preference for ordering session steps.
data OrderPreference
    = Chronological      -- Oldest first
    | Antichronological  -- Newest first

-- | Amount of content to print (lines or characters).
data PrintAmount
    = Lines Int
    | Chars Int

-- | Visibility preference for displaying content.
data PrintVisibility
    = Hidden                    -- Don't show content
    | Elided PrintAmount PrintAmount  -- Show leading/trailing, elide middle
    | ShownFull                 -- Show complete content

-- | Options for controlling session print output.
data SessionPrintOptions = SessionPrintOptions
    { sessionPrintFile :: FilePath
    , showToolCallResults :: PrintVisibility
    , showToolCallArguments :: PrintVisibility
    , nTurns :: Maybe Int
    , repeatSystemPrompt :: Bool
    , repeatTools :: Bool
    , orderPreference :: OrderPreference
    , noFunnyStamp :: Bool
    }

-- | Statistics about a session.
data SessionStatistics = SessionStatistics
    { statTotalTurns :: Int
    , statUserTurns :: Int
    , statLlmTurns :: Int
    , statTotalToolCalls :: Int
    , statToolCallsByName :: Map Text Int
    , statInputBytes :: Int
    , statOutputBytes :: Int
    , statReasoningBytes :: Int
    , statTotalBytes :: Int
    , statMediaAttachments :: Int  -- NEW: Count of media attachments
    }

CLI: session-print Command

# Print full session
agents-exe session-print session.json

# Print with tool call results visible
agents-exe session-print --show-tool-call-results shown session.json

# Show tool call arguments too
agents-exe session-print \
    --show-tool-call-results shown \
    --show-tool-call-arguments shown \
    session.json

# Elide long outputs (show first/last 10 lines)
agents-exe session-print --show-tool-call-results elided session.json

# Limit to N turns
agents-exe session-print --n-turns 5 session.json

# Reverse chronological order
agents-exe session-print --antichronological session.json

# Show system prompts and tools each turn
agents-exe session-print --repeat-system-prompt --repeat-tools session.json

# Skip the ASCII art logo
agents-exe session-print --no-funny-stamp session.json

Formatting Media Attachments

formatToolResponse :: UserToolResponse -> Text
formatToolResponse (TextResponse txt) = txt
formatToolResponse (JsonResponse val) = 
    Text.decodeUtf8 $ LByteString.toStrict $ Aeson.encodePretty val
formatToolResponse (MediaResponse media) = 
    "[Media: " <> media.mediaMimeType <> 
    maybe "" (", " <>) media.mediaFilename <> 
    ", " <> formatBytes (Text.length media.mediaBase64Data `div` 4 * 3) <> "]"
formatToolResponse (MixedResponse parts) = 
    Text.intercalate "\n" $ map formatContentPart parts
  where
    formatContentPart (TextPart t) = t
    formatContentPart (MediaPart m) = formatToolResponse (MediaResponse m)

Content Elision

The elideDocument function intelligently handles content that’s too long:

-- | Elide a document by keeping leading and trailing portions.
elideDocument :: PrintAmount -> PrintAmount -> Text -> Text

-- Examples:
elideDocument (Lines 3) (Lines 3) "line1\nline2\n...\nline7"
-- Shows all 7 lines (no overlap)

elideDocument (Lines 2) (Lines 2) "line1\nline2\nline3\nline4\nline5"
-- "line1\nline2\n... (1 line elided) ...\nline4\nline5"

Statistics Visualization

Session print includes visual bar charts for:

Tool usage: Bar chart showing which tools were called most
Byte usage: Input, output, and reasoning token breakdown
Media attachments: Count and types of media

📊 Statistics

### 🔧 Tool Calls

Total Tool Calls: 15

`read-file`         8   ████████████████████████████████████████
`write-file`        4   ████████████████████
`grep-files`        3   ███████████████

### 💾 Byte Usage

`Input    `      2 KiB   ████████████████████████
`Output   `      5 KiB   ████████████████████████████████████████████████
`Reasoning`      1 KiB   ████████████

`Media    `     50 KiB   ████████████████████████████████████████████████████████████████████████████████████████████████████

Total: 58 KiB

Session Content Injection (`System.Agents.SessionPrint.Inject`)

The SessionInject module allows injecting session content into prompts with various verbosity levels.

Injection Verbosity Levels

data SessionInjectMode
    = SessionXS   -- Minimal: queries/responses only, skips tool-only turns
    | SessionS    -- Low: +thinking, +tool names
    | SessionM    -- Medium: +statistics
    | SessionL    -- High: +tool call results
    | SessionXL   -- Maximum: complete session

CLI: Session Injection Options

# Inject session at minimal verbosity
agents-exe run --session-xs previous-session.json --prompt "Continue..."

# Inject at low verbosity
agents-exe run --session-s previous-session.json --prompt "Continue..."

# Inject at medium verbosity
agents-exe run --session-m previous-session.json --prompt "Continue..."

# Inject at high verbosity (includes tool results)
agents-exe run --session-l previous-session.json --prompt "Continue..."

# Maximum verbosity
agents-exe run --session-xl previous-session.json --prompt "Continue..."

Programmatic Usage

import System.Agents.SessionPrint.Inject

-- Load session content at specific verbosity
loadSessionForPrompt :: SessionInjectMode -> Session -> Text
loadSessionForPrompt mode session = case mode of
    SessionXS -> formatMinimal session   -- Just user queries and LLM responses
    SessionS  -> formatLow session       -- + thinking process
    SessionM  -> formatMedium session    -- + statistics
    SessionL  -> formatHigh session      -- + tool call results
    SessionXL -> formatComplete session  -- Everything

Session Edit

The SessionEdit module provides operations for modifying session files.

CLI: session-edit Command

# Take first N turns
agents-exe session-edit --take --count 10 session.json < input.json > output.json

# Take last N turns
agents-exe session-edit --take-tail --count 5 session.json < input.json > output.json

# Drop first N turns
agents-exe session-edit --drop --count 2 session.json < input.json > output.json

# Drop last N turns
agents-exe session-edit --drop-tail --count 1 session.json < input.json > output.json

# Remove all tool calls
agents-exe session-edit --censor-tool-calls session.json < input.json > output.json

# Remove thinking content
agents-exe session-edit --censor-thinking session.json < input.json > output.json

Edit Operations

data SessionEditOp
    = SessionEditTake Int       -- Take first N turns
    | SessionEditTakeTail Int   -- Take last N turns
    | SessionEditDrop Int       -- Drop first N turns
    | SessionEditDropTail Int   -- Drop last N turns
    | SessionEditCensorToolCalls  -- Remove tool calls
    | SessionEditCensorThinking   -- Remove thinking content

applyEdit :: SessionEditOp -> Session -> Session
applyEdit (SessionEditTake n) session = 
    session { turns = take n (turns session) }
applyEdit (SessionEditDrop n) session = 
    session { turns = drop n (turns session) }
applyEdit SessionEditCensorToolCalls session =
    session { turns = map removeToolCalls (turns session) }
-- etc.

Session Search (`System.Agents.Session.Search`)

The Session Search subsystem provides fast fuzzy text search across session files using SQLite FTS5. It enables searching through conversation history with trigram-based fuzzy matching and metadata filtering.

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Session Search                            │
├─────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐         │
│  │    Types    │  │    Index    │  │    Query    │         │
│  │  (config)   │  │  (SQLite)   │  │  (search)   │         │
│  └─────────────┘  └─────────────┘  └─────────────┘         │
│        ▲                ▲                ▲                  │
│        └────────────────┴────────────────┘                  │
│                    CLI Handlers                              │
│              (session-index, session-search)                 │
└─────────────────────────────────────────────────────────────┘

Core Types (`System.Agents.Session.Search.Types`)

-- | Configuration for the search index.
data SearchIndexConfig = SearchIndexConfig
    { indexDbPath :: FilePath
    -- ^ Path to the SQLite index database (default: .agents-search.db)
    , indexSessionStore :: SessionStore
    -- ^ Session store to index (supports multi-location stores)
    , indexIncludeToolOutputs :: Bool
    -- ^ Whether to include tool outputs in the index
    }

-- | Options for session search queries.
data SearchOptions = SearchOptions
    { searchQuery :: Text
    -- ^ Fuzzy search query text
    , searchDateFilter :: Maybe DateFilter
    -- ^ Optional date filter
    , searchTools :: [Text]
    -- ^ Filter by tools used (any of these)
    , searchAgent :: Maybe Text
    -- ^ Filter by agent slug
    , searchIncludeToolOutputs :: Bool
    -- ^ Include tool outputs in search
    , searchJsonOutput :: Bool
    -- ^ Output results as JSON
    , searchPreviewLines :: Int
    -- ^ Number of context lines to show
    , searchLimit :: Maybe Int
    -- ^ Maximum number of results
    , searchAutoUpdate :: Bool
    -- ^ Auto-update index if stale before searching
    }

-- | Date filter for search queries.
data DateFilter
    = AfterDate UTCTime
    | BeforeDate UTCTime
    | BetweenDates UTCTime UTCTime

-- | Status of the search index.
data SearchIndexStatus
    = IndexCurrent
    | IndexStale Int Int  -- (stale sessions, total sessions)
    | IndexMissing
    | IndexError Text

Search Result Types

-- | Complete search results.
data SearchResult = SearchResult
    { resultItems :: [SearchResultItem]
    , resultTotalMatches :: Int
    , resultQueryTimeMs :: Double
    , resultIndexWasUpdated :: Bool
    }

-- | A single search result item.
data SearchResultItem = SearchResultItem
    { resultMetadata :: SearchResultMetadata
    , resultPreview :: Maybe Text
    , resultMatchedTerms :: [Text]
    }

-- | Metadata about a search result.
data SearchResultMetadata = SearchResultMetadata
    { resultSessionId :: Text
    , resultFilePath :: FilePath
    , resultAgentSlug :: Maybe Text
    , resultTurnCount :: Int
    , resultFirstTurnAt :: Maybe UTCTime
    , resultLastTurnAt :: Maybe UTCTime
    , resultRank :: Double
    }

Index Schema (`System.Agents.Session.Search.Index`)

The search index uses SQLite with FTS5 (Full-Text Search version 5):

-- Session metadata cache
CREATE TABLE session_index (
    session_id TEXT PRIMARY KEY,
    file_path TEXT NOT NULL,
    mtime INTEGER NOT NULL,
    agent_slug TEXT,
    turn_count INTEGER,
    first_turn_at TIMESTAMP,
    last_turn_at TIMESTAMP
);

-- FTS5 virtual table for trigram search
CREATE VIRTUAL TABLE search_content USING fts5(
    session_id,
    content,
    tokenize='trigram'
);

-- Tool call index for filtering
CREATE TABLE tool_index (
    session_id TEXT REFERENCES session_index(session_id),
    tool_name TEXT NOT NULL,
    call_count INTEGER DEFAULT 1,
    PRIMARY KEY (session_id, tool_name)
);

-- Index metadata
CREATE TABLE index_metadata (
    key TEXT PRIMARY KEY,
    value TEXT
);

Index Operations

-- | Create a new search index, replacing any existing index.
-- Indexes sessions from ALL read locations in the SessionStore.
createSearchIndex :: SearchIndexConfig -> IO ()

-- | Incrementally update the search index.
updateSearchIndex :: SearchIndexConfig -> IO ()

-- | Check the status of the search index.
checkIndexStatus :: SearchIndexConfig -> IO SearchIndexStatus

-- | Remove the search index.
removeSearchIndex :: SearchIndexConfig -> IO ()

Search Execution (`System.Agents.Session.Search.Query`)

-- | Execute a search query with the given options.
-- Searches across all indexed sessions from all locations.
executeSearchWithOptions :: SearchIndexConfig -> SearchOptions -> IO SearchResult

-- | Execute a search query against the index.
executeSearch :: SearchIndexConfig -> SearchOptions -> IO SearchResult

-- | Format search results for display.
formatResults :: SearchOptions -> SearchResult -> IO ()

Fuzzy Matching

The search uses SQLite FTS5 with trigram tokenization:

Trigram tokenization: Breaks text into 3-character sequences
Fuzzy matching: “error” matches “errors”, “erroring”, “terror”
Prefix matching: “data” matches “database”, “datagram”
Typo tolerance: Small changes in query still find matches

Example:

-- Find sessions with fuzzy match to "error"
SELECT session_id, rank 
FROM search_content 
WHERE content MATCH 'error'
ORDER BY rank;

CLI: session-index Command

# Build the search index (includes all read locations)
agents-exe session-index --build

# Check index status
agents-exe session-index --status

# Update index incrementally
agents-exe session-index --update

# Remove the index
agents-exe session-index --clean

# Build with tool outputs included
agents-exe session-index --build --include-tool-outputs

CLI: session-search Command

# Basic fuzzy search
agents-exe session-search "database error"

# Search with auto-update
agents-exe session-search "migration" --auto

# Include tool outputs in search
agents-exe session-search "config.yaml" --include-tool-outputs

# Filter by date and tool
agents-exe session-search "auth" --after 2024-01-01 --tool write-file

# Filter by agent
agents-exe session-search "refactor" --agent my-coder

# JSON output for scripting
agents-exe session-search "TODO" --json

# Show context lines
agents-exe session-search "deploy" --preview 5

# Combined filters
agents-exe session-search "fix" \
  --after 2024-01-01 \
  --before 2024-12-31 \
  --tool bash_write-file \
  --json

Search Workflow Example

# 1. Build the initial index (aggregates all locations)
agents-exe session-index --build

# 2. Search for sessions about errors
agents-exe session-search "error" --json | jq '.resultItems[] | .resultMetadata.resultFilePath'

# 3. Find sessions using specific tools
agents-exe session-search "database" --tool write-file --tool read-file

# 4. Update index after new sessions
agents-exe session-index --update

# 5. Search with date filter
agents-exe session-search "config" --after 2024-06-01 --preview 3

Context Window Management

Sessions track context size for LLM limits:

estimateTokens :: Session -> Int
estimateTokens session = 
    sum [estimateMessageTokens m | turn <- turns session
                                 , m <- [userMessage turn, assistantMessage turn]]

manageContext :: Int -> Session -> Session
manageContext maxTokens session
    | estimateTokens session <= maxTokens = session
    | otherwise = manageContext maxTokens (pruneOldestTurn session)

Byte Usage Tracking

Sessions track byte usage for monitoring:

-- | Calculate bytes for all response types
userToolResponseBytes :: UserToolResponse -> Int
userToolResponseBytes (TextResponse txt) = 
    Text.length txt * 4  -- UTF-8 max bytes per char
userToolResponseBytes (JsonResponse val) = 
    fromIntegral $ LByteString.length $ Aeson.encode val
userToolResponseBytes (MediaResponse media) = 
    Text.length media.mediaBase64Data  -- Already base64 = ASCII
userToolResponseBytes (MixedResponse parts) = 
    sum (map contentPartBytes parts)

contentPartBytes :: ContentPart -> Int
contentPartBytes (TextPart txt) = Text.length txt * 4
contentPartBytes (MediaPart media) = Text.length media.mediaBase64Data

Tool Execution Context

Sessions provide context for tool execution:

data ToolExecutionContext = ToolExecutionContext
    { ctxSessionId :: SessionId
    , ctxConversationId :: ConversationId
    , ctxTurnId :: TurnId
    , ctxFullSession :: Maybe Session
    , ...
    }

This allows tools to:

Access conversation history
Store tool-specific state
Make context-aware decisions
Return media responses

Session Store Backends

File Store (Default)

fileSessionStore :: FilePath -> [FilePath] -> SessionStore
fileSessionStore writePrefix readPrefixes = SessionStore
    { sessionWritePrefix = writePrefix
    , sessionReadPrefixes = 
        if writePrefix `elem` readPrefixes
            then readPrefixes
            else writePrefix : readPrefixes
    , readSession = \sid -> do
        -- Search through read prefixes in order
        let paths = map (</> sessionFileName sid) sessionReadPrefixes
        findFirstExisting paths
    , writeSession = \sess -> do
        -- Always write to write prefix
        let path = sessionWritePrefix </> sessionFileName (sessionId sess)
        encodeFile path sess
    , listSessions = do
        -- Aggregate from all read prefixes, deduplicate
        allSessions <- concat <$> mapM listSessionsInDir sessionReadPrefixes
        let deduplicated = dedupeBy sessionId allSessions
        sortOn modificationTime deduplicated
    , ...
    }

Memory Store (Testing)

memorySessionStore :: IO SessionStore
memorySessionStore = do
    ref <- newIORef Map.empty
    return SessionStore
        { readSession = \sid -> Map.lookup sid <$> readIORef ref
        , writeSession = \sess -> 
            modifyIORef ref (Map.insert (sessionId sess) sess)
        , ...
        }

Best Practices

Session Hygiene

Regular cleanup: Delete old sessions to save space
Sensitive data: Be careful with sessions containing secrets
Backup: Sessions are JSON files - back them up
Versioning: Handle schema migrations for old sessions

Conversation Design

Clear turn boundaries: Each user input = one turn
Tool call recording: Always record for reproducibility
Error capture: Record tool failures in the session
Timestamps: Useful for debugging and auditing

Performance

Lazy loading: Don’t load full history unless needed
Pagination: For long sessions, load turns in chunks
Compression: Consider gzip for large session files
Search indexing: Build index periodically for fast searching

Search Best Practices

Regular index updates: Run session-index --update periodically
Include tool outputs: Use --include-tool-outputs for comprehensive search
Use filters: Combine text search with --tool, --agent, or date filters
JSON output: Use --json for scripting and automation
Auto-update: Use --auto flag to ensure fresh results

Multi-Location Best Practices

Write location priority: Keep write location first in read locations for best performance
Archive old sessions: Move old sessions to archive directories instead of deleting
Shared sessions: Use shared directories for team collaboration
Tilde expansion: Use ~ for portable home directory references
Deduplication awareness: Understand that first location wins for duplicate IDs

Media Best Practices

Size limits: Keep media attachments under 50MB
MIME types: Always declare correct MIME types
Base64 encoding: Use proper encoding for binary data
Mixed responses: Use MixedResponse for rich multi-modal output

Example: Session Analysis

-- Count tool usage
toolUsageStats :: Session -> Map Text Int
toolUsageStats session = 
    Map.fromListWith (+) 
        [(toolName tc, 1) | turn <- turns session
                          , tc <- toolCalls turn]

-- Average response time
averageResponseTime :: Session -> NominalDiffTime
averageResponseTime session =
    sum [toolDuration tc | turn <- turns session
                         , tc <- toolCalls turn] 
    / fromIntegral (length $ concatMap toolCalls $ turns session)

-- Find turns with errors
errorTurns :: Session -> [Turn]
errorTurns session =
    [turn | turn <- turns session
          , any (isLeft . toolOutput) (toolCalls turn)]

-- Count media attachments
mediaAttachmentStats :: Session -> (Int, Map Text Int)
mediaAttachmentStats session =
    let attachments = collectAttachments session
        byType = Map.fromListWith (+) 
            [(mediaMimeType m, 1) | m <- attachments]
    in (length attachments, byType)

Example: Search Integration

import System.Agents.Session.Search.Types
import System.Agents.Session.Search.Index
import System.Agents.Session.Search.Query

-- Search for sessions with specific patterns
searchSessions :: Text -> IO SearchResult
searchSessions query = do
    -- Create store with multi-location support
    store <- mkSessionStore 
        "./sessions/"
        ["./sessions/", "~/.config/agents-exe/sessions/"]
    
    let config = defaultSearchIndexConfig store
    let opts = (defaultSearchOptions query)
            { searchJsonOutput = False
            , searchPreviewLines = 3
            , searchLimit = Just 20
            }
    executeSearchWithOptions config opts

-- Find sessions that used specific tools
findToolUsage :: [Text] -> IO SearchResult
findToolUsage tools = do
    store <- mkSimpleSessionStore "./sessions/"
    let config = defaultSearchIndexConfig store
    let opts = (defaultSearchOptions "")
            { searchTools = tools
            , searchJsonOutput = True
            }
    executeSearchWithOptions config opts

Module	Purpose
`System.Agents.Media.Types`	Media type definitions
`System.Agents.Session.Types`	Core session types
`System.Agents.Session.Base`	Session operations
`System.Agents.Session.Loop`	Conversation loop
`System.Agents.Session.Step`	Single turn execution
`System.Agents.Session.Edit`	Session editing operations
`System.Agents.Session.OpenAI`	OpenAI-specific session handling
`System.Agents.Session.Compat`	Backwards compatibility
`System.Agents.Session.Search.Types`	Search configuration types
`System.Agents.Session.Search.Index`	Index building and maintenance
`System.Agents.Session.Search.Query`	Search query execution
`System.Agents.SessionStore`	Persistent storage with multi-location support
`System.Agents.SessionPrint`	Markdown printing and statistics
`System.Agents.SessionPrint.Inject`	Session content injection

Command reference

2026-10-04T15:53:54Z

Generated from README.md, the repository is the canonical source and may be ahead of this page.

Command reference

The exact help text of each command, as printed by the binary this site was produced with: agents-exe –help, check, config, init, run, tui, serve, mcp-server, session, describe, new.

Command Reference

Global Options

These options apply to all commands:

Option	Default	Description
`--api-keys FILE`	`~/.config/agents-exe/secret-keys`	Path to JSON file containing API keys
`--log-file LOGFILE`	`agents-logfile`	Raw log file for debugging
`--log-http URL`	(none)	HTTP endpoint for JSON log sink
`--log-json-file FILE`	(none)	Local JSON file log sink
`--session-json-file-prefix PREFIX`	(none)	Prefix for session JSON files
`--agent-file AGENTFILE`	(auto-discovered)	Root agent description file(s). Can be specified multiple times. Defaults to `agent.json`

Commands

`check`

Validates agent configuration files.

agents-exe check
agents-exe check --agent-file custom-agent.json

`init`

Initializes a new agent project with a sample agent.json and tools directory structure.

agents-exe init
agents-exe init --agent-file my-agent.json

`tui` - Terminal User Interface

Launches an interactive terminal UI for chatting with agents.

agents-exe tui
agents-exe tui --agent-file my-agent.json

`run` - Execute a One-Shot Command

Runs a single prompt through an agent and outputs the result.

Prompt Options:

Option	Short	Description
`--prompt TEXT`	`-p`	Add a text paragraph to the prompt
`--file FILE`	`-f`	Include contents of a file in the prompt
`--shell CMD`		Include stdout of a shell command in the prompt
`--sep4 SEP`	`-s`	Add a short separator (4 repetitions)
`--sep40 SEP`	`-S`	Add a long separator (40 repetitions)
`--session-file FILE`		Resume from or save to a session file

Examples:

# Simple prompt
agents-exe run --prompt "Hello, how are you?"

# Include file contents
agents-exe run --prompt "Summarize this:" --file README.md

# Include command output
agents-exe run -p "Explain this diff:" -s "---" --shell "git diff"

# Multiple prompts (concatenated with newlines)
agents-exe run -p "First part" -p "Second part"

# Resume a session
agents-exe run --session-file previous.json -p "Continue where we left off"

`echo-prompt`

Echoes the constructed prompt without sending it to the agent. Useful for debugging prompt construction.

agents-exe echo-prompt -p "Hello" -f README.md

`describe`

Outputs a JSON self-description of the tool interface. Useful when agents-exe is used as a sub-tool.

agents-exe describe

`mcp-server`

Runs agents-exe as an MCP (Model Context Protocol) server, exposing agents as tools via stdin/stdout.

agents-exe mcp-server
agents-exe mcp-server --agent-file agent1.json --agent-file agent2.json

The MCP server exposes each configured agent as a callable tool to MCP clients like Claude Desktop.

`session-print`

Prints a session file in human-readable markdown format.

Option	Description
`--show-tool-call-results`	Show the results of tool calls
`--n-turns N`	Limit output to first N turns
`--repeat-system-prompt`	Repeat system prompt at each turn
`--repeat-tools`	Repeat available tools at each turn
`--antichronological`	Display newest turns first (default: oldest first)

Examples:

# Print entire session
agents-exe session-print my-session.json

# Print with tool results
agents-exe session-print --show-tool-call-results my-session.json

# Print last 10 turns only, newest first
agents-exe session-print --n-turns 10 --antichronological my-session.json

Architecture

2026-10-04T15:53:54Z

Generated from documentation/architecture.md, the repository is the canonical source and may be ahead of this page.

Architecture

This document describes the runtime architecture and core components of the Agents framework.

Core Architecture

The framework is built around a layered architecture that separates concerns between agent definition, runtime execution, and user interfaces.

Layer Overview

┌────────────────────────────────────────────────────────────────┐
│                    Interface Layer                              │
│  (CLI commands, TUI, MCP server, HTTP endpoints)                │
│  The TUI and the HTTP server (agents-server) are both clients   │
│  of an in-process SessionRunner (Host.Runner), driven through   │
│  the same RunnerClient interface -- see tui.md#architecture.    │
├────────────────────────────────────────────────────────────────┤
│                    OS Model Layer                               │
│  (Entity-Component-System, Resource Management,                 │
│   Conversation Tracking, Concurrent Access)                     │
├────────────────────────────────────────────────────────────────┤
│                    Agent Tree Layer                             │
│  (multi-agent hierarchy, reference validation, cycle detection) │
├────────────────────────────────────────────────────────────────┤
│                    Foundation Layer                             │
│  (sessions, tools, LLM integration, file loading)               │
└────────────────────────────────────────────────────────────────┘

OS Model Architecture

The OS Model provides a centralized, ECS-based architecture for managing agents, toolboxes, and resources.

Entity-Component-System (ECS) Pattern

┌─────────────────────────────────────────────────────────────────┐
│                         World                                    │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐             │
│  │  Component  │  │  Component  │  │  Component  │             │
│  │   Store 1   │  │   Store 2   │  │   Store N   │             │
│  │ (TVar Any)  │  │ (TVar Any)  │  │ (TVar Any)  │             │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘             │
│         │                │                │                     │
│         └────────────────┼────────────────┘                     │
│                          │                                      │
│                   HashMap ComponentTypeId                        │
└──────────────────────────┬──────────────────────────────────────┘
                           │
         ┌─────────────────┼─────────────────┐
         │                 │                 │
         ▼                 ▼                 ▼
    ┌─────────┐      ┌─────────┐      ┌─────────┐
    │ Entity  │      │ Entity  │      │ Entity  │
    │   1     │      │   2     │      │   N     │
    └────┬────┘      └────┬────┘      └────┬────┘
         │                │                │
    ┌────┴────┐      ┌────┴────┐      ┌────┴────┐
    │ Agent   │      │Toolbox  │      │ Conv    │
    │ Config  │      │ Config  │      │ Config  │
    │ Agent   │      │Toolbox  │      │ Conv    │
    │ State   │      │ State   │      │ State   │
    └─────────┘      └─────────┘      └─────────┘

Key Design Principles:

Entities are just IDs: Lightweight identifiers with phantom types for type safety
Components are pure data: Serializable, immutable data structures
Systems are functions: Operate on entities with specific component combinations
Storage is heterogeneous: Uses TVar Any for type erasure with safe casting

Component Types

Agent Components

Component	ID	Purpose
`AgentConfig`	1	Static agent configuration (name, model, system prompt)
`AgentState`	2	Runtime state (status, current conversation)

Toolbox Components

Component	ID	Purpose
`ToolboxConfig`	3	Toolbox type and settings
`ToolboxState`	4	Runtime state and resource reference
`ToolboxBinding`	5	Agent-to-toolbox relationship

Conversation Components

Component	ID	Purpose
`ConversationConfig`	30	Conversation metadata
`ConversationState`	31	Runtime status and timestamps
`AgentConversation`	32	Agent-conversation relationship
`TurnConfig`	33	Turn structure (parent, conversation)
`TurnState`	34	Turn execution state
`ToolCallConfig`	35	Tool call specification
`ToolCallState`	36	Tool call execution state
`Message`	38	Chat messages

Resource Lifecycle Flow

Program Startup
      │
      ▼
┌─────────────┐
│  Initialize │
│    World    │
└──────┬──────┘
       │
       ▼
┌─────────────┐     ┌─────────────┐
│   Create    │────>│  Register   │
│   Agents    │     │  Component  │
└──────┬──────┘     │   Stores    │
       │            └─────────────┘
       ▼
┌─────────────┐     ┌─────────────┐
│   Create    │────>│  Register   │
│  Toolboxes  │     │  Resources  │
└──────┬──────┘     └─────────────┘
       │
       ▼
┌─────────────┐
│    Bind     │
│   Agents    │
│  to Toolboxes│
└──────┬──────┘
       │
       ▼
    ┌────────┐
    │ RUNTIME │
    └────┬───┘
         │
    ┌────┴────┬──────────┬──────────┐
    │         │          │          │
    ▼         ▼          ▼          ▼
┌───────┐ ┌───────┐ ┌───────┐ ┌───────┐
│ Agent │ │ Agent │ │Shared │ │ Shared│
│   1   │ │   2   │ │SQLite │ │  HTTP │
│       │ │       │ │  DB   │ │ Pool  │
└───┬───┘ └───┬───┘ └───┬───┘ └───┬───┘
    │         │         │         │
    └─────────┴─────────┴─────────┘
              │
              ▼
       ┌─────────────┐
       │  Cleanup    │
       │   Scope     │
       │ (on destroy)│
       └─────────────┘

Resource Scopes:

Program Scope: Global resources (HTTP connection pools, shared caches)
Agent Scope: Per-agent resources (sandbox directories, agent-specific state)
Toolbox Scope: Per-toolbox resources (SQLite connections, MCP clients)
Conversation Scope: Per-conversation resources (isolated Lua states, temp files)
Turn Scope: Temporary resources (single turn execution context)
ToolCall Scope: Single-use resources (tool call arguments, results)

Concurrent Access Patterns

┌──────────────────────────────────────────────────────────────┐
│                    Access Patterns                            │
├──────────────────────────────────────────────────────────────┤
│                                                               │
│  Exclusive Access (TMVar)                                     │
│  ┌─────────┐     ┌─────────┐     ┌─────────┐                 │
│  │  Lock   │────>│ Execute │────>│ Release │                 │
│  └─────────┘     └─────────┘     └─────────┘                 │
│  Use: Lua interpreters, process handles                      │
│                                                               │
├──────────────────────────────────────────────────────────────┤
│                                                               │
│  Read-Write Access (RWLock)                                   │
│  ┌─────┐ ┌─────┐       ┌─────────┐      ┌─────┐ ┌─────┐      │
│  │Read │ │Read │──────>│  Data   │<─────│Write│     │      │
│  │  1  │ │  2  │       │         │      │     │     │      │
│  └─────┘ └─────┘       └─────────┘      └─────┘     │      │
│  Use: SQLite databases (especially WAL mode)                 │
│                                                               │
├──────────────────────────────────────────────────────────────┤
│                                                               │
│  Pool Access (TBQueue)                                        │
│  ┌─────────────────────────────────────────────────────┐     │
│  │  Pool: [Token] [Token] [Token] ... [Token]          │     │
│  └─────────────────────────────────────────────────────┘     │
│       ▲    │         ▲    │                                  │
│       │    └─────────┘    │                                  │
│    Acquire             Release                               │
│  Use: HTTP connection pools, DB connection pools             │
│                                                               │
├──────────────────────────────────────────────────────────────┤
│                                                               │
│  Stateless Access (No Lock)                                   │
│  ┌─────────┐ ┌─────────┐ ┌─────────┐                        │
│  │ Access  │ │ Access  │ │ Access  │  (Concurrent, no sync) │
│  └─────────┘ └─────────┘ └─────────┘                        │
│  Use: Immutable data, thread-safe resources                  │
│                                                               │
└──────────────────────────────────────────────────────────────┘

Conversation and Lineage Tracking

Conversation Tree Structure

┌─────────────────┐
│ Conversation 1  │
│   (Entity)      │
└────────┬────────┘
         │
    ┌────┴────┐
    ▼         ▼
┌───────┐ ┌───────┐
│ Turn 1│ │ Turn 2│
│(Entity)│ │(Fork) │
└───┬───┘ └───┬───┘
    │         │
    ▼         ▼
┌───────┐ ┌───────┐
│ Call 1│ │ Call 1│
│       │ │       │
└───┬───┘ └───┬───┘
    │         │
    ▼         ▼
┌───────┐ ┌───────┐
│ Call 2│ │ Call 2│
│(Nested)│ │(Nested)│
└───────┘ └───────┘

Lineage Stack
┌─────────────────────────────────────┐
│ LineageFrame                        │
│ ├─ frameType: ToolCallFrame         │
│ ├─ frameEntityId: <tool-call-id>    │
│ └─ frameTimestamp: <time>           │
├─────────────────────────────────────┤
│ LineageFrame                        │
│ ├─ frameType: TurnFrame             │
│ ├─ frameEntityId: <turn-id>         │
│ └─ frameTimestamp: <time>           │
├─────────────────────────────────────┤
│ LineageFrame                        │
│ ├─ frameType: ConversationFrame     │
│ ├─ frameEntityId: <conversation-id> │
│ └─ frameTimestamp: <time>           │
└─────────────────────────────────────┘

Lineage provides:

Complete call chain for debugging
Recursion depth tracking
Audit trail for accounting
Context for subagent calls

Persistence Layer

┌──────────────────────────────────────────────────────────────┐
│                    Persistence Backends                       │
├──────────────────────────────────────────────────────────────┤
│                                                               │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐     │
│  │ In-Memory│  │   File   │  │  SQLite  │  │PostgreSQL│     │
│  │ (Dev/Test)│  │(Compat) │  │ (Local)  │  │(Production)    │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘     │
│       │             │             │             │            │
│       └─────────────┴──────┬──────┴─────────────┘            │
│                            │                                 │
│                            ▼                                 │
│                   ┌─────────────────┐                        │
│                   │  Unified API    │                        │
│                   │ (persist, load) │                        │
│                   └─────────────────┘                        │
│                                                               │
└──────────────────────────────────────────────────────────────┘

SQLite Schema (simplified)
┌──────────────────────────────────────────────────────────────┐
│  entities (id, entity_type, created_at)                      │
│  components (entity_id, component_type, data)                │
│  events (id, timestamp, type, data, entity_id)               │
│  messages (conversation_id, timestamp, role, content)        │
│  tool_calls (turn_id, timestamp, name, input, output)        │
└──────────────────────────────────────────────────────────────┘

Architecture Benefits

Shared Resources: Multiple agents can share toolboxes (e.g., same SQLite database)
Resource Pooling: HTTP connections pooled across all agents
Better Lifecycle Management: Explicit scopes with predictable cleanup
Foundation for Web API: Centralized state enables HTTP server interface
Durable Persistence: Built-in persistence layer with multiple backends
Thorough Lineage Tracking: Complete call chains for debugging

Core Types

Base Types (`System.Agents.Base`)

-- Unique identifiers
newtype AgentId = AgentId UUID
newtype ConversationId = ConversationId UUID
newtype StepId = StepId UUID

-- Agent configuration
data Agent = Agent
    { slug :: Text                    -- Unique identifier
    , apiKeyId :: Text                -- Reference to API key
    , flavor :: Text                  -- LLM provider (openai, etc.)
    , modelUrl :: Text                -- API endpoint
    , modelName :: Text               -- Model identifier
    , announce :: Text                -- User-facing description
    , systemPrompt :: [Text]          -- System instructions
    , toolDirectory :: FilePath       -- Path to tools
    , mcpServers :: Maybe [McpServerDescription]
    , extraAgents :: Maybe [ExtraAgentRef]
    , builtinToolboxes :: Maybe [BuiltinToolboxDescription]
    }

OS Types (`System.Agents.OS.Core`)

-- Phantom-typed entity IDs
newtype AgentId = AgentId EntityId
newtype ToolboxId = ToolboxId EntityId
newtype ConversationId = ConversationId EntityId

-- Agent components
data AgentConfig = AgentConfig
    { agentName :: Text
    , agentModel :: ModelConfig
    , agentSystemPrompt :: Text
    , agentToolboxBindings :: [ToolboxBindingSpec]
    }

data AgentState = AgentState
    { agentStatus :: AgentStatus
    , agentCurrentConversation :: Maybe ConversationId
    , agentCreatedAt :: UTCTime
    }

Agent Tree System

The AgentTree module manages multi-agent hierarchies and handles agent discovery, reference validation, and cycle detection.

Tree Structure

Agents form a directed graph where:

Parent-child edges: Discovered from tool directory hierarchy
Extra reference edges: Explicit references via extraAgents

┌─────────────┐
│  root-agent │
└──────┬──────┘
       │
   ┌───┴───┐
   ▼       ▼
┌──────┐ ┌──────┐
│tool-a│ │tool-b│
└──┬───┘ └──────┘
   │
   ▼
┌──────┐
│sub-1 │
└──────┘

Subagent Wiring

The Agent Tree system supports dynamic tool registration via STM TVars:

-- OS-native agents use STM TVar for mutable tool storage
wireAgentTools :: Props -> AgentConfigGraph -> Map AgentSlug OSAgentNode -> IO ()

Conversation Flow

┌─────────────┐     ┌─────────────┐     ┌─────────────┐
│   User      │────>│   Session   │────>│    LLM      │
│   Input     │     │   (turns)   │     │   (tools)   │
└─────────────┘     └──────┬──────┘     └──────┬──────┘
                           │                     │
                           │    ┌─────────────┐  │
                           └───>│ Tool Call   │<─┘
                                │ Execution   │
                                └──────┬──────┘
                                       │
                           ┌───────────┴───────────┐
                           ▼                       ▼
                    ┌─────────────┐        ┌─────────────┐
                    │  Bash Tool  │        │  MCP Tool   │
                    └─────────────┘        └─────────────┘

Tool Registration

Tools are registered with the LLM via the ToolRegistration type:

data ToolRegistration = ToolRegistration
    { toolName :: Text
    , toolDescription :: Text
    , toolParameters :: Value  -- JSON Schema
    , toolExecutor :: Value -> IO ToolResult
    }

Tool Sources

BashToolbox: Executable scripts in the tool directory
McpToolbox: MCP servers providing dynamic tool lists
OpenAPIToolbox: REST API operations from OpenAPI specs
IOTools: Haskell functions embedded in the runtime
SystemToolbox: Builtin system information tools
Subagent Tools: Other agents exposed as callable tools

Libraries and executables

agents.cabal builds several components:

Component	Source	Contents
`agents-lib` (public library)	`src/`	The core: agents, tools, sessions, storage, the OS layer, the MCP stdio server, CLI commands other than the TUI. No terminal-UI dependencies.
`agents-postgres` (public library)	`postgres/`	Session and continuation stores in Postgres, for `withHostStores`. Kept apart so that `agents-lib` does not need libpq.
`agents-tui` (public library)	`tui/`	The terminal UI (`System.Agents.TUI.*`, `CLI.TUI`, `CLI.Config`, `CLI`), on brick and vty.
`agents-exe`	`app/`	The command-line tool, on both libraries.
`agents-server-internal`, `agents-server`	`examples/agents-server/`	The HTTP server (wai, warp). See [agents-server.md](/docs-agents-server.html).
`agq`, `durable-workflow-demo`	`agq/`, `examples/durable-workflow-demo/`	Other executables.

Programs that embed agents depend on agents-lib only.

Module Dependencies

Main
  ├── AgentTree
  │     ├── Base
  │     ├── FileLoader
  │     └── OS.Core
  ├── CLI.*
  │     └── AgentTree
  ├── TUI
  │     ├── Host, Host.Runner, Host.Client (RunnerClient)
  │     └── Session (pure views only: usage, signals, trajectory)
  ├── MCP.Server
  │     └── AgentTree
  └── ExportImport.*
  
OS Layer
  ├── OS.Core
  │     ├── OS.Core.Types
  │     └── OS.Core.World
  ├── OS.Resources
  │     ├── OS.Resources.Types
  │     ├── OS.Resources.Sqlite
  │     ├── OS.Resources.Lua
  │     └── OS.Resources.Http
  ├── OS.Concurrent
  │     ├── OS.Concurrent.Types
  │     └── OS.Concurrent.Locks
  ├── OS.Conversation
  │     ├── OS.Conversation.Types
  │     └── OS.Conversation.Lineage
  └── OS.Agents

Key Design Decisions

STM for Concurrency: Tool reloading and subagent wiring use STM for thread-safe updates
ECS Pattern: Enables flexible composition and powerful queries
Explicit Resource Management: Predictable cleanup with explicit scopes
Phantom Types: Type safety for entity IDs without runtime overhead
Type Erasure: Any for heterogeneous storage with safe casting via Component typeclass
Tracer Pattern: All side effects are traced for observability
Two-Phase Initialization: Agent shells created first, then wired together to support cycles

Tradeoffs

ECS Complexity: Adds indirection but enables powerful queries and flexible composition
STM Overhead: Slight performance cost for composability
Storage Overhead: Component storage uses more memory than direct fields but enables dynamic extension
Type Erasure: Using Any requires careful casting but enables heterogeneous storage

Changelog: OS Model Migration

2026-10-04T15:53:54Z

Generated from documentation/CHANGELOG-OS-MIGRATION.md, the repository is the canonical source and may be ahead of this page.

Changelog: OS Model Migration

Date: March 2026
Commits: 0eb5263 to cb52af7

Summary

This release introduces a major architectural refactoring of the agents-exe core, migrating from a Runtime-per-agent model to a centralized Entity-Component-System (ECS) based OS model.

Major Changes

New Architecture: ECS-Based OS Model

The framework now uses an Entity-Component-System (ECS) pattern at its core:

Entities: Lightweight UUID-based identifiers (AgentId, ToolboxId, ConversationId, etc.)
Components: Pure, serializable data attached to entities (AgentConfig, AgentState, ToolboxConfig)
Systems: Functions that operate on entities with specific component combinations

New Modules (27 OS Modules Added)

Module	Purpose
`System.Agents.OS`	Main OS module, exports all OS functionality
`System.Agents.OS.Core`	Core ECS types and World operations
`System.Agents.OS.Core.Types`	Component typeclass and entity ID types
`System.Agents.OS.Core.World`	World storage with STM-based component stores
`System.Agents.OS.Agents`	OS-native agent creation and management
`System.Agents.OS.AgentTree`	OS-native agent tree initialization
`System.Agents.OS.Concurrent`	Concurrent access patterns
`System.Agents.OS.Concurrent.Types`	Access pattern types (Exclusive, Read-Write, Pool)
`System.Agents.OS.Concurrent.Locks`	STM-based locking primitives
`System.Agents.OS.Resources`	Resource lifecycle management
`System.Agents.OS.Resources.Types`	Resource scope definitions
`System.Agents.OS.Resources.Sqlite`	SQLite resource management
`System.Agents.OS.Resources.Lua`	Lua interpreter resources
`System.Agents.OS.Resources.Http`	HTTP connection pool resources
`System.Agents.OS.Conversation`	Conversation and turn management
`System.Agents.OS.Conversation.Types`	Turn, Message, Conversation components
`System.Agents.OS.Conversation.Lineage`	Call chain tracking
`System.Agents.OS.Compat`	Compatibility layer exports
`System.Agents.OS.Compat.Runtime`	Runtime-to-OS bridge
`System.Agents.OS.Interfaces`	Interface layer
`System.Agents.OS.Interfaces.TUI`	TUI adaptation for OS
`System.Agents.OS.Interfaces.OneShot`	OneShot adaptation for OS

Migration Phases

The migration follows a phased approach:

PhaseOldOnly (REMOVED) ──> PhaseDual ──> PhaseNewOnly
    (Legacy only)          (Both)        (OS only)

March 2026 Update: PhaseOldOnly has been removed. The system now operates in:

PhaseDual (default): Both Runtime and OS available
PhaseNewOnly: OS only, Runtime deprecated

Key New Capabilities

Shared Toolboxes: Multiple agents can share the same SQLite database, HTTP connection pool, or other resources
Better Resource Management: Explicit lifecycle scopes (Program, Agent, Toolbox, Conversation, Turn, ToolCall)
Concurrent Access: STM-based synchronization with multiple patterns:
- ExclusiveAccess: Single accessor (TMVar) - for Lua interpreters
- ReadWriteAccess: Multiple readers/single writer (RWLock) - for SQLite
- PoolAccess: Bounded pool (TBQueue) - for HTTP connections
- StatelessAccess: No synchronization needed
Durable Persistence: Pluggable backends (SQLite, PostgreSQL, file-based)
Complete Lineage Tracking: Full call chains for debugging and accounting
Foundation for Web API: Centralized state enables HTTP server interface

Component Type IDs

ID	Component
1	AgentConfig
2	AgentState
3	ToolboxConfig
4	ToolboxState
5	ToolboxBinding
30	ConversationConfig
31	ConversationState
32	AgentConversation
33	TurnConfig
34	TurnState
35	ToolCallConfig
36	ToolCallState
38	Message

New CLI Modules

System.Agents.CLI.Export - Export tools and agents
System.Agents.CLI.Import - Import tools and agents

New Documentation

Document	Purpose
`documentation/architecture.md`	Updated with OS model architecture
`documentation/OS-API.md`	Complete API reference for OS model
`documentation/MIGRATION-OS.md`	Migration guide from Runtime to OS
`documentation/MIGRATION-GUIDE.md`	General migration guidance
`documentation/advanced-configuration.md`	Advanced configuration options

New Tests

Test Module	Purpose
`test/OS/IntegrationTests.hs`	End-to-end OS scenarios
`test/OS/CompatibilityTests.hs`	Runtime/OS compatibility tests
`test/OS/CoreTests.hs`	ECS core functionality
`test/OS/ConcurrentTests.hs`	Concurrent access patterns
`test/OS/ResourcesTests.hs`	Resource management
`test/OS/ConversationTests.hs`	Conversation/lineage
`test/OS/CompatTests.hs`	Compatibility layer
`test/OS/InterfaceTests.hs`	Interface layer

New Benchmarks

Benchmark	Purpose
`bench/OSBenchmarks.hs`	OS model performance benchmarks

API Changes

Old (Still Supported via Compatibility Layer)

-- Legacy Runtime approach
runtime <- newRuntime props agent tracer
result <- runWithRuntime runtime $ do
    tools <- listTools
    callTool "my-tool" args

New (OS Model)

-- OS model approach
import System.Agents.OS

-- Initialize world
world <- atomically $ do
    w <- newWorld
    w' <- registerComponentStore w (Proxy @AgentConfig)
    registerComponentStore w' (Proxy @AgentState)

-- Create agent
let config = AgentConfig
    { agentName = "my-agent"
    , agentModel = ModelConfig "openai" "url" "gpt-4" "key"
    , agentSystemPrompt = "You are helpful"
    , agentToolboxBindings = []
    }
agentId <- createAgent world config

Migration Path

Current (PhaseDual): Use System.Agents.OS.Compat.Runtime for gradual migration
Future (PhaseNewOnly): Direct OS model usage

See documentation/MIGRATION-OS.md for detailed migration instructions.

Build Changes

New Dependencies

deepseq - For benchmark strictness
criterion - Benchmarking framework
async - Concurrent test execution
mtl - Monad transformers

Cabal Updates

Added 27 new OS modules to library
Added new test modules
Added benchmark section

Benefits

Shared Resources: Multiple agents sharing toolboxes
Resource Pooling: HTTP connections pooled across agents
Better Lifecycle: Explicit cleanup with scopes
Web API Ready: Centralized state for HTTP interface
Durable Persistence: Multiple backend options
Lineage Tracking: Complete call chains

Tradeoffs

ECS Complexity: Adds indirection but enables flexible composition
STM Overhead: Slight performance cost for composability
Dual Mode: Maintenance burden during transition
Storage Overhead: More memory than direct fields

Backward Compatibility

Full compatibility layer provided
Old Runtime interface still works
Gradual migration supported
Breaking changes documented in migration guide

#348 - Core Entity and Component Types
#349 - Resource Management
#350 - Concurrent Access
#351 - Conversation and Lineage Tracking
#352 - OS Monad
#353 - Runtime Compatibility
#354 - TUI and OneShot Adaptation
#355 - Persistence Layer
#356 - Documentation and Integration (this release)

Contributors

Lucas DiCioccio

For questions about the migration, see documentation/MIGRATION-OS.md or file an issue with the “migration” label.

Spec: the session mailbox (agent-to-agent mail, suspended tool calls, interrupts, tool-call wrappers)

2026-10-04T15:53:54Z

Generated from todos/session-mailbox.md, the repository is the canonical source and may be ahead of this page.

Spec: the session mailbox (agent-to-agent mail, suspended tool calls, interrupts, tool-call wrappers)

Status: proposal, 2026-09-22. Nothing implemented. Supersedes the designs in issues #563 and #564, and most of #507 (see “What happens to the issues”).

Goal

One mechanism through which everything that happens to a running session reaches it: a user message, a message from another agent’s session, the result of a tool call the LLM was told is “still running”, the result of a deferred call, a timer, a pause request.

Around it:

(A2A) an agent can write to another session, running or idle, and get an answer back, across sessions and (on agents-server) across restarts;
(suspension) a slow tool call is suspended inside the runtime: the LLM gets running, id = call_abc and takes its turn; it may then wait up to N seconds, poll, cancel, or ignore the call;
(interrupts) while the runtime is blocked (on attached tool calls, on a wait, optionally on an LLM completion), higher-priority mail pre-empts the wait;
(wrappers) tool calls are wrapped declaratively (timeout, retries, hooks that may deny / defer / rewrite), which is the existing, currently inert, Decorator made real.

The OS picture

OS	here
CPU running user code	the LLM completing a turn
syscall	a tool call
blocking syscall	an attached call: the step waits for it
`SIGTSTP` / `bg`	detach: the call keeps running, the LLM gets a placeholder
`wait(2)`, `kill(2)`, `/proc`	`wait`, `cancel-tool-call`, `get-tool-call-status`
interrupt controller + message queue	the mailbox
interrupt-enabled points	the receive points R1-R4 below
syscall filter / LSM hooks	wrappers
scheduler run queue	`Host.Runner`: a session with unread mail and no run is runnable

Current state

More of this exists than the issues assume. They predate Session/Async*, Session/Wake.hs and Host/Runner.hs.

What we can reuse as-is

Background calls: RunAsync, the async engine, OS tool-call entities, placeholder tool messages (partialToolMessages), progress, cancellation of the whole process group (documentation/async-tool-calls.md).
get-tool-call-status (with wait_for_completion + timeout_seconds), list-running-tool-calls, cancel-tool-call.
Deferred calls, continuation tokens, wakeSessionWith.
Host.Runner: per-session lock, one run at a time, versioned writes (sbCompareAndStore), SessionEvent broadcast + subscribe, the HTTP event stream, lineage (smParent) and owners (smOwner).
AsyncToolResponse = ToolComplete | ToolYield, which is the right result type for a wrapper that wants to suspend a call.

Gaps

G1. A session has half a dozen unrelated input channels, each with its own delivery rule:

input	channel	"delivered once" is ensured by
user text	`usrQuery :: IO (Maybe UserQuery)`, pulled	the caller
background result	STM over the OS entity (`awaitEntities`)	`tcDeliveredLate` flag in the turn
deferred result	`lsInbox :: TVar [(token, result)]` (`Runner.hs:204`)	`csComplete` + `TokenAlreadyCompleted`
user text, TUI	a `BChan` per conversation + "buffered messages" in the core, drained by `usrQuery` (`tui/…/Event/Conversation.hs`)	clearing the buffer on read
stop	`cancel` of the run thread (`cancelRun`)	-
pause, TUI	`step` polls `isConversationPaused` every 200 ms	-

G2. A busy session cannot be talked to. postMessage answers RunInProgress / NotAcceptingMessages (Runner.hs:664). Nothing can be addressed to a session by another session at all. This is the A2A blocker.

G3. The one interrupt we have, race agent.usrQuery (waitForRunningCalls …) (Step.hs:630), races two IO actions, so usrQuery “must tolerate cancellation”, and it only exists at one point (idle with background calls). Attached calls, wait_for_completion and LLM completions are not interruptible.

G4. Suspension is static: a tool is runAsync or it is not. There is no “run it, and if it is still going after 10 s hand control back” per call; yieldOnTimeout is per agent and per step. And a runSync call runs in the step’s own thread, so it can never be detached.

G5. Decorators are parsed, stored, and never applied. WithTimeout, WithRetries, WithCache, WithLabel (Session/Types.hs:468) are flattened into ccDecorators (Step.hs:347), which nobody reads; Durable.hs binds them to _decorators four times. Policy rules match one exact tool name, first match wins (AgentConfig.hs:26), so a decorator cannot be applied across tools.

G6. “If the model reads a result with get-tool-call-status, the delivery notice repeats it once” (documented limit): a symptom of G1, there is no notion of a message having been consumed.

Design

1. The mailbox

An envelope:

data Envelope = Envelope
    { envId       :: MessageId      -- idempotency key; sender may supply it
    , envSeq      :: Int            -- assigned on accept; total order per session
    , envFrom     :: Sender
    , envPriority :: Priority       -- Normal | Interrupt
    , envHops     :: Int            -- A2A loop guard, see §5
    , envSentAt   :: UTCTime
    , envBody     :: MailBody
    }

data Sender
    = FromUser (Maybe Text)                 -- owner, when known
    | FromSession SessionId (Maybe Text)    -- and its agent slug
    | FromToolCall ToolCallId
    | FromSystem Text                       -- timers, watchers, the runner

data MailBody
    = UserMessage UserQuery
    | AgentMessage Text (Maybe MessageId) Bool      -- text, in-reply-to, expects-reply
    | ToolCallFinished ToolCallId ToolCallState UserToolResponse
    | ContinuationResult ContinuationToken UserToolResponse
    | WatchedEvent SessionId Text Aeson.Value       -- §7
    | Control ControlMsg                            -- Pause | Resume | CancelCalls [ToolCallId] | StopRun

Progress is not mail. It is state (the OS entity keeps the last 50 entries) and is read, not delivered.

The interface, a record of functions like ContinuationStore, carried as ctxMailbox :: Maybe Mailbox on Agent:

data Mailbox = Mailbox
    { mbSend   :: Outgoing -> IO (Either SendError Receipt)  -- any thread, any session
    , mbUnread :: Cursor -> STM [Envelope]                   -- never removes anything
    , mbTrim   :: Cursor -> IO ()                            -- GC at or below the cursor
    }

data SendError = UnknownRecipient | MailboxFull | NotPermitted | TooManyHops
data Receipt   = Receipt { rcptId :: MessageId, rcptSeq :: Int, rcptDuplicate :: Bool }

Transactional means three things, and only these:

Accept is atomic and idempotent. One STM transaction (in memory) or one INSERT … ON CONFLICT DO NOTHING (durable) assigns envSeq. Resending the same envId returns the first receipt with rcptDuplicate.
Reading does not consume. There is no pop. A reader is a cursor.
Consumption commits with its effect. Session gains mailCursor :: Int (JSON default 0). A step that folds envelopes ≤ k into a turn returns a session with mailCursor = k; the versioned store of that session is the ack. A crash between reading and storing redelivers into a session that does not contain the mail yet: exactly-once effect, no two-phase anything.

Point 3 is why #563’s TQueue component is not enough (a readTQueue followed by a crash loses the message), and it removes tcDeliveredLate-style flags: “delivered” is envSeq ≤ mailCursor.

Because mbUnread is STM, a blocking receive is retry, a filtered receive is a filter, and a receive with alternatives is orElse:

awaitMail :: Mailbox -> Cursor -> (Envelope -> Bool) -> STM [Envelope]
awaitMail mb cur p = do
    es <- filter p <$> mbUnread mb cur
    when (null es) retry
    pure es

Two implementations:

in memory (TVar (Seq Envelope)), for run, the TUI, sub-agents, tests;
durable, for Host: a session_mail table next to the continuations (session_id, seq, id UNIQUE, from_json, priority, hops, body_json, accepted_at), SQLite and Postgres, fronted by the same TVar so mbUnread stays STM. The runner is single-process per session already (second writers are detected), so the TVar front is sound; a multi-process Postgres deployment fills it from LISTEN/NOTIFY (mbSync, with the sequence numbers settled by the database).

Bounded: mailboxMaxUnread (default 256). A full mailbox refuses with MailboxFull; the sender’s tool call fails, which is the backpressure. Control and ToolCallFinished are exempt.

2. Receive points

The stepper reads the mailbox at four points. Everything else in this spec is a consequence of what each point does.

	when	blocking	reads
R1	building a user turn, before `AskLlmCompletion`	no	all unread
R2	idle: the LLM issued no calls, or only detached/deferred calls remain	yes	all unread
R3	blocked on attached calls, or inside the `wait` tool	yes, as one arm of an `orElse`	`Interrupt` (R3a), any (R3b, `wait`)
R4	during an LLM completion (opt-in)	`race`	`Interrupt`

R1 generalises collectLateResults; R2 replaces the race in askUserQuery and the block flag; both become:

-- R2, no IO race, nothing to cancel
atomically $ awaitMail mb cur (const True)

-- R3a: attached calls vs. interrupts, one transaction
atomically $ (Left  <$> awaitMail mb cur isInterrupt)
    `orElse` (Right <$> awaitEntities world any attached)

A genuine race (thread cancellation) remains only where an IO action must be abandoned: R4, and an inline call under StopRun.

What the LLM sees. Providers want exactly one tool message per call, then optionally a user message. So mail is folded into the userQuery of the turn being built, as lateResultsQuery does today, one block per envelope with a header the model can quote back:

[mail m-7f3a from session 9c1e (agent "reviewer"), expects reply]
The migration in your PR drops an index that report_daily depends on.

[mail m-7f3b: tool call call_abc (run_tests) completed]
42 tests passed

The raw envelopes are also kept on the turn (userMail :: [Envelope], new optional field of UserTurnContent / PartialUserTurnContent), so the TUI, the chat page and session-print render senders properly instead of parsing a blob, and the audit trail #563 wanted (imSourceAgent) is in the session file.

Control envelopes are consumed (cursor moves) but render nothing.

Compatibility. ctxMailbox = Nothing keeps today’s code paths, usrQuery included. With a mailbox, front-ends post UserMessage instead of answering usrQuery: one-shot runs pre-load one envelope; the TUI’s message editor posts to the selected conversation’s mailbox, which replaces both the per-conversation BChan and the buffered messages (typing at a busy conversation is unread mail), and the TUI’s pause becomes Control Pause instead of a polling step; postMessage posts. usrQuery stays on the record for embedders.

3. Suspended tool calls: attach and detach

In asynchronous mode every call that has an OS entity runs in the engine. A disposition no longer says where a call runs but how long the step stays attached to it:

config	meaning
`{"tag":"runSync"}`	attached until done (today's behaviour, now detachable by an interrupt)
`{"tag":"runAsync"}`	attached per the agent's `asyncYieldStrategy` (unchanged)
`{"tag":"runAsync","attachSeconds":10}`	attached for 10 s, then detached
`{"tag":"runAsync","attachSeconds":0}`	detached at once

attachSeconds is G4’s answer: fast calls look synchronous, slow ones suspend themselves. The wait is R3a with one more orElse arm per distinct deadline (a registerDelay TVar).

Detaching is what executeTrackedCalls already does when it emits a PartialUserTurn with Running calls; the placeholder gains the reason:

{"status": "running", "tool_call_id": "call_abc",
 "detached": "still running after 10s",
 "message": "Its result will arrive as mail. Use wait, get-tool-call-status or cancel-tool-call."}

On completion the engine posts ToolCallFinished to the owning session’s mailbox in the same STM transaction that marks the entity TcCompleted, so a result can neither be lost nor observed by get-tool-call-status before it is mail. Fixing G6: when get-tool-call-status returns a final result, the stepper marks that envelope as read-by-tool on the turn, and R1 renders it as a one-line already read notice instead of repeating the payload.

The LLM’s side, in the System Toolbox:

capability
`wait` (new)	`{"for": ["call_abc"] \\| "any-call" \\| "mail", "timeout_seconds": 30}`. Returns on the first of: a named call is final, any mail, timeout. Says which. Peeks, never consumes: the mail it woke for is delivered by R1 in the same turn.
`get-tool-call-status`	unchanged; `wait_for_completion` becomes sugar for `wait` and gains interruptibility
`cancel-tool-call`, `list-running-tool-calls`	unchanged
`send-to-tool-call` (later)	for a sub-agent call: mail to the child session (§5). For bash: stdin, not planned.

wait needs the mailbox, so ToolExecutionContext gains ctxAwaitMail, a hook like ctxCancelToolCall. wait always takes a timeout, capped by maxWaitSeconds (default 300): two agents waiting on each other time out instead of deadlocking.

Background calls still die with their process (orphaned on reload). Unchanged, out of scope.

4. Interrupts

Priority is set by the sender and checked on accept: the user and Control may always interrupt; a session may interrupt its descendants; peers may not (configurable, §5).

`Interrupt` mail arrives while the step is…	effect
attached to engine calls (R3a)	all attached calls are detached; the LLM is asked at once, with the mail and placeholders. This is `^Z` then talk.
in the `wait` tool (R3b)	`wait` returns `{"woken_by": "mail"}`; any priority does this
in an inline call (no entity, no engine)	nothing until it returns; `StopRun` cancels the thread
in an LLM completion (R4), `interruptCompletions: true`	the completion is cancelled; the head `UserTurn` is amended with the mail (the LLM never answered it, so no turn is lost) and asked again
in an LLM completion, default	delivered at the next R1/R2
idle (R2)	same as `Normal`

Normal mail never pre-empts anything but wait and R2.

Control:

Pause: the run stops at the next receive point, cursor committed, status paused; detached calls keep running unless pauseCancelsCalls. Resume (or, by config, any mail) makes it runnable again.
CancelCalls: the engine’s cancel; results come back as ToolCallFinished … Failed.
StopRun: today’s cancelRun, now reachable by mail.

5. Agent-to-agent

Address: a SessionId. Discovery is by the existing list-sessions / search-sessions, plus smAgent, smParent.

Routing. mbSend needs to find the recipient’s mailbox, in every front-end and not only on the server (helper agents are used extensively in the TUI). A MailRouter is the process’s table of live mailboxes:

data MailRouter = MailRouter
    { mrRegister :: SessionId -> MailboxInfo -> Mailbox -> IO (IO ())  -- returns unregister
    , mrLookup   :: SessionId -> IO (Maybe (MailboxInfo, Mailbox))
    , mrList     :: IO [(SessionId, MailboxInfo)]   -- agent slug, parent, status
    }

front-end	router	a session is live…
`agents-server`	`Host.Runner` (durable mail; unknown-but-stored sessions are loaded on demand)	always
TUI	one per process, held next to `coreRef`; every conversation registers, sub-calls included	while its conversation exists
`run`, MCP server	one per process; the root and every sub-agent session register	until its loop returns

It travels as ctxMailRouter on Agent and is handed down to sub-agents the way ctxWorld and ctxEventQueue are, so a whole call tree shares one. mrList is what makes in-memory sessions discoverable (the “ongoing sessions” #563 wanted to list): list-sessions merges it with the persisted ones.

Scheduling rule: a session with unread mail, no run, and a status that accepts mail is runnable. On the server the runner starts a run; in the TUI and in run a session’s loop is simply parked at R2 and wakes by itself.

Which mail wakes an idle session is a per-agent option, wakeOn, a list among user, tool, parent, child, peer. Mail that does not wake is still queued and delivered with the next thing that does. Default ["user", "tool", "parent", "child"], deliberately a config knob and not a rule of the design: it is too early to know what peers should be allowed to do to each other (D10). The rule also replaces the special case in completeCall: autoResume becomes “post ContinuationResult”, and lsInbox goes away.

postMessage during a run stops failing: it posts UserMessage and returns the receipt (HTTP 202 with message_id, seq), optionally "interrupt": true.

Tools (System Toolbox capabilities):

capability
`send-message`	`{"to": "<session>", "text": …, "in_reply_to": "m-…", "expects_reply": true, "interrupt": false}` → receipt + the recipient's status (`running`, `idle`, `paused`)
`spawn-session`	`{"agent": "<slug>", "message": …}` → the new session's id. A session that is a child in lineage but not call/return: it outlives the tool call that created it, and answers by mail. `<slug>` is one of the caller's helpers, as for `prompt_agent_<slug>`, with the same `with` / `bindings` narrowing.
`wait`	§3, with `"for": "mail"`

spawn-session exists in every front-end:

server: createSessionAs, durable, survives restarts;
TUI: a new conversation, created the way sub-calls are today (OSEvent_SubcallStarted, conversationParentId), so it shows up in the conversation list, the user can open it and type into it, and its mailbox is what the message editor posts to. It lives until the user closes it or its parent conversation goes away;
run: a thread with an in-memory mailbox, cancelled with the root like background calls are (withEngineShutdown). A root that stops while spawned sessions still work loses them; an agent that wants their answers calls wait (D11).

Call/return sub-agents (prompt_agent_<slug>) are unchanged, but when they run detached their placeholder and status report the child session id, so the parent can send-message to a helper that is still working, and the helper’s final answer arrives as ToolCallFinished. “Interacting with a running tool call” and “agent-to-agent chat” are the same mechanism.

Permissions: #563’s scope, checked in mbSend, per sending agent: mailScope: own | children | subtree | all (default subtree), and interruptScope (default children). Through the HTTP API the existing owner check applies.

Loops: a reply carries envHops = hops of the mail being answered + 1 (the stepper knows which mail the current turn consumed); maxMailHops (default 16) refuses with TooManyHops. User mail resets to 0. Together with MailboxFull and the wait cap, two agents cannot ping-pong or deadlock forever.

6. Wrappers

Decorators become middleware over the executor, applied in one place (executeCall, which both the inline path and the engine already go through):

type Exec = ToolExecutionContext -> LlmToolCall -> IO AsyncToolResponse
type ToolMiddleware = Exec -> Exec

interpretDecorator :: WrapperEnv -> Decorator -> ToolMiddleware
-- Decorate [d1, d2] base  ==>  d1 (d2 exec): first listed is outermost

AsyncToolResponse rather than UserToolResponse so that a wrapper can answer ToolYield: that is how a before hook turns a call into a deferred one (an approval gate) from inside the engine, without blocking the step.

Config: rules keep choosing the base disposition (first match). A new wrappers list contributes decorators; every match applies, in file order, outermost first. The result is still one Decorate […] base, recorded in AppliedPolicy, so the session file shows what wrapped each call.

"toolCallPolicyConfig": {
  "default": {"tag": "runSync"},
  "rules": [{"tool": "run_tests", "disposition": {"tag": "runAsync", "attachSeconds": 10}}],
  "wrappers": [
    {"match": {"tool": "*"},        "decorators": [{"tag": "truncate", "maxBytes": 65536}]},
    {"match": {"tool": "http_*"},   "decorators": [{"tag": "retries", "count": 2, "backoffMs": 500},
                                                   {"tag": "timeout", "seconds": 30}]},
    {"match": {"tool": "deploy_*"}, "decorators": [{"tag": "before", "hook": {"command": "hooks/approve-deploy"}},
                                                   {"tag": "after",  "hook": {"tool": "audit_log"}}]}
  ]
}

match: tool (glob on the LLM-visible name), toolbox, later args (a JSON predicate). Decorators:

tag
`timeout`, `retries`, `cache`, `label`	the existing four, now applied. Per-call `timeout` wins over `asyncCallTimeoutSeconds`; each retry reports progress.
`truncate`	cap the result, say so to the model
`before`	hook sees `{tool, arguments, session_id, tool_call_id, agent}`; answers `continue` (optionally `arguments` rewritten), `deny {message}` (the message is the tool result, the tool does not run), `defer {reason}` (→ `ToolYield`, a continuation token, the usual external completion), `answer {result}` (short-circuit)
`after`	hook sees the above plus `result`, `status`, `duration_ms`; answers `continue` (optionally `result` rewritten), `annotate {text}`

A hook target is {"command": path} (JSON on stdin, JSON on stdout, the isolation-envelope conventions, non-zero exit = deny for before, ignored with a trace for after), or {"tool": name}: any tool registered on the agent, visible to the LLM or not, including a sub-agent, which gives LLM-as-guard for free and lets hooks use bindings and secrets like any tool.

Two rules:

Hooks see what the LLM sent. Bindings are merged inside toolRun, below executeCall, so bound values, secrets included, never reach a hook, and a before rewrite cannot override a binding.
A hook failure (crash, timeout, bad JSON) is deny for before (“fail closed”) and continue for after, both traced.

7. Watching other sessions

Host.Runner already broadcasts SessionEvent and serves it over HTTP. No subscription registry, no filter algebra: a watch forwards matching events of one session into the watcher’s mailbox as WatchedEvent, and wait is the only blocking primitive.

watch-session {"session": …, "events": ["run.stopped", "tool.completed"], "tool": "deploy_*", "ttl_seconds": 600}, unwatch-session. Watches are in-memory, scoped like mail, capped per session, and die with their TTL or either session. SessionEvent gains the two tool-call events (tool.started, tool.completed); the rest of #563’s event list is already covered by SessionUpdated carrying the head turn.

What happens to the issues

issue	kept	changed	dropped
#563 ph.1 inbox	inject a message, pause, cancel calls, scopes, audit of the sender	an `Agent`-level interface with a cursor in the session, not an ECS `TQueue` component: the server path does not keep sessions in the World, and a queue pop is not transactional with the session store	`InjectSystemInstruction`; `ForkConversation` (belongs with §5.2 of the partial-application spec)
#563 ph.2 event history	tool-call events	extend `Host.Runner.SessionEvent`; history = the stored session versions	a second `SessionEvent` type, the `TBQueue` ring
#564 ph.3 subscriptions	watching a session	events → mail (§7)	registry, `FilterCombinator`, quotas per subscriber
#564 ph.4 wait-for	blocking with a timeout	`wait` on one's own mailbox (§3)	`EventCondition` DSL. Note the sketch's `readTQueue` then `retry` on a non-match never progresses: `retry` undoes the read, so a non-matching head blocks the queue forever.
#507 suspend	suspended = no run, mail accumulates, cursor committed; `Pause`/`Resume`	-	`ManagedResource` tracking: engine shutdown, process groups and `evictIdle` already release what a run holds. Reopen if a toolbox leaks.

Phases

Each phase ships alone and leaves ctxMailbox = Nothing agents untouched. Phases 0, 3 and 5 depend on nothing but Phase 1 (Phase 0 not even on that); Phase 4 needs Phases 1-2 and, for its server part only, Phase 3.

Phase 0: decorators are real

interpretDecorator for the existing four + truncate; applied in executeCall from tcPolicy; wrappers with glob matching. Independent of everything else. Closes G5.

Phase 1: the mailbox, in memory

Mailbox, Envelope, mailCursor, userMail; R1 and R2; the engine posts ToolCallFinished; late results and the usrQuery race ported; TUI and one-shot post UserMessage. Visible change: typing during a busy run is accepted and delivered at the next R1. Closes G1, G6.

Phase 2: attach / detach

All entity-backed calls through the engine in async mode; attachSeconds; R3a with interrupt-detaches; wait and ctxAwaitMail; an interrupt key in the TUI. Closes G3 (except R4), G4.

Phase 3: durable mail on the server

session_mail on both backends; lsInbox and the completeCall special case replaced; the runnable rule and wakeOn; postMessage during a run → 202; Control by mail; mail in the event stream and on the chat page (plain text, as that page is on purpose). Closes G2.

Phase 4: agent-to-agent

MailRouter in the TUI, run and the server; send-message, spawn-session in all three, scopes, hops, wakeOn; detached sub-agent calls expose the child session id; list-sessions shows live in-memory sessions. The TUI comes first: it is where helpers are used most and where a spawned session can be watched and typed into, which is the cheapest way to find out what peers should be allowed to do (D10).

Phase 5: hooks

before / after, command and tool targets, deny / defer / answer / rewrite.

Phase 6: watches, R4, pause

watch-session, tool.* events, interruptCompletions, Pause/Resume.

Decisions (proposed)

D1. The cursor lives in the session, so the versioned session store is the ack. No per-message delivered flags, no separate consumer table.

D2. Mail is rendered into the user query, and kept raw beside it. The only shape every provider accepts after tool messages; the raw copy is for UIs and audit.

D3. Reads never consume; only the stepper advances the cursor. Tools (wait, status) peek. One consumer per mailbox, so no claim/lease protocol.

D4. In async mode, runSync means “attached forever”, not “in the step’s thread”. Otherwise an interrupt cannot detach it. Calls without an entity stay inline and uninterruptible, as today.

D5. Interrupting a completion amends the unanswered head turn rather than stacking a second user turn or keeping a truncated LLM turn. Opt-in, because a cancelled completion is paid for.

D6. Wrappers are data in the disposition, not a second mechanism: one Decorate, visible in AppliedPolicy. Wrappers compose (all matches); base dispositions do not (first match).

D7. Hooks never see bound values, and before fails closed.

D8. Events become mail; wait is the only blocking primitive. One thing to make interruptible, time-bounded and deadlock-free, instead of three.

D9. Progress is state, not mail. It would drown the mailbox and the context window; a tool that wants the model’s attention can finish, or a later notify progress level can post mail.

D10. What wakes an idle session is configuration (wakeOn), not design. Decided 2026-09-22: too early to settle what peers may do to each other, so the mechanism stays neutral and the default conservative. Same reasoning for mailScope / interruptScope: knobs, with defaults we expect to revisit once the TUI has been used with spawned sessions for a while.

D11. A run never lingers for a reply. It stops when the LLM is done, no background call runs and nothing is unread, whatever expects_reply messages are outstanding. An agent that wants the answer calls wait. expects_reply is a hint to the recipient, never a runtime obligation.

D12. Peer sessions exist in every front-end, not only on the server (decided 2026-09-22: helpers are used extensively in the TUI). Hence the MailRouter, and spawn-session as a TUI conversation. Durability is the only thing the server adds.

documentation/async-tool-calls.md, documentation/durable-workflows-howto.md, documentation/agents-server.md, todos/async-tool-calls.md, todos/tool-partial-application.md (bindings, §5.2 fork).

Spec: partial application of tool arguments

2026-10-04T15:53:54Z

Generated from todos/tool-partial-application.md, the repository is the canonical source and may be ahead of this page.

Spec: partial application of tool arguments

Status: Phases 1-7 and §8.4 (derive_agent) done as of 2026-09-21. Phase 1 done (97961e0). Phase 2 done except MCP server env (G4); the tool-cache key (G8) is done too, see below (03783fb, b0c53a5): parameters, ctxParams (now actually wired at runtime, not just in tests), --set/--set-json/--pin/--pin-json/--params-file, process-scope resolution, secret-argv-mode load-time guard (G7). Phase 3 done (8a41cd8): OpenAPI/PostgREST argument bindings via the generic combinator, ParamSource secrets resolved per request.

Phase 4 (session-level parameters in agents-server) done for the core mechanics, live-verified end to end against a real LLM: params on POST /v1/sessions, /messages, /resume and POST /v1/continuations/:token; the validation table (422 unknown_params, 403 forbidden_params, 422 invalid_params, 422 params_required); a params column and migration on both the SQLite and Postgres session backends (SessionMeta. smParams, non-secret only); secret session values kept only in the in-memory LiveSession (Host/Runner.hs’s new lsParams), never persisted; message-scope values applied for one run only, never stored; agentView’s "parameters" self-description (bound/pinned, never a value); agents-server --set/--set-json/--pin/--pin-json (mirroring agents-exe) so an operator can lock a container to one tenant. Not done from Phase 4: the chat page’s parameter form, and the generated OpenAPI document’s request/response schemas (AgentsServer/Types.hs/Routes.hs are a separate hand-maintained layer used only for /openapi.json and were not updated with the new fields — the real handlers in Api.hs are fully wired and correct; this is a documentation-only gap). recoverOnStartup marks the running calls of a session whose required parameters are no longer bound as failed with a params_required message (§5’s documented recovery behaviour; done 2026-09-25, which also made the persisted non-secret session values seed the live session back after a restart or an eviction, as §5 always said they would). seal and session tokens (§5.1) are done: seal/session_token on POST /v1/sessions, a security column (sealed flag and token SHA-256 digest, never the token) on both backends, session-token routing limited to its own session’s read, messages, events and cancel, and DELETE /v1/sessions/:id/token. Deviations: the token is only honoured on /v1/sessions/:id/... paths of its own session (anything else is a 401, so no probing); seal exempts the owner as §5.1 says, so today it only matters against session-token holders, who never may set parameters anyway; a fork or child does not inherit either. Since then: PUT /v1/sessions/:id/params, 409 params_required on a message or resume to an existing session, and params on POST /v1/sessions/:id/fork (which lacks §5.2’s fork on session creation and the CLI’s --fork; required parameters are not enforced at fork time).

Phase 5 (sub-agents and MCP over HTTP) done, checked live with agents-exe check against fixture agents and with an agents-server integration test: ExtraAgentRef gains with (§7), a map from the child’s parameter names to BindingValues in the parent’s scope; AgentConfigNode.nodeExtraWith carries it from discovery through to wireAgentTools, which checks it at load time (every with key must be a declared parameter of the child; every required session-scope parameter of the child must be covered by with or by the child’s own process value/default) and reports failures through the existing LoadingError/OtherError path. Props.agentToTool and turnAgentRuntimeIntoIOTool gained a Maybe (Map ParamName BindingValue) argument (Nothing for a child reached through a toolDirectory, which has no reference site to hang a with on); runSubAgent resolves it against the caller’s ctxParams at call time (OneShotTool.hs’s new resolveWith) and overlays the result onto the child’s own resolved params, so the child’s own process values and defaults still apply underneath. A Literal in with is never secret; a Param inherits secrecy from the caller’s value. MCP over HTTP (AgentsServer/Mcp.hs, Api.hs) gained Agents-Param-<name> request headers (always strings) and _meta."agents-exe/params" on a tools/call (any JSON, overrides the headers by name); each call is one run, so session and message scope coincide there, and both go through the same validation table as the REST API.

Phase 6 (narrowing helpers down the call chain) done, checked with the agents-tests suite (NarrowingTests.hs: the AgentAddress algebra, and describe_agent end to end against a loaded three-level tree) and with agents-exe check against fixture agents: Agent gains a top-level bindings list (§8.1), applied in loadAgentToolboxes after every toolbox’s own bindings, matching by the LLM-visible tool name across toolboxes (bash tools have no toolbox name of their own to hang a toolbox-level binding on). ExtraAgentRef gains narrowable (default True); AgentConfigNode.nodeExtraNarrowable carries it alongside nodeExtraWith. describe_agent (§8.2) is registered by wireAgentTools next to prompt_agent_<slug> for any agent with at least one helper; it recurses over nodeChildren/nodeExtraRefs, cut at maxDescribeDepth and at cycles, shows each declared parameter with a bound flag (already resolved, or covered by the reference’s with) and each tool’s still-open arguments (its current, already-bound-reduced schema minus anything the caller’s inherited bindings already cover), and reports only announce for a narrowable: false reference. prompt_agent_<slug> gains optional bindings and with arguments (§8.3): AgentBinding/ScopedBinding/ AgentAddress (Bindings/Types.hs) represent a binding addressed at the prompted helper (Here), one of its own helpers (a slash-separated AgentPath), or all of them ("**"/Everywhere); descendAddress/ reRootBindings re-root an address (or a whole list of ScopedBindings) one level down when a call descends into a named child. runSubAgent resolves the call’s own bindings against the caller’s ctxParams (resolveCallBindings; an unresolvable required Param fails the call with a message the calling model can act on, per D8), combines them with the caller’s own ctxInheritedBindings re-rooted at the prompted helper, applies whatever is addressed at the helper itself to a fresh copy of its osNodeTools (applyBindings, scoped to this call only — the shared, loaded toolboxes are never mutated) via a throwaway OSAgentNode passed to buildAgent, and carries the rest into the sub-agent’s own SessionBase.ctxInheritedBindings for it to apply, in turn, when it prompts its own helpers. ToolExecutionContext/Session.Base.Agent both gained ctxInheritedBindings :: [ScopedBinding], threaded through buildContext like ctxParams; ToolExecutionContextSnapshot keeps only its non-secret entries, same as tecsParams. A call-time with cannot refill a parameter the reference’s own static with (§7) already fills. A narrowable: false reference refuses any bindings/with on the call that reaches it, and any inherited binding addressed at it or below, rather than applying them.

Phase 7 (Expose) done, checked with the agents-tests suite (BindingsTests.hs’s new “Expose bindings (§7)” group) and live with agents-exe check against fixture agents (an Expose-bound bash argument stays in the schema when its parameter is a session-scope parameter that check never resolves; a secret parameter bound with whenUnbound: "expose" is a load-time ExposedSecretParameter error). A binding whose whenUnbound is "expose" is left out of applyBindings’s static schema reduction entirely (D3’s opt-in exception): the shared registration keeps the argument declared, so it is never hidden or shown by mistake for every session at once. Bindings.narrowExposedSchema :: Params -> [Binding] -> ToolRegistration -> ToolRegistration re-hides it when the parameter happens to resolve in a given snapshot; System.Agents.Combinators. ProgressiveDisclosure.agentEvaluateActiveTools calls it fresh on every read of the tool list — the same “read fresh on every access” pattern it already used for activation — against a new IO Params action (AgentFactory.AgentDeps.adLiveParams) read only when the node has at least one Expose binding. Outside agents-server, adLiveParams defaults to pure mempty: an Exposed argument then behaves like a process-scope-only binding, decided once from osNodeParams, since there is no notion of a session distinct from the process. agents-server’s Host/Runner.newAgent overrides it with readTVarIO live.lsParams, so the tool list a session sees updates the moment prepareParams writes a new session-scope value in — no agent rebuild, no registration mutation. A message-scope value (this run only, never written to lsParams) is not seen by this mechanism and so is treated as unbound for schema-visibility purposes; the value itself is still merged correctly at call time regardless, since that path (wrapTool, unchanged) already reads ctxParams fresh per call. AgentTree.loadAgentToolboxes collects every Expose binding for a node (toolbox-level, returned out of ToolLoader.loadAgentTools/loadBashTools/loadOpenAPIToolboxes/ loadPostgRESToolboxes; and agent-level, from Agent.bindings) into a new OSAgentNode.osNodeExposeBindings TVar, and rejects any of them bound to a declared secret parameter (Bindings.exposedSecretBindings, a new ExposedSecretParameter LoadingError): exposing a secret back to the model would just let it retype the value in plain text.

§8.4 (derive_agent) done, checked with the agents-tests suite (BindingsTests.hs’s deriveAgentTable group; NarrowingTests.hs’s derive_agent group against the loaded three-level tree) and live with agents-exe check. derive_agent {from, slug, bindings, with} is a new tool, registered by wireAgentTools alongside describe_agent for any agent with at least one helper: it checks from names a known, narrowable direct helper and, if so, echoes back {"stored": true, "from", "slug"} — nothing else happens, and nothing is stored beyond that call landing in the session’s own history. prompt_agent_<from> gains an optional as argument; at call time, System.Agents.Tools.Bindings.deriveAgentTable folds the current session’s turns (matching successful derive_agent calls by function name and response, the same “recomputed fresh, not persisted” treatment as ctxSessionToolCalls) into a Map (from, slug) DerivedNarrowing, exposed on a new ToolExecutionContext. ctxDerivedNarrowings field set in Session/Step.buildContext — so this works for every front-end without any wiring, no includeFullSession op- in needed. runSubAgent looks up (helper, as); an unknown name is an error the calling model can retry (naming derive_agent, per D8). When found, the derived bindings apply before the call’s own explicit ones, and the derived with fills what the call’s own with and the reference’s static with (§7) do not. A narrowable: false helper refuses derive_agent and as the same way it already refuses bindings/with.

Goal

Let whoever runs an agent pin down some arguments of its tools, so that the LLM neither sees nor chooses them. The motivating case: one bundled agent (say, a docker image with agents-server and a bash toolbox) serves several tenants, and each session carries its own tenant-id and API token.

The primitive is partial application, on purpose. A tool is a function from a JSON object to a result:

query-invoices : { tenant_id, token, since } -> Result

Binding tenant_id and token gives a smaller function, and that smaller function is the tool the LLM gets:

query-invoices' : { since } -> Result

Two things follow, and they are the whole feature:

Schema: a bound argument disappears from the tool’s JSON schema. The model is not told that it ever existed. That is the point: with most arguments already given, the model has less to reason about locally.
Call: the bound value is merged into the LLM’s argument object just before dispatch. The bound value always wins.

The deliverable is:

a generic combinator on ToolRegistration in agents-lib, so every kind of toolbox (bash, MCP, OpenAPI, PostgREST, builtin, sub-agents) gets the feature at once;
a way to declare an agent’s parameters and to bind tool arguments to them in the agent file;
ways to supply parameter values: at process start (agents-exe, agents-server) and transiently per session (agents-server);
a changed interface for helper agents exposed as tools, so that a parent can discover the open tool arguments of a helper and bind some of them when it prompts it, all the way down a chain of sub-agents (§8).

Scenarios

The design is checked against these three.

S1. One container, one tenant. A docker image holds agents-server, an agent file and a bash toolbox. The operator starts it with -e BILLING_TOKEN=… and --pin tenant=acme. Nobody who talks to the container can change either. (§4)

S2. A backend mints a credential and leads a chat. A product API receives “open an assistant on project 42” from a logged-in user. It mints a credential that is valid for project 42 only, creates a session on agents-server with params: {project_id: "42", callback_token: "…"} and seal: true, and hands the browser a session token. The agent’s tools are callbacks into the product API, with project_id and the Authorization header bound. Three defences stack up:

the model cannot hallucinate a project id: the argument is not in any schema, and a bound value wins over anything the model sends;
a prompt injection cannot retarget the tools, for the same reason;
enumeration fails even if the two above were bypassed, because the minted credential opens one project. The end user, who only holds the session token, cannot re-parameterise the session either.

(§3.3, §5, §5.1)

S3. A parent narrows its helpers, down the chain. A root agent splits a job and prompts a helper with extra bindings for that part (one repository, one date range), routing secrets by parameter name without ever seeing them. The helper may have helpers of its own: the root can bind arguments of tools anywhere below it, each agent on the way can narrow further, and none can undo what an ancestor bound. (§8)

Non-goals (for this spec)

Sandboxing. A bound token still reaches a bash script that the agent author wrote; we do not defend the value against the tool itself.
A general templating language in agent files (${…} expansion in prompts, paths, URLs). Parameters bind tool arguments, nothing else. See Decisions, D7.
Per-tenant agent definitions. The agent stays one static definition; only parameter values change between sessions.
Encrypting values at rest. Secret values are simply never written.

Current state

What we can reuse as-is

Piece	Where	Notes
One shape for all tools	`ToolRegistration{innerTool, declareTool, findTool, toolActivation}` (`ToolRegistration.hs:154`)	Schema is `declareTool.toolDescriptionParamProperties`; the call receives an `Aeson.Value`. A wrapper can rewrite both without knowing the toolbox kind.
A per-call context, already threaded everywhere	`ToolExecutionContext` (`Tools/Context.hs:238`), built per step in `Session/Step.hs:855`	The run function of every tool gets it.
A serializable part of that context	`ToolExecutionContextSnapshot` (`Tools/Context.hs:422`)	Persisted with deferred and async calls.
Value sources	`SecretSource = Given \\| ApiKeySource \\| FileSystem \\| EnvVar \\| Command` (`Tools/Secrets.hs:105`) and `resolveSecretSource` (`:343`)	Exactly what a parameter default needs.
Header/query serializers	`SecretSerializer` (`Secrets.hs:161`), `applySecretsToHeaders` (`:464`)	Applied per request already; only the resolution is load-time.
Bash argument marshalling	`translateArguments` (`ScriptTypes.hs:186`), `flattenArguments`/`flattenInput` (`Bash.hs:272`, `:284`)	Reads declared args from a JSON object. Merging bound values upstream needs no change here.
Single env funnel for bash tools	`buildToolEnvironment` (`Bash.hs:356`)
Session metadata row with reserved columns	`SessionMeta` (`SessionStore.hs:192`)	Home for non-secret session parameters.
Per-session entry points	`Runner.createSessionAs` (`Host/Runner.hs:510`), `postMessage` (`:530`), `Api.createH` (`Api.hs:340`)
Server self-description	`agentView` (`Api.hs:279`), `GET /openapi.json`	Where clients learn which parameters an agent wants.

Gaps

G1. Nothing flows from the caller to a tool call. agents-server reads agent, prompt, media, run and the bearer token; none of it reaches ToolExecutionContext. The CLI has no --set-like flag.

G2. Toolboxes are frozen at tree load. osNodeTools :: TVar [ToolRegistration] is built once per agent tree and shared by all sessions. Any per-session value must therefore arrive through the call context, not through the registration.

G3. HTTP toolbox secrets resolve once. OpenAPIToolbox.initializeToolbox (OpenAPIToolbox.hs:323) calls resolveSecrets at load. A per-session token is impossible.

G4. MCP servers cannot receive configuration. McpSimpleBinaryConfiguration has name, executable, args and no env. mcpTool (ToolRegistration.hs:1699) ignores the context entirely.

G5. Bash arguments are always required. mapArg (ToolRegistration.hs:262) sets propertyRequired = True and ignores argTypeArity. The skills toolbox already honours arity (Skills/Toolbox.hs:333). Not caused by this feature, but it makes “leave an optional argument out” impossible to express, so it is fixed here.

G6. Bash tools only take values through argv or stdin. A token in argv shows in ps and in traces. There is no calling mode that passes an argument as an environment variable.

G7. Traces record argv. RunCommandStart path args (Bash.hs:193) logs the full command line. A bound secret would land in log files.

G8. The tool cache ignores everything but the LLM’s arguments. computeCacheKey hashes tool name + LLM arguments (Tools/Cache.hs:71, Session/Async.hs:335). With bindings, tenant A’s cached result would answer tenant B’s identical call.

G9. Sub-agents receive nothing from their caller except the call stack and the OS world (OneShotTool.hs:191). A child agent with its own tenant-aware toolbox has no way to learn the tenant.

G10. A parent cannot narrow a helper. Helpers are the agents exposed to a parent as prompt_agent_<slug> tools (OneShotTool.turnAgentRuntimeIntoIOTool, wired by AgentTree.wireAgentTools, AgentTree.hs:724). The tool takes one argument, what (OneShotTool.hs:95). The parent cannot learn which tools a helper has or what arguments they take, and cannot fix any of them for a call. With a chain root → A → B, the root has even less say over B. One would expect narrowing to compose along the chain naturally; today there is nothing to compose. (The developer toolbox’s create-agent is unrelated: it bootstraps agent JSON files for a human and is left alone by this spec.)

Design

Overview

Three separate notions, kept apart deliberately:

  agent file                                  caller
  ──────────                                  ──────
  parameters:   declares names       ◄─────   supplies values
     tenant, api_token                         (process start, or per session)
        ▲
        │ refers to
  bindings:     tool argument := parameter | literal
     query-*.tenant_id := tenant

A parameter is a named hole in the agent. The agent author declares it. An agent with parameters is a function Params -> Agent.
A binding ties one tool argument to a parameter or to a literal. The agent author writes it, next to the toolbox.
A value is supplied by the caller for a parameter, never directly for a tool argument.

The indirection matters for safety: a remote client can fill tenant, and only tenant. It cannot decide to bind the path argument of some other tool. What is bindable is the author’s decision; what it is bound to is the caller’s.

1. Parameters

New optional field on Agent (Base.hs:1595):

"parameters": [
  { "name": "tenant",
    "description": "Tenant identifier, as in the billing system",
    "scope": "session" },
  { "name": "api_token",
    "description": "Bearer token for the billing API",
    "secret": true,
    "scope": "session",
    "default": { "tag": "EnvVar", "contents": "BILLING_TOKEN" } },
  { "name": "region",
    "scope": "process",
    "default": { "tag": "Given", "contents": "eu-west-1" } }
]

data ParameterDecl = ParameterDecl
    { paramName :: ParamName            -- [a-z][a-z0-9_]*
    , paramDescription :: Maybe Text
    , paramSecret :: Bool               -- default False
    , paramScope :: ParamScope          -- default ScopeProcess
    , paramRequired :: Bool             -- default True
    , paramDefault :: Maybe SecretSource
    }

data ParamScope = ScopeProcess | ScopeSession | ScopeMessage

scope: "process": only the operator can set it (CLI flag, environment, default). An HTTP client that tries gets 403. This is the docker case where the operator pins a tenant for the whole container.
scope: "session": a client may also set it per session; the value lasts as long as the session. Process-level values still act as the fallback.
scope: "message": a client may set it with a request that starts a run (create, message, resume, continuation); the value lasts for that run only and is dropped when the run blocks or completes. Meant for short-lived tokens. It is never persisted, secret or not (§5).
secret: true: the value must be a string; it is never persisted, never traced, never returned by the API (§6).
default reuses SecretSource, so a default can be a literal, an env var, a file, a command, or an entry of the API-keys file.

Values are JSON (Aeson.Value), because MCP and OpenAPI arguments are not always strings. Bash arguments need strings (§3.1).

Resolution order for one parameter, first hit wins:

pinned process value (--pin, §4): nothing below is looked at;
message value (only if scope = message);
session value (only if scope = session);
process value (--set, --params-file);
default, resolved once at tree load;
unbound.

An unbound required parameter is an error before any LLM call: at startup for a process-scope parameter, at session creation or message post for a session-scope or message-scope one (§5).

2. Bindings

Every toolbox description gains an optional Bindings list (same prefix-stripping convention as Activation). Example for a bash toolbox:

{ "tag": "FileSystemDirectory",
  "contents": {
    "Path": "tools",
    "Bindings": [
      { "arg": "tenant_id", "value": { "tag": "Param", "contents": "tenant" } },
      { "arg": "token",     "value": { "tag": "Param", "contents": "api_token" } },
      { "tool": "export-*", "arg": "format",
        "value": { "tag": "Literal", "contents": "csv" } }
    ] } }

data Binding = Binding
    { bindTool :: Maybe ToolGlob     -- matched against the toolbox-local name; default: all
    , bindArg :: Text
    , bindValue :: BindingValue
    , bindWhenUnbound :: WhenUnbound -- default Fail
    }

data BindingValue = Param ParamName | Literal Aeson.Value

data WhenUnbound
    = Fail     -- the call fails with a clear error (cannot happen for required params)
    | Omit     -- the argument is left out; only for optional arguments
    | Expose   -- (later, Phase 7) the argument goes back to the LLM

Literal is the degenerate case and is useful on its own: specialise a generic third-party tool without writing a wrapper script.

Load-time checks, all of them errors in agents-exe check:

a Param names an undeclared parameter;
a binding matches no tool, or no matched tool has that argument (typo guard). For MCP, whose tool list arrives late and may refresh, this is a warning trace at each ToolsRefreshed;
two bindings hit the same (tool, argument);
Omit on a required argument;
a secret parameter bound to a bash argument whose mode is positional, dashdashspace or dashdashequal (it would show in ps): an error, pointing to the env mode of §3.1.

3. The combinator

In a new module System.Agents.Tools.Bindings:

applyBindings :: [ResolvedBinding] -> ToolRegistration -> ToolRegistration

declareTool: drop bound ParamPropertys from toolDescriptionParamProperties, and nothing else. No “pre-set” note, no trace of the argument: what the model gets is indistinguishable from a tool that never had it (D8). A tool whose prose description talks about an argument that the author then binds is the author’s to reword; check warns when a bound argument’s name appears in the tool description.
innerTool / findTool: wrap the run function. At call time it reads parameter values from the context, builds the bound object, and calls the original with bound <> llmArgs (left-biased: bound wins). If the LLM sends a bound key anyway, it is dropped and a trace is emitted.

It is applied in ToolLoader.loadAgentTools (ToolLoader.hs:127), once per toolbox, after the toolbox produced its registrations. Because values come from the context at call time, the registrations stay shared across sessions (G2) and the schema stays static.

The context gains one field:

data ToolExecutionContext = ToolExecutionContext
    { …
    , ctxParams :: Params   -- Map ParamName ParamValue
    }

data ParamValue = ParamValue { pvValue :: Aeson.Value, pvSecret :: Bool }

Show and ToJSON of ParamValue print "<secret>" when pvSecret. Session/Step.buildContext fills ctxParams from the agent runtime (process values and defaults) overlaid with the session’s values (§5).

The session itself is unchanged: it records the call as the LLM made it, so bound values never enter the conversation history, never go back to the model, and do not show in session-print.

3.1 Bash tools

New calling mode env. ScriptArgCallingMode gains Env. An argument {"name": "token", "mode": "env"} is passed as the environment variable TOKEN (upper-cased, - to _), not in argv. flattenArguments and flattenInput ignore it; a new flattenEnv feeds buildToolEnvironment. Partial application stays a single concept (everything is an argument); the mode only picks the transport. This closes G6 and is useful without bindings too.
Arity (G5). mapArg sets propertyRequired = (argTypeArity == Single).
Strings only. A value bound to a bash argument must be a JSON string; numbers and booleans are rendered with their JSON text; objects and arrays are a load-time error for Literal, a call-time error for Param.
Tracing (G7). runValue gets the set of secret argument names from the wrapper and replaces their values with <secret> in RunCommandStart / RunCommandStopped. With the load-time check of §2, secrets are env-only anyway; the redaction is the second line of defence.
documentation/binary-tool.md and agents-exe spec bash-tools document the env mode.

3.2 MCP toolboxes

Argument bindings work through the generic combinator; adaptSchema (ToolRegistration.hs:2600) output is filtered like any other.
Separately, close G4: McpSimpleBinaryConfiguration gains env :: Maybe (Map Text BindingValue). The server process is started once per tree, so only Literal and process-scope Params are accepted here; a session-scope parameter is a load-time error with a message that says why.

3.3 OpenAPI and PostgREST toolboxes

Argument bindings (path, query and body parameters) work through the generic combinator.
Headers and tokens are not tool arguments, so they go through the existing secrets. SecretSource gains ParamSource ParamName. Secrets with that source are skipped by the load-time resolveSecrets and resolved per request from ctxParams, then serialized by the unchanged applySecretsToHeaders / applySecretsToQueryString. This closes G3.
Fetching the spec at load time cannot use a session-scope secret. If the spec endpoint needs auth, use a process-scope parameter or a plain secret for it; the loader says so when it fails.

3.4 Builtin toolboxes and sub-agents

Builtins need nothing special. For sub-agents see §7.

4. Supplying values: process level

Shared by agents-exe (all subcommands that load a tree) and agents-server:

--set NAME=VALUE          string value; repeatable
--set-json NAME=JSON      any JSON value
--params-file FILE        {"tenant": "acme", "api_token": "…"}
--pin NAME=VALUE          like --set, and clients cannot override it
--pin-json NAME=JSON

They feed a new processParams :: Map ParamName ProcessValue in AgentTree.Props (AgentTree.hs:384), where a ProcessValue is a value plus a pinned flag. --params-file entries may be written {"value": …, "pinned": true} to the same effect.
--set on a session- or message-scope parameter gives a fallback that a client may override. --pin gives a value that a client may not: for the lifetime of the process the parameter behaves as if it were declared scope: "process". This is how an operator takes a multi-tenant agent file and locks one container to one tenant without editing the file. On a process-scope parameter --pin and --set mean the same.
A name that no loaded agent declares is an error (typo guard).
For secrets, prefer a default with EnvVar / FileSystem, or --params-file: --set shows in ps. The docs say so.
agents-exe check prints each agent’s parameters and how each one resolved (set, default:EnvVar, unbound), never the values of secret ones.
agents-exe describe and describe-tool show the reduced schema, with the bound arguments listed apart (this output is for the operator, not for the model).
agents-exe tool-call and replay-tool-call go through the wrapped registration, so they apply bindings too.

In the docker case this is all that is needed: the image holds the agent file, the operator runs it with -e BILLING_TOKEN=… and --pin tenant=acme.

5. Supplying values: session level (`agents-server`)

Create. POST /v1/sessions accepts params:

{ "agent": "billing", "prompt": "…",
  "params": { "tenant": "acme", "api_token": "s3cr3t" } }

Follow-ups. POST /v1/sessions/:id/messages, /resume and POST /v1/continuations/:token accept the same field. It is merged over the session’s current values; null removes a value.

Validation, before the run starts:

Case	Answer
name not declared by the agent	`422`, lists the unknown names
`scope: "process"` parameter, or one pinned with `--pin`	`403`
`scope: "message"` parameter on a request that starts no run	`422`
`secret` with a non-string value	`422`
a required session or message parameter still unbound after merging	`422`, lists the missing names

Storage.

Non-secret values: a new params JSON column on the sessions table (both SQLite and Postgres backends; SessionMeta.smParams), written through the same versioned store as the rest of the metadata. They survive restarts and live-session eviction.
Secret values: only in the LiveSession (Host/Runner.hs:178), in memory. After a restart or after --live-session-ttl evicts the session, the next message must carry them again, or gets the 422 above. This is the “transient” in the goal, and it keeps tokens out of the database with no encryption story to design.
Message-scope values, secret or not: only in the run’s state inside the LiveSession, cleared when the run ends (blocked, completed, failed or cancelled). A required message-scope parameter must therefore come with every request that starts a run, including /resume and POST /v1/continuations/:token when it resumes. They are never in snapshots either.
ToolExecutionContextSnapshot gains tecsParams, holding non-secret values only. A deferred or async call that is re-hydrated in another process sees non-secret parameters; if its tool needs a secret one, the re-hydrated call fails with parameter api_token is not bound. recoverOnStartup (Runner.hs:700) therefore marks such running calls as failed rather than retrying blind.

Self-description. agentView adds

"parameters": [
  { "name": "tenant", "description": "…", "scope": "session",
    "secret": false, "required": true, "bound": false },
  { "name": "region", "scope": "process", "bound": true }
]

bound tells a client whether the process already supplies the value; "pinned": true says that it cannot be overridden (and the chat page then hides the field). No values are ever returned. GET /v1/sessions/:id returns the non-secret session values and the names of the secret ones currently held. Routes.hs and the generated OpenAPI document follow. The chat page gets a small plain form for session parameters, shown when the chosen agent has unbound ones (password inputs for secret ones).

Stored agents. parameters is a normal field for database agents (Host.putStoredAgent). Bindings lives inside toolbox descriptions, so it follows whatever fileBasedFields already allows.

MCP over HTTP (POST /mcp, the ask_<slug> tools). MCP clients are usually configured with static headers, which is exactly the pin-down case:

Agents-Param-<name>: <value> request headers, string values;
_meta: {"agents-exe/params": {…}} on tools/call, any JSON, wins over headers.

Same validation as above, reported as a JSON-RPC error. Each tools/call is one run, so session and message scope coincide here.

5.1 Sealed sessions and delegated chat (S2)

In S2 two different principals talk to the same session: the backend, which is trusted to choose project_id, and the end user’s browser, which is not. Today the server knows one kind of caller, the bearer-token owner (AgentsServer/Auth.hs), and the backend cannot give its token to a browser. Without more, the backend has to proxy every chat message and SSE stream. Three additions remove that need:

seal: true on POST /v1/sessions. After creation, params on /messages, /resume and continuations is refused with 403, for every caller but the owner. The parameters of a sealed session are those its creator chose.
Session tokens. POST /v1/sessions with "session_token": true returns a random token bound to that session. Authorization: Bearer <session token> allows exactly: post a message, read the session and its events, cancel a run, on that one session. It cannot set or read params, list sessions, complete continuations, or touch agents. Tokens are stored hashed, next to the session, and die with it; the owner can revoke one with DELETE /v1/sessions/:id/token.
PUT /v1/sessions/:id/params, owner only, sets session-scope values without starting a run. The backend uses it to rotate a minted credential before it expires, and to re-supply secrets after a restart or an eviction (secrets are memory-only, §5). When a message arrives on a session whose required parameters are no longer all bound, the answer is 409 params_required with the missing names; a browser holding a session token relays that to its backend, which calls this endpoint.

5.2 Forking a session with new values

A credential that expires in the middle of a long run is not handled: the tool calls fail, the model reports the failure, the run ends. No blocking, no refresh protocol (D14). The way forward is a new session that starts from the old one’s history with a fresh value pinned:

POST /v1/sessions
{ "fork": "<session id>",
  "params": { "callback_token": "…fresh…" },
  "seal": true, "session_token": true,
  "prompt": "The credential was renewed, please carry on." }

The new session gets a copy of the turns of the source, a new session id, and forkedFromSessionId set (the field exists already, Session/Types.hs:1385). agent is taken from the source; prompt is optional, and without it the session is created idle.
Parameters are not shared with the source at run time. The new session starts from the source’s persisted (non-secret) values, overlaid with the request’s params, and is validated like any creation; secrets always have to be given again. seal and session tokens are not inherited either.
This is sound because bound values never enter the history (§3): the copied turns hold no trace of the old token or of any bound argument, so nothing in them contradicts the new values. Changing project_id in a fork is therefore possible and is the owner’s call; it is refused to session-token holders, who cannot fork at all.
Pending deferred calls and running async calls of the source are not copied. The fork’s last turn is closed the way a cancelled run closes it, so the history stays well-formed for the LLM.
Narrowed helper calls (§8) need nothing: their bindings are arguments of prompt_agent_* calls in the copied history, and a Param in a future call resolves against the fork’s values, which is the intent.
Only the owner of the source (or an admin owner) may fork it.
On the CLI the same thing is agents-exe session start --fork <id> with the usual --set / --pin flags.

The callback tools themselves need nothing new. With an OpenAPI toolbox pointing at the product API:

{ "tag": "OpenAPIServer",
  "contents": {
    "SpecUrl": "https://api.example.com/agent-callbacks/openapi.json",
    "BaseUrl": "https://api.example.com/agent-callbacks",
    "Secrets": [
      { "source": { "tag": "ParamSource", "contents": "callback_token" },
        "decoder": { "tag": "Clear", "contents": true },
        "serializer": { "tag": "Header",
                        "contents": ["Authorization", "Bearer {{secret}}"] } } ],
    "Bindings": [
      { "arg": "project_id",
        "value": { "tag": "Param", "contents": "project_id" } } ] } }

Every operation that has a project_id path or query parameter loses it; the model sees list_tasks {status} instead of list_tasks {project_id, status}. The same works with bash tools that curl the API, with token in env mode.

One thing stays with the product API: its error bodies go back to the model verbatim. An API that answers “project 43 belongs to another account” leaks through any amount of binding.

6. Secrecy rules

One list, because they are scattered over the sections above:

A secret value is never written: not in the session JSON, not in the metadata, not in continuation snapshots, not in the tool cache.
A secret value is never traced. ParamValue redacts itself; bash argv is redacted (§3.1); the HTTP toolboxes already keep secret headers out of --log-http and that path is reused.
A secret value is never returned by the API nor shown by check / describe.
A secret value never reaches bash argv (§2 load-time check).
None of this protects the value from the tool that receives it. A tool that echoes its token sends it to the LLM. That is the tool author’s responsibility and the docs say so.

Tool cache (G8), done. Implemented as CacheScope/cacheScopeOf in Tools/Cache.hs: a call’s key also depends on the digest of the agent’s non-secret parameter values and inherited narrowing bindings (the context’s ctxParams/ctxInheritedBindings, at every cache call site), and a call with any secret one is not cached. Deviation: this looks at all the agent’s parameters, not only those a tool binds (the registrations’ bindings are closed over in the tools and invisible where the cache is consulted), so it can only cost hits, never share a result across bound values. Argument hashes are now SHA-256 (the old length + 60-character prefix could collide). As designed: computeCacheKey takes the bound values of the call: non-secret values are hashed into ckArgumentsHash together with the LLM arguments. A call with any secret binding is not cached at all: hashing a token into a persisted key is a needless oracle, and results obtained with one credential should not be replayed for another.

7. Sub-agents

Lexical, not dynamic: a child sees only what the parent passes explicitly. ExtraAgentRef (Base.hs:93) gains with:

"extraAgents": [
  { "slug": "invoice-expert", "path": "invoice-expert.json",
    "with": { "tenant": { "tag": "Param", "contents": "tenant" },
              "mode":   { "tag": "Literal", "contents": "read-only" } } }
]

Keys are the child’s parameter names; values are BindingValues in the parent’s scope.
runSubAgent (OneShotTool.hs:191) evaluates with against the caller’s ctxParams, at call time, so a message-scope value of the parent is passed like any other, and hands the result to the child as its session-level values; the child’s own process values and defaults still apply underneath. Secrecy is inherited: a value that is secret in the parent is secret in the child, whatever the child declares.
Children discovered through a tool directory have no reference site, so they get process values and defaults only. List them in extraAgents to pass something.
The check phase verifies that with keys exist in the child and that the child’s required session parameters are covered.

8. Narrowing helpers, down the call chain

The scenario (S3): a parent splits a job and prompts a helper whose tools are already pinned to one part of it, so that the helper’s model has nothing left to choose. This is partial application done by a model instead of an operator, and it composes: the operator binds tenant, the root binds repo, helper A binds branch, helper B only picks path.

This concerns helper agents exposed as tools, and only them. Their interface changes in two ways: (a) a parent can discover the open arguments below a helper, (b) a parent can bind some of them when it prompts the helper.

8.1 Agent-level bindings

Toolbox-level bindings (§2) need a toolbox to hang on, and bash toolboxes have no name. So Agent also gets a top-level bindings list, where tool globs match the LLM-visible names (bash_query-*, mcp_billing_*). They are applied after the toolbox-level ones. §8.3 uses the same shape at run time.

Successive application is monotone by construction: a later binding can only name an argument that is still open. There is no “unbind” and no override.

8.2 (a) Discovery: `describe_agent`

One new tool next to the prompt_agent_<slug> family, registered by wireAgentTools for any agent that has helpers:

describe_agent { "slug": "repo-worker" }

{ "slug": "repo-worker",
  "announce": "works on one git repository",
  "parameters": [ { "name": "gh_token", "description": "…", "bound": false } ],
  "tools": [
    { "name": "bash_git-log", "description": "…",
      "open": [ { "arg": "repo", "type": "string", "description": "…" },
                { "arg": "since", "type": "string", "description": "…" } ] } ],
  "helpers": [
    { "slug": "diff-reviewer",
      "tools": [ { "name": "bash_git-diff",
                   "open": [ { "arg": "repo", … }, { "arg": "rev", … } ] } ],
      "helpers": [] } ] }

It shows open arguments only, as seen from the caller’s position in the chain: what the agent file bound, what the operator bound, and what any ancestor of the caller bound for this call are all absent. A parent discovers exactly what is left for it to decide, in keeping with D8.
It is recursive over helpers, cut at the configured max depth and at cycles (isAgentInCallStack, Tools/Context.hs:767).
Parameters are listed by name and description, never by value.
It is a read of the loaded tree, not an LLM call. The answer is the same for the whole run, so it caches well.
An author can opt a helper out with "narrowable": false on its extraAgents entry: describe_agent then shows only its announce, and bindings that target it or anything below it are refused.

8.3 (b) Binding: `prompt_agent_<slug>` takes `bindings`

prompt_agent_repo-worker
{ "what": "Summarise last week's changes to the session runner.",
  "bindings": [
    { "tool": "bash_git-*", "arg": "repo",
      "value": { "tag": "Literal", "contents": "lucasdicioccio/agents-exe" } },
    { "agent": "diff-reviewer", "tool": "bash_git-diff", "arg": "repo",
      "value": { "tag": "Literal", "contents": "lucasdicioccio/agents-exe" } },
    { "agent": "**", "tool": "bash_gh-*", "arg": "token",
      "value": { "tag": "Param", "contents": "gh_token" } } ],
  "with": { "tenant": { "tag": "Param", "contents": "tenant" } } }

bindings and with are optional; the schema of the tool is static (D3) and a parent that ignores them sees today’s behaviour.

Addressing. agent is a path of slugs below the prompted helper: absent means the helper itself, diff-reviewer its helper of that name, a/b a grandchild, ** the helper and everything below. tool and arg are as in §2.
Values are resolved at the call site. Literal is what the parent’s model wrote. Param names a parameter of the calling agent and is resolved against its ctxParams when the call is made (lexical, as in §7); secrecy travels with the value. The parent’s model routes a token it has never seen. with fills the helper’s parameters as in §7, and cannot refill one that the extraAgents entry already fills.
How they travel. ToolExecutionContext gains ctxInheritedBindings :: [ScopedBinding], a resolved binding plus the remaining agent path. runSubAgent (OneShotTool.hs:191):
1. takes the caller’s inherited bindings, re-rooted at the helper, and adds the ones from this call;
2. applies those addressed to the helper itself, wrapping the helper’s registrations with applyBindings for this call only. The loaded toolboxes stay shared: no script is re-described, no MCP server restarted;
3. puts the rest in the helper’s context, from where step 1 picks them up when the helper prompts its helpers. The non-secret part goes in ToolExecutionContextSnapshot, like parameters.
The chain composes, and only narrows. Every agent on the way may add bindings for what is below it. It can only name arguments that are still open from its position, which is all describe_agent shows it. If a model names an argument that an ancestor already bound (it cannot know about it, so this is a hallucination), the ancestor’s value stays, the new binding is dropped, and a trace is emitted; the call goes on (D8).
Errors the parent’s model can fix are told to it: unknown agent path, no tool matching the glob, no such open argument, wrong value type, a narrowable: false target. The call fails before the helper runs, with a message that names the open arguments, so one retry suffices.
Literals bound to env-mode arguments are allowed. Nothing makes a model-chosen literal secret.
Knowing parameter names is not knowing arguments. D8 is about the surface of the tools an agent calls. An agent that narrows other agents works one level up and sees configuration: names and descriptions, never values. An operator who does not want even that pre-binds the argument in the helper’s file: bound arguments are not shown.
Recovery. The bindings of a call are in the parent’s history, as the arguments of its prompt_agent_* call, so a resumed or forked (§5.2) parent re-issues them as they were; Param values resolve against the current values, which is the intent after a fork.
Tool cache. Inherited non-secret bindings enter the helper’s cache keys as in §6; any secret one disables caching for the calls it touches.

8.4 Naming a narrowing (optional convenience)

Repeating the same bindings on ten prompts costs tokens and invites inconsistency. derive_agent {from, slug, bindings, with} stores a narrowing under a slug for the session, and prompt_agent_<from> accepts "as": "<slug>" to use it. It is sugar over §8.3 and adds no power: the stored thing is the same data, the table is a fold over the session’s successful derive_agent calls (in the manner of ToolboxSessionState, Tools/Activation.hs:75), so nothing new is stored. To be built only if §8.3 proves verbose in practice.

Phases

Each phase is shippable and leaves agents-exe check and the test suite green.

Phase 1: the combinator, literals, bash fixes

System.Agents.Tools.Bindings: Binding, BindingValue, applyBindings, schema reduction, argument merge.
Bindings field on all toolbox descriptions; Literal only.
Bash: arity in mapArg (G5), env calling mode (G6).
Load-time checks of §2 that do not involve parameters.
Tests: schema reduction; bound wins over an LLM-sent key; glob matching; env mode end to end with a fixture script.

Phase 2: parameters at process level

ParameterDecl, parameters on Agent, Param binding values.
ctxParams, ParamValue with redaction, buildContext wiring.
--set, --set-json, --params-file, --pin, --pin-json; Props.processParams; defaults through resolveSecretSource.
check, describe, describe-tool, tool-call output.
Bash argv redaction (G7) and the secret-in-argv check.
Cache key (G8).
MCP env (G4).

After this phase the docker use case works with one tenant per container.

Phase 3: HTTP toolboxes

ParamSource in SecretSource, per-request resolution (G3).
Same for PostgREST.

Phase 4: session-level parameters in `agents-server`

params on create / messages / resume / continuations, validation table.
params column and migration (SQLite and Postgres), smParams.
Secrets in LiveSession only; tecsParams; recovery behaviour.
Message scope: per-run values, cleared at run end.
seal, session tokens, PUT /v1/sessions/:id/params, 409 params_required (§5.1).
fork on session creation and session start --fork (§5.2).
Pinned parameters refused with 403; pinned in agentView.
agentView, session view, OpenAPI document, documentation/agents-server.md.
Chat page form.

After this phase one container serves many tenants.

Phase 5: sub-agents and MCP over HTTP

with on ExtraAgentRef, evaluation in runSubAgent, checks (G9).
Agents-Param-* headers and _meta on /mcp.

Phase 6: narrowing helpers

Agent-level bindings (§8.1), with the still-open check.
describe_agent (§8.2), recursive, position-aware; narrowable.
bindings and with on prompt_agent_<slug>; ctxInheritedBindings; per-call wrapping in runSubAgent; snapshot; cache keys (§8.3).
Tests on a three-level chain: root binds for the grandchild; the middle agent narrows further; the middle agent cannot rebind; describe_agent from the middle does not show what the root bound; resume and fork.
Later, if needed: derive_agent (§8.4).

Depends on Phases 1, 2 and 5 (with).

Phase 7 (optional): `Expose`

whenUnbound: "expose" puts the argument back in the schema when the parameter is unbound for this session. It makes the tool list a function of the session, which the progressive-disclosure machinery (Combinators/ProgressiveDisclosure.hs) already does for activation, so the hook exists. Deferred until a real use shows up.

Decisions

D1. Callers fill parameters, not tool arguments. The author decides what is bindable. Otherwise any client of agents-server could pin any argument of any tool, and every toolbox would need its own allow-list.

D2. Values travel in the call context, not in the registration. Toolboxes load once per tree and are shared (G2). Rebuilding a toolbox per session would mean re-running describe on every script and restarting MCP servers.

D3. The schema is static. A bound argument is hidden whether or not a value is present; a missing value is an error, not a fallback to the LLM. It keeps the tool list identical across sessions (prompt caching, simpler reasoning about what the model can do). Expose is the opt-in exception, postponed.

D4. env is a calling mode, not a second kind of binding. One concept (arguments) with a transport choice, rather than “argument bindings” plus “environment bindings” with different rules. The exception is the MCP server process environment, which is not a tool argument at all.

D5. Secrets are never persisted. The cost is that clients re-send them after eviction or restart. The alternative needs key management, and an agents-server database that holds customer tokens is a much more sensitive asset than one that does not.

D6. Sub-agent passing is explicit. Dynamic scoping (“the child inherits every parameter with the same name”) is shorter to write and leaks a token to any child that happens to declare the same name.

D7. No templating. ${tenant} in a system prompt or a path is tempting and is a different feature: it changes the agent definition per session and puts values in front of the model. Parameters here are deliberately invisible to the LLM.

D8. The model does not know. A partially applied argument leaves no mark in what the model sees: not in the schema, not as a “pre-set” note in the description, not in error messages. The purpose of the feature is to shrink the local reasoning a call requires; telling the model about a key it cannot use, or a tenant it cannot choose, adds reasoning back and invites it to try. Consequently, if the LLM sends a bound key anyway, the call proceeds with the bound value and only a trace is emitted; and the error for an unbound parameter at call time goes to the operator’s trace, while the model gets a generic “tool unavailable” failure.

D9. Pinning is an operator flag, not an agent-file field. The agent file says what may vary per session; the deployment says what does. The same file then serves both the shared server and the one-tenant container.

D10. Message scope means one run. Not one HTTP request, not one LLM turn: a run is the unit after which control goes back to the client, which is the only moment a client can supply a fresh token.

D11. Narrowing composes along the chain, and only narrows. Each agent may bind what is still open below it; none may rebind. Whatever a narrowed helper can do, the un-narrowed one could, so giving this power to a model cannot widen anything, which is why it is on by default (narrowable: false opts out).

D12. Bindings ride on the call, not on a new kind of agent. The helper interface gains discovery and a bindings argument, rather than a registry of derived agents. A call is self-describing, lives in the parent’s history, and needs no lifetime rules. Naming a narrowing (§8.4) is sugar.

D13. Session tokens belong to this spec. They look like a separate authentication feature, but without them S2 either proxies everything through the backend or hands the browser a token that can re-parameterise the session, which undoes the point of binding.

D14. An expired credential is a failed run, and the remedy is a fork. No mid-run refresh, no “blocked on credential” state. PUT …/params covers rotation between runs, and a fork with a fresh value covers a run that died. Forking is cheap to specify precisely because bound values are absent from the history.

D15. No saved narrowings for now. It is kept possible, not planned: a narrowing is plain serializable data (helper slug, scoped bindings, with) and has the shape of an agent-level bindings list (§8.1), so a later “save this helper” writes that data to the agent store and resolves the reference at load. Nothing here should tie a narrowing to the in-memory identity of a session. It stays secondary to a good experience for the user and for the model.

Nice to have, later

Saving a narrowed helper as a stored agent (D15).
derive_agent, naming a narrowing within a session (§8.4).
whenUnbound: "expose" (Phase 7).

documentation/binary-tool.md: bash tool protocol, to be extended with env mode.
documentation/tools.md, documentation/advanced-configuration.md: toolbox configuration.
documentation/agents-server.md: HTTP API.
todos/web-server-embedding.md: the host, the runner, session metadata.
todo: “partial-application of tool values”.

Durable Workflows — Implementation Progress

2026-10-04T15:53:54Z

Generated from todos/durable-workflows.progress.md, the repository is the canonical source and may be ahead of this page.

Durable Workflows — Implementation Progress

Progress tracker for todos/durable-workflows.md.

Phase 1 — Core primitives (types & interfaces) ✅ COMPLETE

1.1 Rich tool-call state inside a turn

Added durable workflow primitives to System.Agents.Session.Types:
- ToolCallId — stable UUID-based identifier for matching external results
- ToolCallState — Ready | Running | Deferred | Completed | Failed
- TrackedToolCall — wraps LlmToolCall with id, state, result, continuation token, and applied policy
- AppliedPolicy — records the policy decision applied to a call
Refactored PartialUserTurnContent to hold [TrackedToolCall] instead of separate completed/pending/continuation lists.
Added backward-compatible JSON parsing for legacy sessions and helper views:
- partialCompletedResponses
- partialPendingCalls
- partialPendingContinuations
- cacheKeyForTrackedCall
Updated dependent modules (Session.Step, Session.Loop, Session.OpenAI, Session.Search.Index, Session.Edit) to use the new model.

1.2 Tool-call policy / decorator

Added to System.Agents.Session.Types:
- ToolCallDisposition — RunSync | RunAsync | RunIsolated IsolationSpec | Defer Reason | Decorate [Decorator] ToolCallDisposition
- IsolationSpec — Docker Text | LocalProcess FilePath | FunctionRunner Text
- Reason — human-readable reason for defer/isolation
- Decorator — WithTimeout | WithRetries | WithCache | WithLabel
Added ToolCallPolicy type alias in new System.Agents.Session.Durable module.
Provided defaultToolCallPolicy = const $ const RunSync for backward compatibility.
Added flattenDisposition helper to extract decorators from a disposition tree.

1.3 Pluggable executor interface

Created System.Agents.Session.Durable with:
- ToolExecutor record (execSync / execAsync)
- inProcessExecutor — current behavior
- yieldingExecutor — always returns ToolYield
- cachingExecutor — cache wrapper
- isolatedExecutor — delegates RunIsolated calls to a DeploymentRunner
- DeploymentRunner and IsolationError abstractions
Updated System.Agents.Session.Base.Agent:
- Added ctxToolCallPolicy, ctxToolExecutor, ctxContinuationStore, ctxDeploymentRunner
- Removed ctxAsyncToolCall
- Added combinators: withToolCallPolicy, withToolExecutor, withContinuationStore, withDeploymentRunner
Updated all agent construction sites (OneShot, MCP.Server, AgentTree.OneShotTool) to initialize the new fields.

Verification

Library builds with -Wall -Werror.
Test suite agents-tests passes.

Phase 2 — Async scheduler in the session loop ✅ COMPLETE

2.1 Policy-driven async scheduler

Updated runStepMAsync / executeTrackedCalls in System.Agents.Session.Step to:
- classify every Ready TrackedToolCall via Agent.ctxToolCallPolicy;
- execute all RunSync / RunIsolated calls in one batch (with cache lookup);
- move RunAsync and Defer calls to the Deferred state, generate continuation tokens, and store snapshots.
If all calls complete, the scheduler emits a full UserTurn; otherwise it yields a PartialUserTurn.
Updated naiveStep and naiveTilNoToolCallStep so partial turns with remaining ready/deferred calls continue via AskUserPrompt rather than prematurely asking the LLM.

2.2 Serializable continuation snapshots

Added ToolExecutionContextSnapshot to System.Agents.Tools.Context containing only serialisable fields (ctxSessionId, ctxConversationId, ctxTurnId, ctxCallStack, ctxAllowedTools, ctxParentConversation).
Added ToolContinuationSnapshot to System.Agents.Session.Async and a helper mkToolContinuationSnapshot.
Deduplicated ContinuationToken by re-exporting the one from System.Agents.Session.Types in System.Agents.Session.Async.

2.3 Wake / resume primitives

Created System.Agents.Session.Wake with:
- wakeSession :: Session -> [(ContinuationToken, UserToolResponse)] -> IO Session
- wakeSessionWithCache :: Maybe ToolCache -> Session -> [(ContinuationToken, UserToolResponse)] -> IO Session
- resumeSession :: ConversationId -> Agent r -> Session -> IO (Either r Session)
wakeSession finds the latest PartialUserTurn, matches deferred calls by token, moves them to Completed, updates the cache when provided, and converts the turn to a full UserTurn when every call is complete.
resumeSession runs the agent step-by-step until it either completes (Left r) or yields again because deferred calls remain (Right Session).

2.4 SQLite `ContinuationStore` load/list

Finished sqliteLoadContinuation and sqliteListPending with proper JSON round-tripping of ToolContinuationSnapshot.
Changed ContinuationStore to store/load ToolContinuationSnapshot instead of the in-memory ToolContinuation.
Updated resumeAsyncToolCall to use the snapshot’s cache key.

Verification

Library builds with -Wall -Werror.
Test suite agents-tests passes, including new DurableWorkflowTests covering policy classification, continuation-store round-tripping, wake/resume, cache integration, and snapshot serialisation.

Phase 3 — Durable session storage ✅ COMPLETE

3.1 Generalized `SessionBackend` interface

Added SessionBackend record to System.Agents.SessionStore:
- sbStore :: SessionId -> Session -> IO ()
- sbLoad :: SessionId -> IO (Maybe Session)
- sbList :: IO [(SessionId, UTCTime)]
- sbDelete :: SessionId -> IO ()
Added sessionIdToConversationId / conversationIdToSessionId conversion helpers.

3.2 Concrete backends

FileSessionStore FilePath:
- Newtype wrapper and mkFileSessionStore constructor.
- Implements SessionBackend by reusing the existing file-based SessionStore logic (mkSimpleSessionStore, storeSession, readSession, findSessionFiles).
SqliteSessionStore Connection:
- Newtype wrapper and mkSqliteSessionStore constructor.
- Stores sessions as JSON in a sessions table with session_id, created_at, updated_at, and json columns.
- initializeSessionSchema creates the table and an index on updated_at.
CompositeSessionStore [SessionBackend]:
- Newtype wrapper and mkCompositeSessionStore constructor.
- Reads fall back across all backends in order.
- Writes (store/delete) go to the first backend only (primary target).

3.3 `Agent` integration

Added ctxSessionBackend :: Maybe SessionBackend to Agent in System.Agents.Session.Base.
Added withSessionBackend :: SessionBackend -> Agent r -> Agent r combinator.
Updated System.Agents.Combinators.StoreSessionProgress:
- Added backendStoreCallback for storing progress via a SessionBackend.
- agentStoreSession now uses ctxSessionBackend when present, falling back to the provided file SessionStore when absent.
- The optional explicit FilePath still receives an additional copy in both cases.
Updated all Agent construction sites (OneShot, MCP.Server, AgentTree.OneShotTool) and the test helper to initialize ctxSessionBackend = Nothing.

3.4 Tests

Extended DurableWorkflowTests with a Phase 3 group covering:
- File backend store/load/delete/list round-trip.
- SQLite backend store/load/delete/list round-trip.
- Composite backend read fallback, primary-only write, primary-only delete, and aggregated listing.
- withSessionBackend + agentStoreSession integration ensuring the backend is used and the file store is bypassed.

Verification

Library builds with -Wall -Werror.
Test suite agents-tests passes, including all Phase 2 and Phase 3 durable-workflow tests.

Phase 4 — Wake / resume API ✅ COMPLETE

4.1 Inject external results

wakeSession and wakeSessionWithCache are implemented in System.Agents.Session.Wake (introduced in Phase 2).
They find the latest PartialUserTurn, match deferred calls by token, move them to Completed, update the cache, and convert the turn to a full UserTurn when all calls are complete.

4.2 Resume execution

resumeSession is implemented in System.Agents.Session.Wake (introduced in Phase 2).
It resumes a session from a partial or completed user turn, running the scheduler until completion or until deferred calls remain.

4.3 Complete continuations from external workers

resumeAsyncToolCall uses the SQLite ContinuationStore with proper JSON round-tripping of ToolContinuationSnapshot.
csLoad / csListPending implementations are complete (Phase 2).

Note

Phase 4’s core functions were already implemented during Phase 2. CLI exposure (Phase 7) is out of scope for this phase.

Verification

Library builds with -Wall -Werror.
Test suite agents-tests passes, including Phase 2 wake/resume tests.

Phase 5 — Isolated deployment primitives ✅ COMPLETE

5.1 `DeploymentRunner` abstraction

Moved DeploymentRunner and IsolationError to a new module System.Agents.Session.Isolation.
Changed DeploymentRunner to accept a stable IsolationEnvelope instead of raw IsolationSpec + LlmToolCall, so external workers receive a language-agnostic document.
System.Agents.Session.Durable re-exports the runner/error types and the new envelope helpers for backward compatibility.

5.2 Serialization contract for isolated calls

Added stable envelope types in System.Agents.Session.Isolation:
- IsolationEnvelope — input envelope with token, toolCall, contextSnapshot, policy, and optional reason.
- IsolationResultEnvelope — result envelope with token, status (success/error), result, and error.
- IsolationResultStatus — IsolationSuccess / IsolationFailure.
Added helpers:
- mkIsolationEnvelope
- mkIsolationSuccessEnvelope
- mkIsolationErrorEnvelope
- parseIsolationResultEnvelope
- parseIsolationResultEnvelopeLBS
Documented the JSON shapes in Haddock comments.

5.3 Concrete runners

localProcessRunner :: FilePath -> DeploymentRunner
- Serialises the envelope to compact JSON.
- Forks the worker process and writes the envelope to its stdin.
- Reads stdout and parses the result envelope.
- Validates that the result token matches the input token.
- Returns IsolationError for non-zero exit codes, unparseable output, or token mismatches.
dockerRunner :: Text -> DeploymentRunner
- Runs docker run --rm -i <image> with the envelope on stdin. Reads stdout and parses the result envelope.
- Returns IsolationError if Docker is unavailable or the container fails.
functionRunner :: DeploymentRunner
- Documented placeholder for future serverless/FaaS execution.
- Always returns IsolationError "functionRunner is a future placeholder ...".

5.4 Integration with the policy-driven scheduler

Updated isolatedExecutor in System.Agents.Session.Durable to build an IsolationEnvelope for each isolated call (fresh continuation token + serialisable context snapshot).
Updated executeCall in System.Agents.Session.Step so that when an agent has ctxDeploymentRunner configured but no explicit ctxToolExecutor, isolated calls are dispatched through the runner and non-isolated calls fall back to the agent’s toolCall.
This makes RunIsolated policy decisions transparent to the session loop.

5.5 Example isolation-by-tool-name policy

Added isolatedToolNamePolicyTest showing:
- bash_command isolated via localProcessRunner.
- Other tools executed synchronously in-process.
- The scheduler produces a full UserTurn containing both responses.

5.6 Tests

Extended DurableWorkflowTests with a Phase 5 group covering:
- IsolationEnvelope JSON round-trip.
- IsolationResultEnvelope JSON round-trip and parser helpers.
- localProcessRunner with a simple bash worker script.
- dockerRunner envelope construction (execution skipped when Docker is unavailable).
- Integration test for isolation-by-tool-name policy.

Verification

Library builds with -Wall -Werror.
Test suite agents-tests passes, including all Phase 2, Phase 3, Phase 4, and Phase 5 durable-workflow tests.

Phase 6 — Integration & agent combinators ✅ COMPLETE

6.1 `Agent` record defaults

Verified that System.Agents.Session.Base.Agent exposes all durable-workflow fields:
- ctxToolCallPolicy :: ToolCallPolicy (defaults to defaultToolCallPolicy)
- ctxToolExecutor :: Maybe ToolExecutor (defaults to Nothing)
- ctxContinuationStore :: Maybe ContinuationStore (defaults to Nothing)
- ctxDeploymentRunner :: Maybe DeploymentRunner (defaults to Nothing)
- ctxSessionBackend :: Maybe SessionBackend (defaults to Nothing)
Confirmed all existing agent construction sites (OneShot, MCP.Server, AgentTree.OneShotTool) still compile unchanged and initialise every field.

6.2 Agent combinators

Added/verified all requested combinators in System.Agents.Session.Base:
- withToolCallPolicy
- withToolExecutor
- withContinuationStore
- withDeploymentRunner
- withSessionBackend
- Convenience withDurableWorkflows :: SessionBackend -> ContinuationStore -> Agent r -> Agent r
- Convenience withAsyncConfig :: ExecutionMode -> Maybe ToolCache -> ToolCallPolicy -> Agent r -> Agent r
- Convenience withDurableExecutor :: Maybe ToolCache -> Maybe DeploymentRunner -> Agent r -> Agent r

6.3 Progress / storage helpers

Added backendWithCallbackStoreCallback :: SessionBackend -> OnSessionProgress -> OnSessionProgress to System.Agents.Combinators.StoreSessionProgress.
- Stores each progress event via the configured backend and then forwards the event to an additional user-supplied callback.
Verified agentStoreSession still correctly wires ctxSessionBackend, file fallback, and optional extra file copy.

6.4 Durable executor helpers

Verified System.Agents.Session.Durable provides:
- mkDurableExecutor — cache + runner + native tool call
- cachedInProcessExecutor — in-process execution with caching
- composeExecutors — conditional dispatch by disposition

6.5 Integration tests

Fixed and extended test/DurableWorkflowTests.hs Phase 6 group:
- durableWorkflowIntegrationTest — end-to-end session with a policy that defers calls, a continuation store, a SQLite session backend, and a cache; verifies yield, persistence, wake, cache population, and resume.
- withDurableWorkflowsTest
- withAsyncConfigTest
- mkDurableExecutorTest
- cachedInProcessExecutorTest
- composeExecutorsTest
- agentStoreSessionWithCallbackTest

Verification

Library builds with -Wall -Werror.
Test suite agents-tests passes, including all Phase 2, Phase 3, Phase 4, Phase 5, and Phase 6 durable-workflow tests.

Phase 7 — CLI / operator API ✅ COMPLETE

7.1 New `session` subcommand group

Added System.Agents.CLI.SessionDurable module with:
- SessionDurableOptions and SessionDurableCommand types.
- handleSessionDurable dispatcher.
Integrated the new command group into app/Main.hs under agents session <subcommand>.
Added uuid to the agents-exe executable build-depends for parsing session/token UUID arguments.

7.2 `agents session pause <session-id>`

Loads the session from the configured file-based SessionStore.
Loads the first supplied agent file (same mechanism as one-shot) and sets execution mode to Asynchronous.
Runs one step via runStepM.
Persists the resulting session (completed or yielded) back to the SessionStore.
Prints the yielded state and any deferred continuation tokens.

7.3 `agents session resume <session-id>`

Loads the session and agent as above.
Calls resumeSession from System.Agents.Session.Wake.
Runs until completion or until the session yields again, persisting after each yield.
Prints the final LLM response or the yielded partial-turn summary.

7.4 `agents session pending <session-id>`

Loads the session from the SessionStore.
Finds the latest PartialUserTurn.
Lists all deferred TrackedToolCalls with their ToolCallId, ContinuationToken, tool name, and policy/disposition.

7.5 `agents session complete <token> <result-file>`

Reads the result file. If it parses as a UserToolResponse JSON value, that value is used; otherwise the raw file contents are treated as a plain-text response.
Scans all sessions in the SessionStore to locate the session containing the matching continuation token.
Calls wakeSession with the token/result pair.
Persists the updated session.
Prints whether the turn is now complete or still partial.

7.6 `agents session run-isolated <session-id>` (optional)

Loads the session and agent.
Finds deferred calls whose disposition is RunIsolated.
If the agent has a configured ctxDeploymentRunner, executes those calls through the runner and injects the results via wakeSession.
Persists the updated session and prints per-call results/errors.
If no runner is configured, reports the number of deferred isolated calls found.

7.7 Pure helpers and tests

Exported pure helpers from System.Agents.CLI.SessionDurable for testing:
- formatContinuationToken / parseContinuationToken
- parseResultFile
- extractDeferredCalls
- extractIsolatedCalls
Added test/SessionDurableTests.hs with unit tests for:
- Continuation token formatting/parsing round-trip.
- Result-file parsing for JSON and plain-text responses.
- Deferred-call extraction from partial turns.
- Isolated-call extraction from manually constructed partial turns.
- Pending/complete integration test via a temporary file session store.
Registered the new test module in test/Main.hs and agents.cabal.

Verification

Library builds with -Wall -Werror.
Executable agents-exe builds with -Wall -Werror.
Test suite agents-tests passes, including all Phase 2–6 durable-workflow tests and the new Phase 7 SessionDurableTests.

Next: Phase 8 — Testing strategy

Remaining work from the plan:

Policy classification tests (already covered in Phase 2 DurableWorkflowTests).
Partial turn serialization through SQLite/file backends (already covered in Phase 3).
Wake/resume tests (already covered in Phase 2/4).
Cache integration tests (already covered in Phase 2).
Isolation contract tests (already covered in Phase 5).
Determinism/resume-twice tests (can be added as part of Phase 8).

Phase 8 — Testing strategy ✅ COMPLETE

8.1 Review of existing Phase 2–7 tests

Verified that the durable-workflow tests already covered the first five Phase 8 categories:

Policy classification (Phase 2 policyClassificationTest).
Partial turn serialization through SQLite/file backends (Phase 3 fileBackendRoundTripTest, sqliteBackendRoundTripTest).
Wake/resume (Phase 2 wakeAndResumeTest).
Cache integration (Phase 2 cacheIntegrationTest).
Isolation contract (Phase 5 localProcessRunnerTest, isolatedToolNamePolicyTest).

Updated the Haddock header of test/DurableWorkflowTests.hs to explicitly map each existing test group to the Phase 8 category it covers, and to reference the new Phase 8 determinism/edge-case tests in test/DurableWorkflowDeterminismTests.hs.

8.2 Determinism / resume-twice tests

Added test/DurableWorkflowDeterminismTests.hs with:

determinismTest: constructs a yielded session with one sync and two deferred calls, wakes both deferred calls, then resumes the identical woken session twice. Asserts that both resumes produce the same semantic outcome by comparing LLM turn content and turn history while ignoring generated turnId values.
wakeIdempotenceTest: wakes a session once, then wakes the resulting session again with the same token/result pair, asserting the session state is unchanged.

8.3 Edge-case tests

Added focused edge-case tests in DurableWorkflowDeterminismTests:

wakeNoPartialTurnTest: wakeSession on a session without a partial turn is a no-op.
completeLastDeferredTest: completing the single remaining deferred call converts a PartialUserTurn into a full UserTurn.
compositeBackendDeduplicationTest: a composite backend’s sbList returns only one entry when the same session is stored in multiple backends.

Implemented the deduplication behavior in src/System/Agents/SessionStore.hs: mkCompositeSessionStore now applies dedupeBy fst to the concatenated backend lists so the primary backend’s entry wins. Updated the composite backend Haddock to document this.

Registered DurableWorkflowDeterminismTests in test/Main.hs.
Added DurableWorkflowDeterminismTests to the other-modules of the agents-tests test suite in agents.cabal.

Verification

Also added the missing default-language: Haskell2010 field to the agq executable stanza in agents.cabal; without it the executable defaulted to Haskell98 and failed to compile under the current GHC/cabal setup.
Library builds with -Wall -Werror.
Test suite agents-tests passes, including all Phase 2–7 durable-workflow tests and the new Phase 8 determinism/edge-case tests.

Addendum — User-facing how-to and demonstrator ✅ COMPLETE

After completing the implementation phases, a runnable demonstrator and a user-facing how-to were added so the durable-workflow mode can be exercised without writing custom code first.

Additions

examples/durable-workflow-demo/Main.hs — self-contained executable that:
- simulates an LLM turn with three tool calls (fetch_local, fetch_remote_a, fetch_remote_b);
- uses a ToolCallPolicy that runs the local call synchronously and defers the two remote calls;
- prints the yielded partial turn with continuation tokens;
- wakes the session with external results for the deferred calls;
- resumes the session and prints the final LLM response.
- Uses a mock LLM, so it needs no API keys and runs deterministically.
documentation/durable-workflows-howto.md — guide covering:
- how to run the demonstrator;
- core durable-workflow concepts (execution mode, policy, executor, persistence, wake/resume);
- the agents session pause|resume|pending|complete|run-isolated CLI commands;
- how to build a custom durable agent in Haskell;
- how to add durable storage, caching, and isolated execution.
documentation/README.md — updated the documentation index to link to the new how-to and mention durable workflows in the overview and quick-start sections.
agents.cabal — registered the new durable-workflow-demo executable and fixed the pre-existing missing System.Agents.TUI.Clipboard entry in the library’s exposed-modules.

Verification

The durable-workflow-demo executable builds with -Wall -Werror.
The full test suite agents-tests passes.
Library, agents-exe, and agq executables all build with -Wall -Werror.

Summary — What we gained

Implementing all eight phases of the durable-workflows plan turned agents-exe from a purely synchronous, in-process runner into a durable, resumable, and isolatable workflow engine.

Capabilities delivered

Per-call execution policy: ToolCallPolicy lets an agent decide, for every tool call, whether to run it synchronously, defer it, yield it for external completion, or run it in isolation (Docker/subprocess).
Tracked tool-call lifecycle: TrackedToolCall gives every call a stable ID and explicit state (Ready / Running / Deferred / Completed / Failed), enabling external systems to match results to calls.
Yield and wake: a session can pause mid-turn, persist, and later resume when external results arrive via wakeSession / resumeSession.
Pluggable executors: ToolExecutor, DeploymentRunner, and concrete runners (localProcessRunner, dockerRunner) make how a call runs independent of the session loop.
Durable session storage: SessionBackend abstracts file, SQLite, and composite backends so sessions can survive process restarts and be shared across machines.
Operator CLI: agents session pause|resume|pending|complete|run-isolated gives operators direct control over durable executions.
Runnable demonstrator and how-to: users can exercise the canonical flow immediately with cabal run durable-workflow-demo and follow documentation/durable-workflows-howto.md to build their own durable agents.

Architectural gains

Separation of concerns: policy, execution, persistence, and isolation are now distinct, composable layers.
Backward compatibility: existing synchronous agents continue to work unchanged; durable features are opt-in via combinators.
Testability: 848 tests cover policy classification, serialization, wake/resume, caching, isolation envelopes, backend round-trips, CLI helpers, and determinism.

New modules and examples

System.Agents.Session.Durable — policy/executor primitives
System.Agents.Session.Isolation — isolated execution envelopes and runners
System.Agents.Session.Wake — wake/resume API
System.Agents.CLI.SessionDurable — operator CLI
test/DurableWorkflowTests.hs, test/SessionDurableTests.hs, test/DurableWorkflowDeterminismTests.hs
examples/durable-workflow-demo/Main.hs — runnable mock-LLM demonstrator
documentation/durable-workflows-howto.md — user-facing how-to guide

The codebase now supports the requested example flow end-to-end: an agent issues three tool calls, executes one and defers two, yields, persists, wakes with the two results, and resumes to completion.

Follow-up — Phase 2 scheduler compilation fix

A subsequent review found that src/System/Agents/Session/Step.hs had drifted into a non-compiling state despite the Phase 2 completion note. The following fixes were applied to make the async scheduler build and the test suite pass again:

Added missing imports:
- System.Agents.Session.Base (agent/session/action types)
- System.Agents.Session.Async (mkToolContinuationSnapshot, storeContinuation)
- System.Agents.Session.Compat (parseToolCallFromLlmToolCall)
- System.Agents.Session.Types (StepByteUsage, calculateStepByteUsage)
- System.Agents.Media.Types (ContentPart, MediaAttachment)
- System.Agents.OS.Core.World (getComponent)
- Control.Concurrent.STM (atomically)
Qualified the System.Agents.OS.Conversation import as OSConv and updated pollRunningCall to use OSConv.ToolCallState, OSConv.tcStatus, and the OSConv.TcCompleted / OSConv.TcFailed / OSConv.TcCancelled constructors, resolving name clashes with the session-layer ToolCallState / tcResult.
Removed the dead import of System.Agents.Session.Async.Engine.
Enabled TypeApplications so getComponent @OSConv.ToolCallState is valid.

Verification

Library builds with -Wall -Werror.
Executable agents-exe builds with -Wall -Werror.
Executable durable-workflow-demo builds with -Wall -Werror.
Test suite agents-tests passes, including all durable-workflow tests.

AI Context - Working on the Agents Framework

2026-10-04T15:53:54Z

Generated from documentation/AI-CONTEXT.md, the repository is the canonical source and may be ahead of this page.

AI Context - Working on the Agents Framework

READ THIS FIRST when invoked on this codebase.

Immediate First Steps

When you’re invoked on this project, run these queries immediately:

-- 1. What is this project?
SELECT * FROM project_overview;

-- 2. What docs exist?
SELECT doc_path, title FROM docs_index ORDER BY doc_path;

-- 3. What's the module structure?
SELECT module_name, purpose, documented_in 
FROM code_index 
ORDER BY filepath;

-- 4. Check for recent changes
SELECT * FROM code_index 
WHERE module_name LIKE '%OS%' 
ORDER BY filepath;

Project Snapshot

Agents is a Haskell framework for AI agent orchestration with:

NEW (2026-03): ECS-based OS Model for agent management
Multi-agent hierarchies (agents can call other agents as tools)
Tool system: bash scripts, MCP servers, OpenAPI specs
Session persistence
TUI + CLI + MCP server interfaces
OpenAI-compatible LLM integration

Architecture Overview

┌────────────────────────────────────────────────────────────────┐
│                    Interface Layer                              │
│  (CLI commands, TUI, MCP server, HTTP endpoints)                │
├────────────────────────────────────────────────────────────────┤
│                    OS Model Layer     [NEW]                     │
│  (Entity-Component-System, Resource Management,                 │
│   Conversation Tracking, Concurrent Access)                     │
├────────────────────────────────────────────────────────────────┤
│                    Agent Tree Layer                             │
│  (multi-agent hierarchy, reference validation, cycle detection) │
├────────────────────────────────────────────────────────────────┤
│                    Foundation Layer                             │
│  (sessions, tools, LLM integration, file loading)               │
└────────────────────────────────────────────────────────────────┘

Key architectural pattern: The OS Model (Entity-Component-System) now manages agents, toolboxes, and resources, replacing the legacy Runtime-per-agent model.

Critical Files

File	Why It Matters
`src/System/Agents/OS.hs`	NEW: Main OS module, exports ECS-based agent management
`src/System/Agents/OS/Core.hs`	NEW: Core ECS types: `World`, `EntityId`, `Component`
`src/System/Agents/OS/Agents.hs`	NEW: OS-native agent creation and management
`src/System/Agents/OS/Compat/Runtime.hs`	NEW: Migration bridge from old Runtime to new OS
`src/System/Agents/Base.hs`	Core types: `Agent`, `AgentId`, `ConversationId`, `ExtraAgentRef`
`src/System/Agents/AgentTree.hs`	Multi-agent discovery, cycle detection, reference validation
`src/System/Agents/Runtime/Runtime.hs`	Legacy Runtime (deprecated, use OS model)
`app/Main.hs`	CLI entry point - all commands defined here
`agents.cabal`	Dependencies, build config

Documentation Map

documentation/
├── README.md              # Start here for overview
├── architecture.md        # System design, data flow, module relationships
├── OS-API.md              # [NEW] OS Model API reference
├── MIGRATION-OS.md        # [NEW] Migration guide: Runtime -> OS
├── CHANGELOG-OS-MIGRATION.md # [NEW] Recent changes summary
├── tools.md               # Tool registration (bash/MCP/OpenAPI/IO)
├── mcp.md                 # Model Context Protocol integration
├── sessions.md            # Session persistence and lifecycle
├── tui.md                 # Terminal UI (Brick library)
├── cli-commands.md        # All CLI commands reference
├── export-import.md       # Tool sharing (tar.gz/git)
├── file-loader.md         # JSON loading, section extraction
├── AI-CONTEXT.md          # This file
└── ai-assistant-guidelines.md # Detailed reference

The OS Model (Entity-Component-System)

The OS Model is the new architectural foundation (Phase 2+):

Core Concepts

-- Entity: Just a unique ID
newtype EntityId = EntityId UUID
newtype AgentId = AgentId EntityId
newtype ToolboxId = ToolboxId EntityId

-- Component: Pure data attached to entities
data AgentConfig = AgentConfig
    { agentName :: Text
    , agentModel :: ModelConfig
    , agentSystemPrompt :: Text
    , agentToolboxBindings :: [ToolboxBindingSpec]
    }

data AgentState = AgentState
    { agentStatus :: AgentStatus
    , agentCurrentConversation :: Maybe ConversationId
    , agentCreatedAt :: UTCTime
    }

-- World: Container for all component stores
newtype World = World 
    { componentStores :: HashMap ComponentTypeId (TVar Any) 
    }

Basic Usage

import System.Agents.OS

-- Initialize world with component stores
world <- atomically $ do
    w <- newWorld
    w' <- registerComponentStore w (Proxy @AgentConfig)
    registerComponentStore w' (Proxy @AgentState)

-- Create agent
let config = AgentConfig
    { agentName = "my-agent"
    , agentModel = ModelConfig "openai" "url" "gpt-4" "key"
    , agentSystemPrompt = "You are helpful"
    , agentToolboxBindings = []
    }
result <- createAgent world config

Migration Phases

PhaseOldOnly (REMOVED) ──> PhaseDual ──> PhaseNewOnly
    (Legacy only)          (Both)        (OS only)

Current: PhaseDual is the default, with PhaseNewOnly available.

Resource Scopes

Resources have explicit lifetimes:

Program Scope: Global resources (HTTP pools)
Agent Scope: Per-agent resources
Toolbox Scope: Per-toolbox resources (SQLite)
Conversation Scope: Per-conversation resources
Turn Scope: Temporary turn resources
ToolCall Scope: Single-use resources

Concurrent Access Patterns

Pattern	Use Case	Implementation
`ExclusiveAccess`	Lua interpreters	TMVar
`ReadWriteAccess`	SQLite (WAL mode)	RWLock
`PoolAccess`	HTTP connections	TBQueue
`StatelessAccess`	Immutable data	None

Knowledge Base Schema

The SQLite knowledge base (agents_kb.sqlite) tracks state:

-- Key tables:
--   project_overview  - project metadata
--   code_index        - 108 Haskell modules with docs linkage
--   docs_index        - 15+ documentation files
--   commits_seen      - git commits already processed

-- Common queries:
-- Find OS modules:
SELECT * FROM code_index WHERE filepath LIKE '%OS%';

-- Find where a module is documented:
SELECT documented_in FROM code_index 
WHERE module_name = 'System.Agents.OS.Core';

-- Check documentation coverage:
SELECT 
    COUNT(*) as total_modules,
    SUM(CASE WHEN documented_in IS NOT NULL THEN 1 ELSE 0 END) as documented
FROM code_index;

Module-to-Doc Mapping

If working on...	Update...
`System.Agents.OS.*`	`documentation/OS-API.md`
`System.Agents.OS.Compat*`	`documentation/MIGRATION-OS.md`
`System.Agents.Base`	`documentation/architecture.md`, `documentation/README.md`
`System.Agents.Runtime*`	`documentation/architecture.md` (legacy)
`System.Agents.AgentTree*`	`documentation/architecture.md`
`System.Agents.Tools.*`	`documentation/tools.md`
`System.Agents.MCP.*`	`documentation/mcp.md`
`System.Agents.Session*`	`documentation/sessions.md`
`System.Agents.TUI.*`	`documentation/tui.md`
`System.Agents.CLI.*`	`documentation/cli-commands.md`
`app/Main.hs`	`documentation/cli-commands.md`
`System.Agents.ExportImport*`	`documentation/export-import.md`
`System.Agents.FileLoader*`	`documentation/file-loader.md`

Code Patterns to Recognize

Pattern: ECS Component Definition

-- Define component type
newtype ComponentTypeId = ComponentTypeId { unComponentTypeId :: Int }

class Component a where
    componentId :: Proxy a -> ComponentTypeId

-- Create a component
instance Component MyComponent where
    componentId _ = ComponentTypeId 100  -- Unique ID

-- Use with world
atomically $ do
    setComponent world eid myComponent
    mComp <- getComponent @MyComponent world eid

Pattern: OS Agent Creation

-- OS model approach
import System.Agents.OS

result <- createAgent world $ AgentConfig
    { agentName = "my-agent"
    , agentModel = ModelConfig "openai" "url" "gpt-4" "key"
    , agentSystemPrompt = "You are helpful"
    , agentToolboxBindings = []
    }

case result of
    Left err -> handleError err
    Right agentId -> do
        -- Agent created successfully

Pattern: Resource Management

import System.Agents.OS.Resources

-- Create registry
registry <- atomically newResourceRegistry
let ctx = ResourceContext [ProgramScope] registry

-- Create resource
rid <- createResource ctx (SqliteResource config) $ \rid -> do
    conn <- openConnection config
    pure ResourceHandle
        { handleId = rid
        , handleCleanup = closeConnection conn
        , handleAccess = \f -> f (SqliteAccessor conn)
        }

-- Cleanup
cleanupScope registry (AgentScope agentId)

Pattern: Migration Compatibility

-- During migration, use compatibility layer
import System.Agents.OS.Compat.Runtime

result <- initializeWithMigration defaultMigrationConfig
case result of
    Right os -> do
        -- Use new OS with compatibility bridge
        bridge <- newRuntimeBridge agentId os
        runWithBridge bridge $ do
            tools <- listTools
            callTool "my-tool" args
    _ -> error "Migration not configured"

Pattern: Tracer Usage

-- All operations are traced
import Prod.Tracer

data MyTrace = Started | Completed | Failed String

myFunction :: Tracer IO MyTrace -> IO ()
myFunction tracer = do
    runTracer tracer Started
    result <- try operation
    case result of
        Right _ -> runTracer tracer Completed
        Left e  -> runTracer tracer (Failed $ show e)

Pattern: Concurrent Access with STM

-- SQLite with concurrent reads (WAL mode)
result <- runResourceM ctx $ withRead rid $ do
    queryDatabase sql

-- Lua with exclusive access
result <- runResourceM ctx $ withExclusive luaRid $ do
    runLuaScript script

-- HTTP with connection pool
result <- runResourceM ctx $ withPooled httpRid $ do
    makeHttpRequest req

Key Types Reference

OS Model Types

-- ECS Core
newtype EntityId = EntityId UUID
newtype AgentId = AgentId EntityId
newtype ToolboxId = ToolboxId EntityId
newtype ConversationId = ConversationId EntityId

-- Agent Components
data AgentConfig = AgentConfig
    { agentName :: Text
    , agentModel :: ModelConfig
    , agentSystemPrompt :: Text
    , agentToolboxBindings :: [Text]
    }

data AgentState = AgentState
    { agentStatus :: AgentStatus
    , agentCurrentConversation :: Maybe ConversationId
    , agentCreatedAt :: UTCTime
    }

-- Resource Management
data ResourceScope
    = ScopeGlobal
    | ScopeAgent AgentId
    | ScopeConversation ConversationId

data AccessPattern
    = ExclusiveAccess      -- TMVar
    | ReadWriteAccess      -- RWLock
    | PoolAccess Int       -- TBQueue
    | StatelessAccess      -- No lock

Legacy Types (Still Supported)

-- Original Runtime types
data Agent = Agent
    { slug :: Text
    , apiKeyId :: Text
    , flavor :: Text
    , modelUrl :: Text
    , modelName :: Text
    , systemPrompt :: [Text]
    , toolDirectory :: FilePath
    , mcpServers :: Maybe [McpServerDescription]
    , extraAgents :: Maybe [ExtraAgentRef]
    }

data ExtraAgentRef = ExtraAgentRef
    { extraAgentSlug :: Text
    , extraAgentPath :: FilePath
    }

Git Workflow

When documentation work spans multiple invocations:

First invocation: Plan, gather info, create initial docs
Intermediate: Continue where left off, check commits_seen
Final: Update all indices, ensure consistency

Always include a summary line as the first line of response.

Testing Documentation

Verify docs are correct by checking:

Type definitions match source code
File paths in examples exist
JSON examples are valid
Module names are correct (use bash_grep-files to verify)

Running Tests

# All tests
cabal test

# OS-specific tests
cabal test --test-option=--pattern="OS"

# Benchmarks
cabal bench

Red Flags

Watch for these issues:

Issue	Action
Undocumented OS modules	Add to `code_index`, update `documentation/OS-API.md`
Docs referencing wrong modules	Update `docs_index.related_modules`
Missing migration guidance	Update `documentation/MIGRATION-OS.md`
Broken code examples	Fix or mark as "illustrative"
New CLI commands not in `cli-commands.md`	Add with examples
Commits not tracked	Process and insert to `commits_seen`

Quick Queries

-- Find all OS-related modules:
SELECT module_name, filepath 
FROM code_index 
WHERE module_name LIKE '%OS%' OR filepath LIKE '%OS%'
ORDER BY filepath;

-- Find all documentation:
SELECT doc_path, title FROM docs_index ORDER BY doc_path;

-- Check documentation coverage:
SELECT 
    COUNT(*) as total,
    SUM(CASE WHEN documented_in IS NOT NULL THEN 1 ELSE 0 END) as documented,
    ROUND(100.0 * SUM(CASE WHEN documented_in IS NOT NULL THEN 1 ELSE 0 END) / COUNT(*), 1) as pct
FROM code_index;

-- Find modules by documentation file:
SELECT module_name, filepath 
FROM code_index 
WHERE documented_in = 'documentation/OS-API.md'
ORDER BY module_name;

Working with Cabal

The project uses agents.cabal. Key stanzas:

-- Library exports modules under System.Agents
library
    exposed-modules:
        System.Agents
        System.Agents.OS           -- NEW
        System.Agents.OS.Core      -- NEW
        System.Agents.OS.Agents    -- NEW
        System.Agents.Base
        System.Agents.Runtime
        ...

-- Executable is app/Main.hs
executable agents-exe
    main-is: Main.hs
    hs-source-dirs: app
    build-depends: agents, ...

-- Test suite
test-suite agents-tests
    other-modules:
        OS.CoreTests
        OS.IntegrationTests
        OS.CompatibilityTests
        ...

-- Benchmarks
benchmark os-benchmarks
    main-is: OSBenchmarks.hs
    hs-source-dirs: bench

Common Import Patterns

-- OS Model (New)
import System.Agents.OS
import System.Agents.OS.Core
import System.Agents.OS.Agents
import System.Agents.OS.Compat.Runtime

-- Core types
import System.Agents.Base

-- Legacy Runtime (being deprecated)
import System.Agents.Runtime
import qualified System.Agents.Runtime.Runtime as Runtime

-- Agent tree
import qualified System.Agents.AgentTree as AgentTree
import qualified System.Agents.OS.AgentTree as OSAgentTree

-- Tools
import qualified System.Agents.Tools.Bash as Bash
import qualified System.Agents.Tools.BashToolbox as BashToolbox

-- Sessions
import qualified System.Agents.Session.Base as Session

-- Tracing
import Prod.Tracer

-- STM for concurrency
import Control.Concurrent.STM

Summary Checklist

Before finishing work:

[ ] Summary line included (first line of response)
[ ] Knowledge base tables updated if needed
[ ] New OS modules added to code_index
[ ] New documentation files added to docs_index
[ ] Documentation created/updated for changes
[ ] Cross-references checked
[ ] Code examples verified

Emergency Contacts

If completely lost:

Read documentation/README.md for overview
Check documentation/CHANGELOG-OS-MIGRATION.md for recent changes
Read documentation/MIGRATION-OS.md for migration guidance
Look at src/System/Agents/OS.hs for OS entry point
Check app/Main.hs for current command structure
Query code_index for module purposes

Remember: This is a well-structured Haskell project in transition from Runtime-per-agent to ECS-based OS model. Follow the types, follow the imports, and you’ll find your way.

Recent Changes (March 2026)

See documentation/CHANGELOG-OS-MIGRATION.md for detailed change log.

Key highlights:

27 new OS modules added for ECS-based architecture
PhaseOldOnly removed - now only PhaseDual and PhaseNewOnly
Full compatibility layer for gradual migration
New documentation: OS-API.md, MIGRATION-OS.md
New tests: Integration, Compatibility, benchmarks

Spec: a salmon-driven preview deployment of agents-server

2026-10-04T15:53:54Z

Generated from todos/salmon-preview-deployment.md, the repository is the canonical source and may be ahead of this page.

Spec: a salmon-driven preview deployment of agents-server

Status: proposal, 2026-10-03. Nothing implemented. A design pass with a recommendation; the points marked Decide need the owner before any phase starts.

Goal

Run the built agents-server as it would run in production (its own process, its files on disk, a real database, later a real init system), check it over the network, and tear it down. The same recipe, left up instead of torn down, is a preview of a branch.

salmon is the tool that brings the deployment up and down: it expresses the deployment as a graph of idempotent ops with up/down/check, and it already has the pieces this needs (podman containers, systemd units, postgres, qemu VMs, files).

Where things stand

What is tested today, and how:

What	Where	How real
Runtime, sessions, tools	`agents-tests` (`test/`)	in-process, in-memory persistence
HTTP API, auth, events, mail	`agents-server-tests` (`examples/agents-server/test`)	in-process WAI application on a free port, the LLM replaced by a Haskell function, a temporary SQLite file
Postgres store, leases, takeover	`agents-postgres-tests` (`postgres/test`)	a real cluster from `initdb`/`pg_ctl` (or `AGENTS_TEST_POSTGRES_URL`); skipped when neither is there
TUI against a real binary	`checks/*-e2e`	the built `agents-exe` under a pty, with `checks/lib/fake_llm.py` as an OpenAI-compatible LLM

What none of them runs:

the agents-server executable itself: its option parser, its startup order, its exit codes, its JSON log lines;
the units in bundling/systemd/ (User=, StateDirectory=, TimeoutStopSec against --shutdown-grace, Restart=on-failure);
SIGTERM and SIGKILL on a real process, and what a restart finds;
a database written by one version and opened by the next;
two server processes on one Postgres, one of them killed;
a client on another machine: --bind, --auth-tokens, --cors-origin, agents-exe tui --attach URL;
the binary on a machine other than the one that built it.

salmon is not used by this repository. The two mentions are incidental (dbs/README.md, website/README.md).

What salmon provides

Read from salmon’s README and resources/howto-ops.md (section 10); none of it has been run for this spec.

Builtin nodes for Podman (and quadlets), Systemd, Postgres, Qemu, Filesystem, Cabal, Daemon (a process salmon keeps running where there is no systemd), Certificates.
A binary built on salmon speaks config <seed> | run up, run down, run tree. Teardown is the graph walked backwards.
Its own tests are in four layers by cost: graph shape only (0), temporary directories (1), disposable podman containers (2), a qemu VM for systemd as PID 1 and real interfaces (3). Layers 2 and 3 skip loudly when podman or the VM privileges are missing.
SreBox.Gcp.PreviewEnvironment groups several Cloud Run deploys into one node per branch. It is the precedent for “one thing to point run up and run down at”, on a host this spec does not propose.
salmon-core, salmon-ops and salmon-ops-recipes are on Hackage. The test harness (Test.Harness: withContainer, withVm, sshToVm, …) is in salmon’s test tree, not in a library, so it cannot be imported from here as it is.

Recommendation

Build one salmon recipe, in this repository, that describes an agents-server deployment, and use it two ways: a test driver runs up, checks, down; an operator runs up and keeps it. Start with a container and SQLite, and add Postgres and a VM as later phases. Do not make it a tier of cabal test.

One graph, two uses

agents-preview config --name NAME --host container|vm --db sqlite|postgres ...
    | agents-preview run up        # or: run down, run tree

The graph, from the leaves:

the agents-server binary for the target (see “The binary” below);
the host: a podman container, or a qemu VM;
files on the host: agent file, keys file, token file, and for the VM the unit from bundling/systemd/, copied as it is shipped;
the LLM: checks/lib/fake_llm.py with a script, listening next to the server; the agent file’s modelUrl points at it, as checks/lib/e2e.py already does. No API key, so no secret is involved;
optionally Postgres and its database;
the server process (container command or Daemon in the container, the systemd unit in the VM);
a last node whose check is GET /healthz answering 200.

run up prints where the server listens and a token. A preview is this, left standing; run down removes it.

Where it lives

A separate cabal package, preview/, with its own project file (cabal.preview.project), depending on salmon-ops and salmon-ops-recipes from Hackage and on agents-lib for the HTTP client. The repository has no cabal.project today, only a freeze file, so the default build and cabal test are untouched and nobody needs salmon, podman or qemu to work on agents-exe. This is what salmon does for its own experimental packages (cabal.perso.project).

Not recommended:

A new test-suite in agents.cabal. It would put salmon and all its dependencies in the freeze file of every build, for tests that most machines skip.
The recipe in the salmon repository. agents-server’s flags and files change here; the recipe must change in the same commit.
A shell script around podman. It would work for phase 1. It gives no down that matches up, no check, and nothing to reuse for the VM.

The checks

Written in Haskell in the same package, against the address run up printed, with System.Agents.Host.Client.Http (the client the TUI attaches with). They take a URL and a token and nothing else, so they also run against a server deployed by other means.

Phase 1, container and SQLite:

/healthz answers; the server.started log line says authentication: bearer.
A request without a token is refused; one owner cannot read another’s session.
A scripted run: create a session with a prompt, wait, read the answer the fake LLM was scripted to give, with one tool call on the way.
The events stream of that run carries run.started … run.stopped.
SIGTERM during a run: the process exits within --shutdown-grace, the session is stored.
SIGKILL during a run, then start: sessions.recovered names the session, its status is the one its turns imply, resume continues it.
agents-exe tui --attach URL from outside the container (the existing checks/phase4-attach-e2e, given a URL).
Upgrade: bring up the binary built from main, run check 3, replace the binary with the branch’s, start on the same database, read the old session and run check 3 again.

Phase 2, Postgres:

The server creates its tables in an empty database.
Two servers on one database: a message sent to one reaches a run on the other; kill the owner, and the other logs sessions.taken_over after the lease expires.
Check 8 with Postgres.

Phase 3, a VM with systemd:

bundling/systemd/agents-server.service, installed as the documentation says, starts as the agents-server user and creates its state directory.
systemctl stop during a run ends cleanly, before TimeoutStopSec.
A killed server is restarted by systemd and recovers its sessions.
journalctl -u agents-server -o cat is one JSON object per line.
The VM reboots; the service comes back with its sessions.

The binary

A binary built on the developer’s machine is linked against that machine’s glibc and libraries, and may not start in a debian:bookworm container. Two ways:

build inside a container of the target’s distribution and copy the binary out (bundling/Containerfile.build does this for agents-exe, from a clone of GitHub rather than the working tree, and on haskell:9.8.4 while agents.cabal asks for base >=4.20: it looks out of date, and was not built for this spec);
choose the container image to match the host that builds.

The first is right for a preview that must be like production and slow (a full build, once per image, cached after). The second is enough for phase 1 on the owner’s machine. Phase 1 takes the path of a binary as a seed argument, so the choice can be made later.

Phases

Each is useful alone and can stop there.

Phase	Adds	Needs on the machine
0	the `preview/` package, the graph with a container and no server (`run tree`, `up`, `down`), a graph-shape test	podman
1	server, fake LLM, SQLite, checks 1 to 8	podman, python3
2	Postgres, a second server, checks 9 to 11	podman
3	VM host, the shipped unit, checks 12 to 16	qemu, root or the `setcap` grants salmon's harness documents
4	a preview on a host other than this machine	Decide: see below

Phase 3 depends on salmon’s VM harness being usable from outside salmon’s test tree. That is a change in salmon and is not part of this work: ask for it there, or copy the few functions needed here until it exists.

The other reading: salmon as a feature

The request also said “or integrate as a new feature”. Two candidates were considered, and neither is recommended now.

Agents that operate salmon. salmon run serve --http has an HTTP API with an OpenAPI description (salmon-ops/openapi/serve-api.openapi.json), and agents-exe has an OpenAPI toolbox. An agent that lists, converges and inspects a salmon world may need no code at all, only an agent file and a page of documentation. Worth one experiment after phase 1, when there is a salmon graph in this repository to point it at.
salmon as the way to install agents-server. The units in bundling/systemd/ and a page of documentation do this today. A salmon recipe for production installs is the phase 1 graph with a real host, and should wait until the preview has shown the graph is right.

Not in scope

Deploying anywhere but the machine that runs the recipe (until phase 4 is decided).
TLS, a reverse proxy, DNS. The server binds a plain port; the documentation puts a proxy in front. A later check, not a first one.
A real LLM. Every check uses the fake one. A preview an operator keeps may be given real keys by hand; the recipe does not handle them.
CI. The repository has no CI configuration; this spec adds none.
Changes to the salmon repository.

Decide

Scope of the first step. Recommended: phases 0 and 1 (container, SQLite), then stop and look. The alternative is to go straight to Postgres and systemd, which is closer to production and much more work before the first result.
Host. Recommended: podman container first, qemu VM in phase 3 for systemd. The alternative is the VM from the start (one host kind, but every run needs VM privileges and a rootfs).
Package location. Recommended: preview/ with its own project file. Is a second cabal project in this repository acceptable?
The checks’ language. Recommended: Haskell with the existing HTTP client. The alternative is Python next to checks/lib, which reuses fake_llm.py’s conventions and adds no Haskell dependency, but duplicates the client.
Phase 4. Is a preview on another host wanted at all (a URL to open per branch), and if so on what: a machine of the owner’s over ssh, or Cloud Run as salmon’s PreviewEnvironment does? This one involves real hosts, credentials and cost, and nothing is proposed until it is answered.
The salmon harness. May the VM helpers move from salmon’s test tree to a library, or should phase 3 copy them?
bundling/Containerfile.build. Fix it as part of phase 1 (build from the working tree, a GHC that satisfies base >=4.20, also produce agents-server), or leave it?

Asynchronous / Interruptible Tool Calls

2026-10-04T15:53:54Z

Generated from todos/async-tool-calls.md, the repository is the canonical source and may be ahead of this page.

Asynchronous / Interruptible Tool Calls

Goal

Make tool calls in a TUI/oneshot session truly interruptible and inspectable:

Tool calls can execute concurrently instead of blocking the whole turn.
The LLM can receive partial answers as soon as individual calls finish.
The LLM can later query the status of a still-running or previously-yielded tool call.
Tool calls become first-class entities in the ECS-style OS, visible to the rest of the system.

Today the flow is:

LLM turn emits tool calls.
agents-exe schedules them synchronously:
- tool call 1 runs
- tool call 2 runs
agents-exe returns the LLM turn.

We want:

LLM turn emits tool calls.
agents-exe promotes each call to an OS entity and starts them concurrently.
As soon as at least one call has a useful partial result, the scheduler can yield a partial user turn back to the LLM.
The LLM can continue the conversation, optionally calling a system capability to inspect any call it is interested in.
When all calls for the turn are complete, the turn finalizes normally.

Current State

There is already async scaffolding in place:

System.Agents.Session.Types defines TrackedToolCall, ToolCallState (Ready, Running, Deferred, Completed, Failed), ToolCallDisposition (RunSync, RunAsync, RunIsolated, Defer, Decorate), ContinuationToken, etc.
System.Agents.Session.Async provides continuation tokens, snapshots, and a ContinuationStore (SQLite-backed).
System.Agents.Session.Step has runStepMAsync, which classifies calls via ctxToolCallPolicy and can defer calls.
System.Agents.Session.Loop has runAsync / runAsyncWithProgress that pause when a PartialUserTurn is present.

What is missing:

RunAsync / Running are mostly nominal. Calls are still executed sequentially in one batch; RunAsync just defers them.
There is no real concurrent execution engine.
There is no way for the LLM to ask “what is the status of tool-call X?” while it is running.
Tool calls are not yet represented as ECS entities with runtime components.
Partial answers are not surfaced: a call is either done or deferred.

Design Overview

We introduce three layers:

ECS OS layer — every tool call becomes an EntityId with ToolCallConfig and ToolCallState components. The OS is the source of truth for status.
Async execution engine — a small scheduler that runs approved calls concurrently, updates OS components, and emits heartbeat/partial progress.
System-toolbox introspection capability — a new get-tool-call-status capability (and possibly list-running-tool-calls) that any agent can invoke.

The key idea is that the existing PartialUserTurn mechanism is extended so that a turn can be yielded before all calls are complete, carrying intermediate responses. The LLM can then decide whether to keep waiting, do something else, or inspect a specific call.

ECS Integration

System.Agents.OS.Conversation.Types already has the right component shapes:

ToolCallConfig — static info: turn, tool name, input args, parent call.
ToolCallState — runtime status, timestamps, result value.

We extend them with async-specific fields:

data ToolCallConfig = ToolCallConfig
    { tcTurnId :: TurnId
    , tcToolName :: Text
    , tcToolInput :: Value
    , tcParentCallId :: Maybe ToolCallId
    , tcSessionId :: SessionId          -- NEW: which session owns this call
    , tcConversationId :: ConversationId -- NEW
    }

data ToolCallState = ToolCallState
    { tcStatus :: ToolCallStatus
    , tcStartedAt :: Maybe UTCTime      -- NEW: was required, now optional until actually started
    , tcCompletedAt :: Maybe UTCTime
    , tcResult :: Maybe Value
    , tcProgress :: [ToolCallProgress]  -- NEW: ordered structured partial updates
    }

-- NEW component-like value stored inside ToolCallState.
-- All progress payloads are structured JSON.
data ToolCallProgress = ToolCallProgress
    { progressAt :: UTCTime
    , progressKind :: ProgressKind
    , progressPayload :: Value
    }
    deriving (Show, Eq, Generic)

data ProgressKind
    = ProgressStarted
    | ProgressLog Text          -- free-form log line (still wrapped in JSON)
    | ProgressPartial Value     -- structured partial result
    | ProgressHeartbeat         -- "still alive"
    deriving (Show, Eq, Generic)

When the scheduler receives tool calls from an LLM turn, it:

Creates one OS entity per call.
Attaches ToolCallConfig and ToolCallState { TcPending }.
Links the call back to the current TurnId and ConversationId.
Returns a lightweight ToolCallHandle (entity id + ToolCallId) to the session layer.

The existing TrackedToolCall keeps its ToolCallId but gains a tcEntityId :: Maybe EntityId so the session can map back to the OS.

Async Execution Engine

A new module System.Agents.Session.Async.Engine provides:

data AsyncEngine = AsyncEngine
    { aeWorld :: World
    , aeExecutor :: ToolExecutionContext -> LlmToolCall -> IO UserToolResponse
    , aeMaxConcurrency :: Int
    }

-- | Start executing a batch of calls concurrently.
-- Returns handles and, if some calls already completed synchronously, their results.
startAsyncBatch :: AsyncEngine -> [TrackedToolCall] -> IO AsyncBatch

-- | Block until at least one call in the batch makes progress, then return.
waitForProgress :: AsyncBatch -> IO AsyncBatchUpdate

-- | Finalize any completed calls into UserToolResponses.
finalizeCompleted :: AsyncBatch -> IO [(ToolCallId, UserToolResponse)]

-- | Cancel all still-running calls (best effort).
cancelAsyncBatch :: AsyncBatch -> IO ()

-- | Cancel a single running call by its handle.
cancelToolCall :: AsyncEngine -> ToolCallId -> IO Bool

The engine uses async/Async (or forkIO + TVar) to run calls. For each call:

It sets the OS state to TcExecuting.
It runs the call in a background thread.
The call can emit progress via a callback injected into ToolExecutionContext:
```
ctxProgressCallback :: Maybe (Value -> IO ())
```
All payloads passed to this callback must be valid JSON values.
On completion it sets the OS state to TcCompleted result (or TcFailed err / TcCancelled).

Policy integration:

RunSync calls are still executed inline.
RunAsync calls go to the engine.
RunIsolated calls are delegated to the configured deployment runner but also tracked as OS entities.
Defer calls stay in TcPending with a continuation token.

Partial Answers

A tool can emit a partial answer by calling the progress callback if the tool supports streaming. For tools that do not support streaming, the engine can still yield after each individual call finishes, so the LLM gets some responses earlier.

When the session stepper asks for a user turn, the scheduler can produce a PartialUserTurn with the subset of calls that have completed so far. The LLM then sees:

{
  "tool_call_id": "...",
  "status": "completed",
  "result": "..."
}

and for still-running calls:

{
  "tool_call_id": "...",
  "status": "running",
  "progress": [
    {"at": "...", "kind": "log", "payload": "downloading..."},
    {"at": "...", "kind": "partial", "payload": {"percent": 50}}
  ]
}

The LLM can choose to continue immediately (if the completed answers are enough) or call get_tool_call_status for more detail.

Tool-Call Status System Capability

Add new capabilities to SystemToolCapability:

| SystemToolGetToolCallStatus
| SystemToolListRunningToolCalls
| SystemToolCancelToolCall

and expose them through System.Agents.Tools.SystemToolbox.

`get_tool_call_status`

Input schema:

{
  "tool_call_id": "uuid",
  "include_progress": true,
  "wait_for_completion": false,
  "timeout_seconds": 5
}

Output schema:

{
  "tool_call_id": "uuid",
  "status": "pending|running|completed|failed|cancelled|orphaned",
  "tool_name": "bash",
  "started_at": "...",
  "completed_at": null,
  "result": null,
  "progress": [...],
  "is_final": false
}

Status values:

pending — call is queued but has not started.
running — call is executing.
completed — call finished successfully.
failed — call finished with an error.
cancelled — call was cancelled before completion.
orphaned — the call is referenced in the chat history but no longer exists in the OS (process restarted, entity was dropped, etc.). This is the clear “task no longer exists” status.

If wait_for_completion is true, the tool blocks up to timeout_seconds and returns the final result when available. This lets an LLM poll efficiently.

`list_running_tool_calls`

Returns calls filtered by the current session/conversation scope (reusing the existing session-introspection scope rules). Useful for the LLM to discover what it can inspect.

`cancel_tool_call`

Input schema:

{
  "tool_call_id": "uuid",
  "reason": "user request"
}

Output schema:

{
  "tool_call_id": "uuid",
  "cancelled": true,
  "previous_status": "running"
}

Cancellation is best-effort. If the call is already final, the capability returns cancelled: false.

Changes to Session Step / Loop

runStepMAsync currently defers RunAsync calls. We change it to:
- Classify calls.
- Execute RunSync calls inline.
- Hand RunAsync calls to the async engine.
- If all calls complete quickly, produce a normal UserTurn.
- Otherwise produce a PartialUserTurn and yield.

Add a new agent configuration field:

ctxAsyncYieldStrategy :: AsyncYieldStrategy

data AsyncYieldStrategy
    = YieldOnAnyProgress       -- yield as soon as one call finishes
    | YieldWhenAllDone         -- DEFAULT: yield only once every synchronous and asynchronous call in the batch has reached a final state
    | YieldOnTimeout Int       -- yield after N milliseconds even if nothing finished
    deriving (Show, Eq)

The default is YieldWhenAllDone for backward compatibility with the existing synchronous mental model.

On resume (continuePartialTurn), the stepper checks the OS for calls that have completed since the last step and finalizes them. Any still-running calls remain in the PartialUserTurn.
When the LLM invokes get_tool_call_status and the call is still running, the stepper does not block the whole session; it returns the current snapshot. If the call completed, the result is copied back into the TrackedToolCall.

Sub-Conversations and Lineage

Lineage remains important. Tool-call entities already carry tcParentCallId for nested calls. Sub-conversations will eventually need richer primitives so that one conversation can ask another conversation for more information. The work described here is a prerequisite: once a tool-call result can be replaced by a later status update, the LLM can treat a long-running sub-conversation as an “interruptible” tool call. For now we keep the existing call-stack/lineage plumbing and ensure child tool-call entities are created with the correct parent reference.

TUI / OneShot Integration

The TUI already has a buffered-message queue and pause mechanism (ConversationStatus_Paused).
When a partial turn is yielded, the TUI should render running tool calls with a spinner/progress indicator.
The TUI event loop should listen for AppEvent_ToolCallProgress events emitted by the async engine and refresh the conversation view.
OneShot mode should support a --wait-for-async flag; without it, oneshot exits with a JSON description of pending calls and a continuation token.

Lifecycle State Machine

Pending -> Executing -> Completed
        -> Executing -> Failed
        -> Executing -> Cancelled
        -> Deferred   -> Pending  (wake/resume)
        -> Deferred   -> Completed (external completion)

When a session resumes after a process restart, any tool call referenced in the chat history that has no corresponding OS entity is reported as orphaned by get_tool_call_status. No in-progress call persistence is required; the LLM is responsible for deciding what to do with an orphaned call.

Ready / Running / Deferred / Completed / Failed from ToolCallState map cleanly to ECS ToolCallStatus:

Session `ToolCallState`	ECS `ToolCallStatus`
Ready	TcPending
Running	TcExecuting
Deferred	TcPending (with continuation token)
Completed	TcCompleted
Failed	TcFailed

Cancelled session calls become TcCancelled. Missing OS entities become the orphaned status in the capability response.

New Modules / Files

System.Agents.Session.Async.Engine — concurrent execution engine.
System.Agents.OS.Conversation.ToolCalls — helpers to create/update/query tool-call entities.
System.Agents.Tools.SystemToolbox.ToolCallStatus — implementation of the new capabilities.
System.Agents.TUI.Event.ToolCallProgress — TUI event handling for progress updates.

Modified Modules

System.Agents.OS.Conversation.Types — add progress, session/conversation ids to tool-call components; add TcCancelled.
System.Agents.Session.Types — add tcEntityId to TrackedToolCall; add AsyncYieldStrategy.
System.Agents.Session.Step — integrate the async engine and partial-yield logic.
System.Agents.Session.Loop — handle resume from partial turns with running background calls.
System.Agents.Session.Async — reuse continuation store; add progress-related helpers.
System.Agents.Base — add new SystemToolCapability constructors.
System.Agents.Tools.SystemToolbox.Core — register new capabilities.
System.Agents.Tools.SystemToolbox.Types — add query/error types for tool-call status.
System.Agents.TUI.Types — add events for tool-call progress.

Implementation Phases

Phase 1: ECS Tool-Call Entities

Promote tool-call creation from session layer to OS layer.
Add tcEntityId to TrackedToolCall.
Ensure every tool call in a session has a corresponding OS entity.
Write tests in test/OS/ConversationTests.hs.

Phase 2: Async Engine

Implement AsyncEngine with background threads and progress callbacks.
Integrate engine into runStepMAsync for RunAsync calls.
Keep RunSync behavior unchanged.
Add AsyncYieldStrategy with default YieldWhenAllDone.

Phase 3: System Capability

Add SystemToolGetToolCallStatus, SystemToolListRunningToolCalls, and SystemToolCancelToolCall.
Implement lookup by ToolCallId / entity id from the OS World.
Add wait_for_completion optional blocking.
Implement cancellation and the orphaned status.
Expose capabilities in agent configs.

Phase 4: TUI / OneShot UX

Render running calls and progress in the TUI.
Add progress events to the TUI event channel.
Add oneshot --wait-for-async / continuation-token resume.

Phase 5: Cleanup & Hardening

Cancellation on session abort.
Expiration of stale running calls.
Rate limiting / max concurrency enforcement.
Ensure orphaned status is returned consistently after process restart.

Open Questions

None remaining. Decisions from review:

Progress payloads are structured JSON.
Cancellation capability is included.
Lineage is preserved; sub-conversation primitives will be built later on top of interruptible tool calls.
In-progress calls are not persisted; missing OS entities are reported as orphaned.
Default AsyncYieldStrategy is YieldWhenAllDone.

Success Criteria

[ ] Two long-running tool calls issued in one LLM turn execute concurrently.
[ ] The LLM receives a partial user turn as soon as the first call finishes (when YieldOnAnyProgress is configured).
[ ] The LLM can call get_tool_call_status for any call and see current structured progress or final result.
[ ] The LLM can call cancel_tool_call to stop a running call.
[ ] Tool calls appear as ECS entities in the OS World.
[ ] Existing synchronous behavior is unchanged when ExecutionMode = Synchronous.
[ ] get_tool_call_status returns orphaned for calls referenced in history that no longer exist.

Spec: embedding agents-exe in a web server

2026-10-04T15:53:54Z

Generated from todos/web-server-embedding.md, the repository is the canonical source and may be ahead of this page.

Spec: embedding agents-exe in a web server

Status: proposed (2026-09-18)

Goal

Run agents from a long-lived HTTP server, with no TUI and no CLI, where:

every session lives in a database, not in conv.<uuid>.json files;
a client can start a session, send follow-up messages, watch progress live, list and complete deferred tool calls, resume, and cancel;
the server can restart at any time without losing or corrupting sessions.

The deliverable is:

a small library API in agents-lib for hosting agents, with no HTTP dependencies;
a reference HTTP server, agents-server, built on that API.

Non-goals (for this spec)

Multi-tenancy: per-user isolation, per-user API keys, and sandboxing of bash or MCP tools. The data model reserves an owner column so this can come later without a migration (see Milestone 2).
A Postgres backend. The interfaces below are designed so it can be added as a separate backend.
Streaming LLM tokens. Clients get updates at step granularity.
Agent definitions stored in the database. Agents still come from JSON files loaded at startup.
Splitting the library to drop the brick/vty dependencies.

Current state

What we can reuse as-is

Piece	Where	Notes
Loading the agent tree once, sharing it across concurrent runs	`AgentTree.withAgentTree`; used by `MCP/Server.hs:127` and `:222`	The MCP server already runs one `async` per request over a shared tree.
Stepping a session	`Session.Step.runStepM`, `Session.Loop.runUntilBlocked` (`Loop.hs:105`), `Session.Wake.resumeSession`	`runUntilBlocked` returns the session when it waits only on deferred calls.
Injecting external results	`Session.Wake.wakeSession`	Pure load, modify, return.
Session storage interface	`SessionStore.SessionBackend` (`SessionStore.hs:97`)	store / load / list / delete; file, SQLite, and composite versions.
Continuation index	`Session.Async.ContinuationStore` (`Async.hs:248`), SQLite table `tool_continuations`	Written by the scheduler (`Step.hs:490`, `:728`).
Declarative policy per agent	`toolCallPolicyConfig`, `executionMode`, etc.; applied by `Session.AgentConfig.applyAgentDurableConfig`
Progress hook	`Combinators.StoreSessionProgress.agentWithSessionProgress`	Fires `SessionUpdated` before every step.
Restart handling for background calls	`Step.pollRunningCall` (`Step.hs:505`)	A `Running` call whose process is gone becomes `Failed` ("orphaned").
Background execution	`Session.Base.withAsyncEngine`; on-demand World at `Step.hs:917`	Needs an OS `World` plus an `AsyncEngine`.

Gaps

G1. A backend set on an agent after construction is ignored for progress storage. OneShot.nodeToAgent wraps the agent with agentStoreSession while ctxSessionBackend = Nothing. agentStoreSession (StoreSessionProgress.hs:130) checks ctxSessionBackend on the agent it receives when it wraps, not at store time. So withSessionBackend backend =<< nodeToAgent … still writes every step to the file store.

G2. Agent construction is copied three times with small differences: OneShot.nodeToAgentWithThinking (OneShot.hs:293), AgentTree.OneShotTool.nodeToAgent (OneShotTool.hs:511, used for sub-agents), and the MCP server (MCP/Server.hs:~300). The one-shot copy also prints thinking to stdout/stderr. Progressive-disclosure tool filtering (agentEvaluateActiveTools) is applied only on the one-shot path.

G3. Sub-agent sessions always go to the file store. OneShotTool.nodeToAgent takes the file SessionStore, generates a fresh ConversationId, and records no link to the parent session.

G4. Session-reading tools and search read only the file store (tools fixed in Phase 3; the search index is CLI-only and stays on files). The SystemToolbox session tools (Tools/SystemToolbox/Session.hs, via SessionIntrospectionConfig.introspectionStore) and the search index (Session/Search/Index.hs, via indexSessionStore) call SessionStore.listSessions / readSession directly. Props.sessionStore passes this down from AgentTree.

G5. Sessions have two keys. The file store is keyed by ConversationId, durable commands by SessionId, and the two are generated separately (SessionDurable.hs handleStart). runOneShotWithConfig stores a paused session under both.

G6. No concurrency control. Every mutation is load, modify, store. Two concurrent requests on one session (two complete calls, or complete during a run) silently lose an update.

G7. Continuation table and sessions drift apart. wakeSession never calls csComplete, so tool_continuations rows stay pending forever. Finding a session from a token scans every session (findSessionForToken in SessionDurable.hs) even though tool_continuations.session_id already maps it.

G8. Background calls only live as long as one loop call. The loops create an engine on demand and shut it down on exit (withEngineShutdown, Loop.hs:88). A server that pauses and resumes over several HTTP requests needs a World and engine per live session that outlive a single request.

G9. No metadata beyond the JSON blob. The sessions table has only session_id, created_at, updated_at, json. Listing sessions by agent or status, or finding ones interrupted by a restart, means decoding every blob.

G10. The prompt-to-first-turn code lives in the CLI. handleStart builds the initial UserTurn inside CLI/SessionDurable.hs, so the library has no reusable version.

Design

Overview

                 +-------------------------------------------+
  HTTP / SSE --> | agents-server (wai + warp)                |   examples/agents-server/
                 +---------------------+---------------------+
                                       |
                 +---------------------v---------------------+
                 | System.Agents.Host.Runner                 |   per-session lock, live runs,
                 |   SessionRunner                           |   events, cancellation, recovery
                 +---------+-------------------+-------------+
                           |                   |
        +------------------v-----+     +-------v----------------------+
        | System.Agents.Host     |     | SessionBackend (extended)    |
        |   Host, on top of      |     | ContinuationStore            |
        |   AgentFactory         |     | SQLite, one database file    |
        +------------------------+     +------------------------------+

The library adds three modules:

System.Agents.AgentFactory: the single agent factory (done).
System.Agents.Host: loaded agents, database, and stores.
System.Agents.Host.Runner: session lifecycle on top of a Host.

None of them imports anything HTTP-related. The HTTP layer is a new executable.

1. Keys: one id per session

Rule: a session’s SessionId is its only key. Wherever a ConversationId is needed for the same session, it is sessionIdToConversationId sid. The Host API never generates a separate ConversationId for a session.

Sub-agent sessions get their own SessionId and record their parent (§3).

2. The agent factory and the Host

2.1 `System.Agents.AgentFactory` (implemented, Phases 1–2)

data AgentDeps = AgentDeps
    { adApiKeys           :: LoadedApiKeys
    , adSessionSink       :: SessionSink
    , adContinuationStore :: Maybe ContinuationStore
    , adToolCache         :: Maybe ToolCache
    , adCompletion        :: Maybe (OSAgentNode -> Completion)
    -- ^ Replaces the LLM call (tests pass a mock).
    }
defaultAgentDeps :: LoadedApiKeys -> AgentDeps               -- SinkNone
fileAgentDeps    :: SessionStore -> LoadedApiKeys -> AgentDeps  -- SinkFiles

data AgentRole
    = RootAgent
    | SubAgent { subParentConversation :: ConversationId, subCallStack :: [CallStackEntry] }

-- | The only way to turn an OSAgentNode into a runnable Agent.
buildAgent :: Tracer IO Trace -> AgentDeps -> AgentRole -> ConversationId -> OSAgentNode
           -> IO (Agent (LlmTurnContent, Session))

buildAgent takes a ConversationId, not a SessionId: the CLI and TUI key their file store by conversation ID, and changing that is not part of this work. Callers that follow the §1 rule pass sessionIdToConversationId sid.

What buildAgent does, in this order:

Builds the base Agent record: prompt, tools, toolCall, toolPortal, and complete (from adCompletion, or else the agent’s OpenAI config).
Applies applyAgentDurableConfig for the node’s JSON config, and installs adContinuationStore, adToolCache, and (for SinkBackend) the backend.
Applies agentEvaluateActiveTools (progressive disclosure) to every agent.
Wraps the agent with agentPersistSession adSessionSink convId last (fixes G1). SessionSink lives in Combinators.StoreSessionProgress:
```
data SessionSink = SinkBackend SessionBackend | SinkFiles SessionStore | SinkNone
agentPersistSession :: SessionSink -> ConversationId -> Agent r -> Agent r
```
agentStoreSession keeps its old behaviour for existing callers.
Never writes to stdout or stderr. OneShot.nodeToAgentWithThinking adds the thinking printer, the media injection, and the extra --session-file copy as decorators on the result.

All front-ends use it: one-shot run and the TUI (through OneShot.nodeToAgent), sub-agent tools, the MCP server, and the durable session commands.

Sub-agents: turnAgentRuntimeIntoIOTool takes an AgentDeps instead of a SessionStore and API keys. It builds the sub-agent with SubAgent parentConvId stack and the same conversation ID it uses for the call stack and the OS World entity. That ID now also names the stored sub-session; it used to be an unrelated random ID. The parent link is kept at runtime (ctxParentConversation), and backends store it as parent_session_id (Phase 3).

2.2 `System.Agents.Host` (implemented, Phase 5)

data Host = Host
    { hostAgents         :: Map Text OSAgentNode   -- root agents, by slug
    , hostDeps           :: AgentDeps              -- root agents: SinkNone, the runner stores
    , hostSubAgentDeps   :: AgentDeps              -- sub-agents: SinkBackend hostBackend
    , hostBackend        :: SessionBackend
    , hostContinuations  :: ContinuationStore
    , hostTracer         :: Tracer IO HostTrace
    , hostLiveSessionTtl :: NominalDiffTime
    }

data HostConfig = HostConfig
    { hcAgentFiles :: [FilePath], hcApiKeysFile :: FilePath, hcDatabasePath :: FilePath
    , hcCompletion :: Maybe (OSAgentNode -> Completion), hcLiveSessionTtl :: NominalDiffTime }
defaultHostConfig :: [FilePath] -> FilePath -> FilePath -> HostConfig   -- TTL 15 minutes

withHost :: HostConfig -> Tracer IO HostTrace -> (Host -> IO a) -> IO a

withHost opens the database with journal_mode = WAL and busy_timeout = 5000, runs the session and continuation migrations, loads each agent file (sub-agent tools get hostSubAgentDeps, and session tools a backendCatalog), and refuses duplicate root slugs (HostError). Both dependency sets share the continuation store and the completion override. The bundled SQLite is built with THREADSAFE=1 (serialized), so one connection is shared by all threads; no statement sequence relies on changes().

3. Storage

3.1 Extended `SessionBackend` (implemented, Phase 3)

SessionStatus, sessionStatusOf, isBlockedOnDeferredCalls, and hasBackgroundCalls live in Session.Types (pure, re-exported by Session.Base); the metadata types live in SessionStore.

data SessionStatus = StatusIdle | StatusReady | StatusRunning | StatusWaitingExternal | StatusFailed

data SessionLabels = SessionLabels   -- Nothing keeps the stored value
    { slAgent :: Maybe Text, slParent :: Maybe SessionId, slOwner :: Maybe Text }

data SessionMeta = SessionMeta
    { smSessionId    :: SessionId
    , smAgent        :: Maybe Text
    , smParent       :: Maybe SessionId
    , smOwner        :: Maybe Text          -- reserved, unused for now
    , smStatus       :: SessionStatus
    , smStatusDetail :: Maybe Text          -- why it failed
    , smVersion      :: Int                 -- incremented on every write; 0 = never stored
    , smCreatedAt, smUpdatedAt :: UTCTime
    }

data SessionQuery = SessionQuery
    { sqAgent :: Maybe Text, sqStatuses :: Maybe [SessionStatus], sqParent :: Maybe SessionId
    , sqUpdatedBefore :: Maybe UTCTime, sqLimit :: Maybe Int }

data VersionConflict = VersionConflict { vcSessionId :: SessionId, vcExpected, vcActual :: Int }

data SessionBackend = SessionBackend
    { sbStore  :: SessionId -> Session -> IO ()     -- unconditional
    , sbLoad   :: SessionId -> IO (Maybe Session)
    , sbList   :: IO [(SessionId, UTCTime)]
    , sbDelete :: SessionId -> IO ()
    , sbStoreLabelled   :: SessionLabels -> SessionId -> Session -> IO ()
    , sbLoadMeta        :: SessionId -> IO (Maybe (Session, SessionMeta))
    , sbCompareAndStore :: SessionMeta -> Session -> IO (Either VersionConflict SessionMeta)
    , sbQuery           :: SessionQuery -> IO [SessionMeta]
    }

Unconditional stores (sbStore, sbStoreLabelled) increment the version, keep stored labels where the new ones are Nothing, and set the status from sessionStatusOf, unless the stored status is StatusRunning. sbCompareAndStore writes the given metadata exactly, only if the stored version equals smVersion (a missing session counts as 0).

Backend implementations:

SQLite: compare-and-store is one conditional statement (UPDATE … WHERE version = ? RETURNING …, or an upsert with DO UPDATE … WHERE sessions.version = 0 for version 0), so it is atomic without a lock. Rows from before the migration have version 0.
File: metadata in a sidecar meta.<uuid>.json (not conv.*, so session listings ignore it). A session file without a sidecar reads as version 0. Compare-and-store is read-compare-write, not atomic across processes.
Composite: writes and queries go to the primary backend; loads fall back in order.

AgentFactory stores through sbStoreLabelled with the agent’s slug and, for sub-agents, the parent session. A sub-agent’s session ID is its conversation ID, so its own sub-agents name it correctly as their parent.

3.2 SQLite schema and migrations (implemented, Phase 3)

SessionStore.runMigrations conn component migrations applies each not-yet-applied Migration in a transaction and records it in schema_migrations(component, version, applied_at), so the continuation store and tool cache can have their own migration lists later. initializeSessionSchema (called by mkSqliteSessionStore) runs the sessions migrations:

-- migration 1: the original table and index (CREATE IF NOT EXISTS)
-- migration 2
ALTER TABLE sessions ADD COLUMN agent_slug TEXT;
ALTER TABLE sessions ADD COLUMN parent_session_id TEXT;
ALTER TABLE sessions ADD COLUMN owner TEXT;
ALTER TABLE sessions ADD COLUMN status TEXT NOT NULL DEFAULT 'ready';
ALTER TABLE sessions ADD COLUMN status_detail TEXT;
ALTER TABLE sessions ADD COLUMN version INTEGER NOT NULL DEFAULT 0;
CREATE INDEX IF NOT EXISTS idx_sessions_status ON sessions(status, updated_at);
CREATE INDEX IF NOT EXISTS idx_sessions_parent ON sessions(parent_session_id);
-- then each existing row's status is derived from its JSON

The tool_continuations(session_id, completed_at) index belongs to the continuation store’s migrations (Phase 4). Connection settings (journal_mode=WAL, busy_timeout) are set by withHost when it opens the database (Phase 5), not by the library backends, which work on connections their callers own.

3.3 Reading sessions from tools (G4, implemented, Phase 3)

A read-only interface both stores implement:

data CatalogEntry = CatalogEntry
    { ceConversationId :: ConversationId, ceUpdatedAt :: Maybe UTCTime
    , ceSession :: Maybe Session   -- Nothing when unreadable (locked file)
    , ceBusy :: Bool }             -- locked file, or status running
data SessionCatalog = SessionCatalog
    { catList :: IO [CatalogEntry], catRead :: ConversationId -> IO (Maybe Session) }
fileCatalog    :: SessionStore -> SessionCatalog
backendCatalog :: SessionBackend -> SessionCatalog

Entries are keyed by conversation ID, the vocabulary of the session tools (and the same UUID as the session ID in a backend). SessionIntrospectionConfig.introspectionCatalog and AgentTree.Props.sessionCatalog take a catalog; the CLI and TUI pass fileCatalog store, and the Host will pass backendCatalog.

The search index (Session/Search) stays on the file SessionStore: only the session-index and session-search CLI commands use it, and it is built around file paths and modification times.

3.4 Continuations stay consistent (G7, implemented, Phase 4)

The session JSON is the source of truth. tool_continuations is an index from token to session.

ContinuationStore.csFindSession :: ContinuationToken -> IO (Maybe SessionId) answers for pending and completed tokens (csLoad only returns pending ones, so it cannot tell a completed token from an unknown one).
Session.Wake.wakeSessionWith :: Maybe ContinuationStore -> Maybe ToolCache -> Session -> [(ContinuationToken, UserToolResponse)] -> IO WakeOutcome reports, per token, whether it was applied, already completed, or unknown, and calls csComplete for the applied ones. A token is already completed when a call of the session still carries it but is no longer deferred, or when the store knows it for this session (once a turn is complete, the session no longer holds its tokens). wakeSession and wakeSessionWithCache are wrappers returning the session.
Session.Wake.findSessionForToken :: Maybe ContinuationStore -> SessionBackend -> ContinuationToken -> IO (Maybe SessionId) asks csFindSession first and scans the backend’s sessions only for tokens the store does not know.
csComplete uses UPDATE … RETURNING instead of SELECT changes(), which another user of the same connection could change in between.
The continuation table’s schema goes through runMigrations (component continuations); migration 2 adds the (session_id, completed_at) index.

The CLI complete command keeps its scan over the file store: CLI agents have no continuation store. The runner (§4.3) uses findSessionForToken and wakeSessionWith.

Agents store their session before each step, so the session a run stops in is not stored by the agent: callers store it (the session commands and one-shot run do; the runner stores after every step, §4.2).

4. `System.Agents.Host.Runner`: session lifecycle (implemented, Phase 5)

data SessionRunner  -- opaque

newSessionRunner      :: Host -> IO SessionRunner
shutdownSessionRunner :: SessionRunner -> IO ()   -- cancels active runs, stores their sessions
withSessionRunner     :: Host -> (SessionRunner -> IO a) -> IO a
runnerStats           :: SessionRunner -> IO RunnerStats   -- live sessions, active runs

data RunMode = StepOnce | UntilBlocked

createSession :: SessionRunner -> Text -> NewMessage -> Maybe RunMode -> IO (Either RunnerError SessionMeta)
postMessage   :: SessionRunner -> SessionId -> NewMessage -> Maybe RunMode -> IO (Either RunnerError SessionMeta)
resume        :: SessionRunner -> SessionId -> RunMode -> IO (Either RunnerError SessionMeta)
completeCall  :: SessionRunner -> ContinuationToken -> UserToolResponse -> Bool {- auto-resume -} -> IO (Either RunnerError SessionMeta)
cancelRun     :: SessionRunner -> SessionId -> IO (Either RunnerError SessionMeta)
getSession    :: SessionRunner -> SessionId -> IO (Maybe (Session, SessionMeta))
awaitRun      :: SessionRunner -> SessionId -> NominalDiffTime -> IO (Either RunnerError (SessionMeta, Bool {- run still active -}))
deleteSession :: SessionRunner -> SessionId -> DeleteMode -> IO (Either RunnerError DeletionPlan)
subscribe     :: SessionRunner -> SessionId -> IO (IO SessionEvent)   -- blocking "next event"
recoverOnStartup :: SessionRunner -> IO [SessionId]

data NewMessage = NewMessage { nmText :: Text, nmMedia :: [MediaAttachment] }

data DeleteMode = DryRun | DeleteForReal

data DeletionPlan = DeletionPlan
    { dpSessions      :: [SessionId]   -- ^ the session and all descendants, deepest first
    , dpContinuations :: Int           -- ^ continuation rows removed (or that would be)
    , dpDryRun        :: Bool
    }

data RunnerError
    = UnknownAgent Text | UnknownSession SessionId | UnknownToken ContinuationToken
    | TokenAlreadyCompleted ContinuationToken
    | RunInProgress SessionId          -- ^ a run owns the session (on delete: one of the tree, or an ancestor)
    | NoActiveRun SessionId            -- ^ cancel without a run (HTTP 409 no_active_run)
    | NotAcceptingMessages SessionId SessionStatus
    | Conflict VersionConflict

Agents built by the runner always run asynchronously (withExecutionMode Asynchronous), like the CLI session commands, so that runs can stop and resume.

4.1 Per-session state

data LiveSession = LiveSession
    { lsLock        :: MVar ()                          -- serialises every change
    , lsRun         :: TVar (Maybe (Async ()))          -- the active run, if any
    , lsAgent       :: TVar (Maybe (Agent …))           -- kept between runs (G8)
    , lsLatest      :: TVar (Maybe (Session, SessionMeta))  -- last version stored or loaded
    , lsInbox       :: TVar [(ContinuationToken, UserToolResponse)]  -- see 4.3
    , lsLastTouched :: TVar UTCTime
    , lsEvicted     :: TVar Bool
    }
-- held in: TVar (Map SessionId LiveSession)

The runner creates a LiveSession on first access. A reaper thread (every half TTL, between 50 ms and 60 s) evicts sessions idle for longer than hostLiveSessionTtl that have no active run and no Running call, and shuts their engine down. An operation that took a LiveSession just before it was evicted notices lsEvicted once it holds the lock, and retries with a fresh one.
The first run of a session builds its agent with buildAgent and keeps it in lsAgent. runStepM installs a World and an engine on demand and returns the agent holding them; the runner keeps that agent, so the engine survives between runs. The runner has its own loop over runStepM, with the stop conditions of runUntilBlocked, and shuts the engine down only on cancel, eviction, deletion, or runner shutdown.
Running calls whose engine was lost (restart or eviction) are resolved as orphaned by pollRunningCall on the next step. No new code is needed.

4.2 Mutation protocol

Every operation that changes a session:

Takes lsLock.
If a run is active and the operation is not cancelRun, returns RunInProgress. completeCall is the exception: see 4.3.
sbLoadMeta, then applies the change (wakeSessionWith, or pushes a UserTurn), then sbCompareAndStore.
Emits a SessionEvent, then releases the lock.

A run takes the lock for each step’s store, not for the whole run. Inside the run loop each step: runStepM, then sbCompareAndStore under lsLock, then emit session.updated. A VersionConflict inside a run stops it with StatusFailed with detail “concurrent modification”. That cannot happen with a single server process; it guards against a second process or the CLI writing to the same database.

The runner sets status = running when a run starts and sessionStatusOf s when it ends. On an exception it sets StatusFailed with displayException e as detail. Progress storage installed by buildAgent (§2.5) is disabled for runner-built agents (SinkNone), because the runner does the versioned stores itself.

4.3 Completing a deferred call during a run

runUntilBlocked only stops once nothing can progress in-process, so an active run and a pending deferred call can coexist. For example, a deferred call waits while a background call is still running.

Writing the woken session while the run is in a step would make the run’s next versioned store conflict. So completeCall on a session with an active run, under the lock:

checks the token against the run’s latest version (and the queue), and answers UnknownToken or TokenAlreadyCompleted right away;
otherwise appends the result to lsInbox and returns.

Before each step and before deciding to stop, the run, under the lock, applies the queued results with wakeSessionWith (which also marks the continuations completed) and stores the result. A run that was about to stop on deferred calls therefore carries on when their results are queued. cancelRun applies the queue too.

Without an active run, completeCall applies the result and stores it at once, and with auto-resume starts an UntilBlocked run when the session is ready. Two concurrent completions of one turn are serialised by the lock: the first stores, the second sees the first’s version (tested).

4.4 Follow-up messages

postMessage is valid only when the status is StatusIdle (head is a final LlmTurn). It pushes

UserTurn (UserTurnContent sysPrompt sysTools (Just (UserQuery text media)) []) Nothing

onto turns (newest first, as handleStart does), taking sysPrompt and sysTools from the live agent. naiveTilNoToolCallStep then sends it to the LLM on the next step (Step.hs:1081). Agents built by the runner keep usrQuery = pure Nothing.

The code that builds the first turn from a prompt moves out of CLI/SessionDurable.handleStart into Session.Types.newSessionFromPrompt :: SystemPrompt -> [SystemTool] -> NewMessage -> IO Session (fixes G10). createSession and handleStart both use it.

4.5 Events

data SessionEvent
    = RunStarted   SessionId RunMode
    | SessionUpdated SessionId SessionMeta Turn   -- ^ the new head turn
    | CallsDeferred SessionId [DeferredCallView]
    | RunStopped   SessionId SessionStatus
    | SessionFailed SessionId Text

DeferredCallView is the same data agents-exe session pending prints: tool name, call id, token, disposition, and the call with its arguments. It lives in Session.Types with pendingDeferredCalls :: Session -> [DeferredCallView]; the CLI’s extractDeferredCalls wraps it.

Every stored version emits SessionUpdated (with the head turn), including the stores at the start and end of a run. Events go through one runner-wide broadcast channel: subscribe returns an action yielding the next event of one session, so a subscriber keeps its stream when the session is evicted and loaded again. subscribeSTM gives the same stream as a transaction that consumes one event of any session and yields Nothing for another session’s, to combine with timers or flags (the HTTP events stream uses it). Filtering must consume other sessions’ events one transaction at a time: a retry after readTChan rolls the read back, which in the first version of subscribe blocked a subscriber for good on another session’s event. Each event is also traced as HostRunnerTrace kind sid.

4.6 Startup recovery

recoverOnStartup queries status = running. For each session found that this runner is not running itself:

Load it.
Set its status to sessionStatusOf s.
Emit nothing, since no subscribers exist yet.

It does not resume automatically. Rows left running mean the process died mid-step. The partially started step is lost, and any Running calls become orphaned on the next resume. The server logs how many sessions it recovered. Automatic resume can be added later as --resume-interrupted.

4.7 Cancellation

cancelRun cancels the run’s Async without holding the lock (the run takes it to store its steps), then, under the lock and only if that run still owns the session:

shuts the engine down, which marks background calls cancelled in the World, and keeps the agent (and World) with no engine, so the next run gets a fresh engine on the same World;
applies queued external results;
refreshes the head partial turn from the World, so its cancelled calls become Failed "async tool call was cancelled";
stores the session with the status its turns imply, and emits RunStopped.

Background calls below the head (the LLM already got a placeholder for them) stay Running in the stored session. The next run’s late-result collection finds them cancelled in the kept World and tells the LLM, in a user message, as for any late result (tested). If the session is evicted first, they are reported as orphaned instead. Sub-agent runs execute inside the parent’s tool call, so cancelling the parent cancels them.

4.8 Waiting for a run

awaitRun blocks until the session’s active run stops or the timeout expires, whichever comes first. It returns the current meta and whether a run is still active. With no active run it returns at once. It watches lsRun (STM waitCatch on the Async, raced against a timer), so waiting never holds lsLock. A waiter going away (for example an HTTP client disconnecting) never cancels the run.

4.9 Deleting sessions

Deletion always cascades: a session goes together with every descendant (sub-agent sessions, found recursively through sbQuery on parent) and their continuation rows. deleteSession:

Collects the tree and builds a DeletionPlan, deepest sessions first.
Returns RunInProgress if any session in the tree has an active run. This applies in both modes, so a dry run shows whether the real delete would succeed.
With DryRun, returns the plan and changes nothing.
With DeleteForReal, walks the plan in order. For each session it takes that session’s lsLock, deletes its continuation rows, calls sbDelete, drops its LiveSession (shutting down its engine), and releases the lock.

Deleting leaves first means a crash part-way through never leaves a sub-session whose parent is gone; running the delete again finishes the job. This needs two new ContinuationStore fields: csCountSession :: SessionId -> IO Int for dry runs, and csDeleteSession :: SessionId -> IO Int, which deletes every row of a session (pending or completed) and returns the count. Deletion is also refused while an ancestor of the session has an active run: its sub-agent calls write into the tree, and would recreate what was deleted. The runner does the cascade, not SQL foreign keys, so the file backend behaves the same.

5. HTTP API (`agents-server`) (implemented, Phase 6)

The server lives in examples/agents-server/, on wai and warp. The application code is a private sub-library, agents-server-internal (AgentsServer.Api, AgentsServer.Log, AgentsServer.Server), shared by the agents-server executable and its agents-server-tests suite; agents-lib does not depend on wai or warp. SSE is written by hand with responseStream (no wai-extra). All bodies are JSON. Session objects embed the raw Session JSON that is already stored, so clients can reuse session-print logic. The user guide is documentation/agents-server.md.

agents-server --agent-file a.json [--agent-file b.json …] --api-keys keys.json \
              [--db ./agents-server.db] [--port 8080] [--bind 127.0.0.1] \
              [--live-session-ttl 900] [--shutdown-grace 10]

The server binds to 127.0.0.1 by default. There is no authentication in milestone 1 (see non-goals); the server.started log line says so.

Method & path	Body	Success	Errors
`GET /v1/agents`		`200 [{slug, description, tools:[name]}]`
`POST /v1/sessions?wait=&timeout=`	`{agent, prompt, media?:[{mime, base64, filename?}], run?: "none"\\|"step"\\|"until_blocked"}` (default `until_blocked`)	`201 SessionView`, `Location` header	404 `unknown_agent`, 400 `bad_request`
`GET /v1/sessions?agent=&status=&parent=&limit=&before=`		`200 {sessions:[SessionMetaView], next_before}`	400
`GET /v1/sessions/:id`		`200 SessionView`	404 `unknown_session`
`POST /v1/sessions/:id/messages?wait=&timeout=`	`{prompt, media?, run?}`	`202`/`200 SessionView`	404, 409 `run_in_progress`, 409 `not_accepting_messages`
`POST /v1/sessions/:id/resume?wait=&timeout=`	`{mode?: "step"\\|"until_blocked"}` or no body	`202`/`200 SessionView`	404, 409 `run_in_progress`
`POST /v1/sessions/:id/cancel`		`200 SessionMetaView`	404, 409 `no_active_run`
`GET /v1/sessions/:id/pending`		`200 {calls:[DeferredCallView]}`	404
`POST /v1/continuations/:token?wait=&timeout=`	`{result: UserToolResponse \\| string, resume?: bool}` (default `true`)	`202`/`200 SessionView`	404 `unknown_token`, 409 `token_already_completed`, 409 `conflict`
`GET /v1/sessions/:id/events`		`200 text/event-stream`	404
`DELETE /v1/sessions/:id?dry_run=`		`200 {sessions:[id], continuations, dry_run}`	404, 409 `run_in_progress` (also for a dry run, when the real delete would fail)
`GET /healthz`		`200 {ok:true, live_sessions, active_runs}`

Other errors: 404 not_found (unknown path), 405 method_not_allowed, 413 payload_too_large (bodies over 32 MiB), 500 internal_error. A malformed session id answers 404 unknown_session, a malformed token 404 unknown_token.

SessionMetaView is the SessionMeta JSON: {session_id, agent, parent_session_id, owner, status, status_detail, version, created_at, updated_at} (owner is always null in milestone 1). SessionView = SessionMetaView + {session: <Session JSON>, pending: [DeferredCallView]}. Errors are {error: "<code>", message: "<text>"}.

Listing. Newest first by updated_at. status takes a comma-separated list; limit is 1–500, default 50. When a page is full, next_before is the updated_at of its last session, used as before= (strictly older) for the next page. A session with exactly the boundary’s updated_at would be skipped; timestamps have nanosecond precision, so that takes two writes in the same instant.

Waiting. Every endpoint that can start a run (create, messages, resume, continuations) takes the same two query parameters:

wait=true|false, default false (?wait alone means true). With false, the server answers as soon as the run has started. With true, it answers when the run stops (idle, blocked on deferred calls, or failed), using awaitRun.
timeout=<seconds>, only used with wait=true. Default 120, maximum 600; larger values are clamped. If it expires, the server answers with the current state (status: "running") and the run carries on. Clients then follow it through the events stream or by polling.

The response body is always the SessionView at the time of answering, so a waiting client gets the final turn and any pending deferred calls in one round trip. A client disconnecting while waiting does not cancel the run. Creation always answers 201. The other endpoints answer 202 when the stored status is running at the time of answering (a run is still going, including a continuation queued into an active run) and 200 otherwise. On shutdown, waiting requests answer at once with the current state.

Deleting. dry_run=true returns the same body as a real delete, listing every session and the number of continuation rows that would be removed, and changes nothing.

SSE stream. Each SessionEvent becomes event: <kind> plus one data: <json> line, where kind is run.started ({session_id, mode}), session.updated (SessionMetaView + head_turn), calls.deferred ({session_id, calls}), run.stopped ({session_id, status}), or session.failed ({session_id, message}). On connect the server first sends event: snapshot with the SessionMetaView, so clients never need a separate GET to sync; it subscribes before loading the snapshot, so no event falls in between. A run’s session.updated for the running version comes just before its run.started. The server sends a : keepalive comment after 15 s without events (warp pauses its idle timeout while the handler runs, so quiet streams and long waits are not cut). Streams end on shutdown. Events are not replayed on reconnect. The stream reads events with subscribeSTM, combined with the keepalive timer and the shutdown flag in one transaction, so a timer firing never drops an event.

Shutdown. On SIGTERM or SIGINT: stop accepting connections, end event streams and release waiting requests, give open requests --shutdown-grace seconds, then shutdownSessionRunner (cancels active runs, storing their sessions) and close the database.

6. Tracing and logs (implemented, Phase 6)

HostTrace wraps the existing traces (OneShot.Trace, tool registration, OpenAI) plus runner events. The server prints them as JSON lines on stderr, one object per line with ts, kind, and session_id when known. AgentsServer.Log summarises each trace field by field instead of showing it: LLM traces give byte and token counts, the HTTP client trace gives method, host, path, and status only (its request carries the API key in a header), and tool and agent-tree traces give their constructor name. Prompts, payloads, headers, and API keys are never printed; a smoke test with a fake key checked the log. Each HTTP request logs method, path, status, and time to first byte.

Phases

Each phase leaves cabal build all and cabal test agents-tests passing with -Wall -Werror, and gets its own entry in a todos/web-server-embedding.progress.md tracker.

Phase 1: storage wiring fix (G1, G5) ✅

SessionSink and agentPersistSession; keep agentStoreSession as a wrapper.
session start derives its conversation ID from the session ID.
Tests: an agent built with a SQLite backend and a mock LLM runs to completion and writes no files under a temp session directory; the row is keyed by the session’s SessionId.

Phase 2: single agent factory (G2, G3, G10) ✅

System.Agents.AgentFactory with AgentDeps, buildAgent, and AgentRole; the three copies moved onto it; sub-agents get their parent and call stack from SubAgent.
newSessionFromPrompt in the library.
Tests: the existing suite unchanged; new AgentFactoryTests.

Phase 3: metadata, versions, migrations, catalog (G4, G9) ✅

Extended SessionBackend, SessionLabels, SessionMeta, sessionStatusOf, SQLite migrations, and the file sidecar.
SessionCatalog; the session tools and Props switched to it.
Sub-agents store their agent slug and parent_session_id.
Tests (SessionMetadataTests): status derivation; version increments; labels kept; CAS success and conflicts (stale, and version 0 twice); running status kept by unconditional stores; sbQuery filters and limit; composite fallback; migrating a database from before metadata; migrations run once; file sidecar and legacy files; file and backend catalogs; list-sessions over a backend catalog; a parent agent calling a sub-agent tool (mock LLM) leaves a sub-session row naming the parent.

Phase 4: continuation consistency (G7) ✅

csFindSession, wakeSessionWith / WakeOutcome, findSessionForToken, continuation migrations, RETURNING in csComplete.
Tests (ContinuationConsistencyTests, on a real paused run: async agent, defer policy, mock LLM, one SQLite database): a wake marks the continuation completed; a second wake reports the token as already completed and leaves the session unchanged; an unknown token is reported; the woken session resumes to a final answer; a token lookup through the index loads no session; without an index the sessions are searched; migrations run once.

Phase 5: `SessionRunner` (G6, G8) ✅

System.Agents.Host (withHost) and System.Agents.Host.Runner (§4).
DeferredCallView / pendingDeferredCalls; csCountSession and csDeleteSession.
Tests (RunnerTests, mock LLM through the host’s completion override): deferred call completed with auto-resume, with the event sequence; two concurrent completions of one turn both land; messages refused during a run and accepted after, giving a second LLM turn; a background call started in one run picked up by the next; cancel during a background call, then the cancellation reported on the next run; recovery of a session left running, its call orphaned on resume; awaitRun with timeout and completion; delete cascade (refused during the parent’s run, also for the sub-session; dry run changes nothing; real delete removes sessions and continuation rows); idle eviction and reload; withHost over agent and database files.

Phase 6: `agents-server` executable ✅

wai/warp app, routes, SSE, CLI flags, graceful shutdown on SIGTERM (stop accepting, shutdownSessionRunner, close the database).
documentation/agents-server.md user guide, plus links from documentation/durable-workflows-howto.md.
Tests (agents-server-tests, threaded, the application on a random port with a mock LLM): the demo flow over SSE (snapshot, then per run run.started … calls.deferred, run.stopped, then the continuation’s run ending idle); the same flow with wait=true and no events stream (the create call returns the blocked session with its pending call, the continuation call the final answer); keepalives; listing pages and filters; delete with dry run; agents and health; error codes; shutdown releasing waiting requests and ending streams.

Milestone 2: the later work

Recorded 2026-09-19, when the user asked to continue with the later work. Each item becomes a phase, in this order: each phase is useful on its own, and the riskier ones come after the ones they build on. Choices marked default were made without asking; they are listed again under Decisions.

Phase 7: authentication and owners ✅

agents-server --auth-tokens tokens.json: {"tokens": [{"owner": "alice", "sha256": "<hex>"}, {"owner": "bob", "token": "<plain>"}]}. With it, every endpoint but /healthz needs Authorization: Bearer <token> (401 unauthorized otherwise, with WWW-Authenticate: Bearer). Without it, nothing changes. Tokens are compared by SHA-256 digest (default: a static file, like the API keys file; no token issuing or expiry).
Sessions created by a caller record its owner (createSessionAs). Sub-sessions record none: a session belongs to the owner of its root session (sessionOwner walks up parent_session_id). This needs no change to how sub-agents store themselves.
Another owner’s session answers 404, as if it did not exist, for every endpoint including continuations (404 unknown_token). Listing filters by owner (SessionQuery.sqOwner, index (owner, updated_at), migration 3); ?parent= lists the sub-sessions of an owned session.
Sessions stored before authentication was turned on have no owner and are invisible to every caller (default).
Not in this phase: per-owner API keys and a default runIsolated policy for bash and MCP tools. Both need agents built per owner, including sub-agent tools, which the host builds once at load time today. They stay in Remaining later work.

Phase 8: MCP over HTTP ✅

POST /mcp on agents-server (AgentsServer.Mcp) speaks MCP’s Streamable HTTP transport. It handles JSON-RPC with aeson directly and reuses the MCP/Base.hs types for tools, tool lists, and the initialize result; tool results are written by hand, because TextContentImpl always writes "annotations": null. Each request, or batch, gets a plain JSON response. Messages that need no answer (notifications, client responses) get 202. GET /mcp answers 405: there are no server-initiated messages. Protocol versions 2025-06-18, 2025-03-26, and 2024-11-05: the client’s is echoed when supported, else the latest.
Methods: initialize, ping, tools/list, tools/call, and empty resources/list and prompts/list. Anything else is -32601; a bad or unknown tool is -32602; malformed JSON answers 400 with -32700.
Tools: one ask_<slug> per root agent, input {prompt}. A call creates a session through the runner, owned by the caller, and waits up to 120 s. The result carries _meta.session_id. An idle session gives the final answer; waiting_external gives a sentence plus the pending calls as JSON (not an error); a run still going gives the session id; failed gives isError: true.
The same bearer tokens and owners apply. Mcp-Session-Id is not used.
The Streamable HTTP transport requires validating Origin against DNS rebinding. Without authentication, every endpoint (not only /mcp) refuses an Origin other than localhost, 127.0.0.1, or [::1] with 403 forbidden_origin. With authentication, origins are not checked.

Phase 9: TUI out of the core library ✅

New public sub-library agents-tui (source directory tui/) with the 25 modules that need brick or vty, or import them: System.Agents.TUI.* (except TUI.ToolCallActivity, which is pure and used by the tests), System.Agents.CLI.TUI, System.Agents.CLI.Config, and System.Agents.CLI. Module names do not change. Only agents-exe imports them.
agents-lib drops brick, vty, text-zipper, and the unused data-clist; nothing in its dependency closure pulls brick or vty any more. agents-exe depends on both libraries.
The moved files live in their own source directory: with a shared src/, GHC would compile core modules again inside agents-tui instead of using agents-lib.
Default: agents-lib itself is the core instead of a new agents-core name, so nothing that depends on agents-lib breaks.

Phase 10: Postgres backend ✅

Public sub-library agents-postgres (postgres/System/Agents/Postgres.hs, on postgresql-simple and the resource-pool already used by agents-lib): withPostgresStores, openPostgresPool (10 connections), mkPostgresSessionStore, mkPostgresContinuationStore, isPostgresUrl, runPostgresMigrations.
Same schema and semantics as SQLite. Postgres starts at migration 1 with the full current schema: TIMESTAMPTZ, and session JSON as TEXT, not JSONB, which rejects \u0000. CAS is INSERT … ON CONFLICT DO UPDATE … WHERE version = 0 RETURNING or UPDATE … WHERE version = ? RETURNING. Timestamps are truncated to microseconds, as Postgres stores them, so the metadata handed out matches what a query returns.
Migrations run in one transaction per component, under a pg_advisory_xact_lock, with client_min_messages = warning so that Postgres notices do not reach stderr.
System.Agents.Host.withHostStores takes HostStores from the caller; withHost is the SQLite case. agents-server --db takes a postgres:///postgresql:// URL; the startup log drops its user and password.
Tests (agents-postgres-tests, threaded): a throwaway cluster (initdb + pg_ctl from pg_config --bindir, a free port on 127.0.0.1), one fresh database per test; AGENTS_TEST_POSTGRES_URL uses an existing server instead, and without either the suite skips. They cover migrations running once; CAS with conflicts, labels, and the kept running status; queries (agent, owner, parent, status, limit, before); 16 concurrent CAS with exactly one winner; and a runner flow (deferred call, completion, token already completed, cascade delete).
Coordinating runs between servers came later: see “Several server processes on one Postgres database” under Remaining later work.

Phase 11: token streaming ✅

System.Agents.LLMs.OpenAIStream: reads a streamed chat completion (server-sent events, data: [DONE]) and folds the chunks back into the JSON of a non-streamed completion. It handles content, reasoning, tool calls assembled by index from name and argument pieces, finish_reason, the final usage chunk, and an error object sent mid-stream. Parsing, tool calls, and storage do not change. Each non-empty text delta goes to a callback.
HttpClient.Runtime.postStream: POST with a body reader for successful answers (others are read whole, so the overloaded-retry check still works). OpenAI.callLLMPayloadStreaming shares the retry logic (withOverloadedRetry) with callLLMPayload.
OpenAICompletionConfig.cfgOnTextDelta switches a completion to streaming and adds "stream": true (plus stream_options.include_usage for the OpenAIv1 flavor only, as other providers may refuse it). AgentDeps.adOnTextDelta passes it through buildAgent.
HostConfig.hcStreamTokens / hostStreamTokens; the runner gives each session’s agent a callback emitting TextDelta sid text, which is not traced (one per token, and it is content). agents-server --stream-tokens; event: text.delta with {session_id, text}.
Tests: five unit tests of the fold (text, tool call pieces, null content, a body read in 7-byte pieces with CRLF, comments and [DONE], a stream error), and an end-to-end server test against a fake streaming endpoint (warp) that checks the deltas, the stored answer, and the request’s stream fields.

Phase 12: agents from the database ✅

System.Agents.AgentStore: StoredAgent (config, updated_at, updated_by), AgentStore (asList, asPut, asDelete), mkSqliteAgentStore (table agents, component agents), fileBasedFields. agents-postgres has mkPostgresAgentStore. HostStores.hsAgents :: Maybe AgentStore; withHost and withPostgresStores provide one.
AgentTree.loadAgentTreeFromConfig: a tree of one agent from an in-memory Agent, reusing the loader’s later phases (create, wire tools, build) on a one-node graph, with no file discovery.
Host: hostStoredAgents (loaded stored agents in a TVar, the store, a loader, and a lock serialising edits); hostAllAgents, lookupAgent, putStoredAgent, deleteStoredAgent, AgentEditError. The runner looks agents up through lookupAgent, so new sessions see changes at once.
agents-server: GET/PUT/DELETE /v1/agents/:slug, source in listings, MCP tools/list includes stored agents, and --admin-owners.
Differences from the plan:
- extraAgents is refused, like the other file-based fields: stored agents have no sub-agents yet.
- Editing needs --admin-owners, which needs --auth-tokens. The plan let anyone edit agents without authentication. An agent definition can start MCP servers, which are commands on the server’s machine, so edits are off by default.
- A file agent hides a stored agent with its slug: the stored one is skipped at startup, and the skip is logged. Refusing to start instead would leave no way to fix the database through the API.
Known limitation: replacing or deleting a stored agent does not stop MCP servers it started (the tree loader has no cleanup); they end with the server.
Tests:
- runner level: store, replace, and refuse (file-based fields, a file agent’s slug); run a session on a stored agent; reload after a restart; a file agent hiding a stored one, with the skip traced; delete persists;
- server: admin and non-admin, 201 and then 200, listing with sources, refusals, sessions and MCP on a stored agent, delete, and edits disabled by default;
- Postgres: put, replace, list, and delete.

Phase 13: self-description and a chat page ✅

The server answered 404 at / and published nothing a client could read: the only reference was documentation/agents-server.md, which a caller holding just a URL does not have.

Routes as a servant type (AgentsServer.Routes). Every endpoint whose shape OpenAPI can express is now described by DocumentedAPI, and GET /openapi.json is generated from it, so the published document cannot drift from the routes. The event stream and POST /mcp stay hand-written (AgentsServer.OpenApi): neither server-sent events nor JSON-RPC is expressible, and a wrong schema would be worse than a described one.
Bodies as types (AgentsServer.Types), carrying both the JSON encoding the API already used and an OpenAPI schema, with per-field descriptions and enums added by hand because openapi3 does not read Haddock. The turn tree stays an opaque object: it belongs to the library, not to the protocol.
A chat page at / (AgentsServer.UI): one self-contained HTML document, no build step, no assets. It is served on a loopback bind, or when --auth-tokens is on; --no-ui turns it off.
access_token on the event stream only, because EventSource cannot set headers. The request log records no query strings, so the token does not reach the logs.

Deviations from the plan:

The rewrite is staged. The servant types describe and generate the document; the wai router still dispatches. Moving dispatch onto servant needs UVerb for the 200/201/202 answers and Raw for the stream, and would rewrite 646 tested lines for no change a client can see. A test asserts the document’s paths are exactly the router’s, so the two cannot drift in the meantime.
openapi3 needed the freeze’s index-state bumped: 3.2.4 caps QuickCheck below the pinned 2.16, and 3.2.5 (which allows it) was published after the pinned index date. Every previously pinned version is unchanged.

Database agents with tools and helpers ✅

Lifts the two refusals of Phase 12 that decision 10 recorded.

Tool files are stored with the agent. StoredAgent.saFiles (contents by relative path; column files, SQLite and Postgres migration 2); asPut takes them. PUT /v1/agents/:slug reads them from a files object in the body, and config.files shows them.
Loading writes them to disk. Each load of a stored agent gets its own directory under a temporary directory of the host (materializeToolFiles); the loaded configuration’s toolDirectory and bashToolboxes paths point there. The directory is removed when the node is released, and with the host.
extraAgents name stored agents by slug (path is now optional in the JSON, and refused for a stored agent). resolveHelpers computes what an agent reaches; AgentTree.loadAgentTreeFromConfigs loads the root and those helpers as one tree, so each root has its own copy of its helpers, like roots from files.
putStoredAgentWithFiles; AgentEditError gains AgentInvalidPaths, AgentHelperError (UnknownHelper, HelperCycle), and AgentInUse.
Choices made here (default, none was planned in detail):
- Cycles are refused, self-reference included, although agent files allow them: an edit is checked against the agents stored at that time, and a helper must exist first, so a cycle can only be an error.
- Replacing a helper reloads its dependents; a dependent that fails to load again keeps its previous version, and the failure is traced.
- A helper in use cannot be deleted (409 agent_in_use).
- Only stored agents can be named, not agents from files.
- openApiToolboxes, postgrestToolboxes, skillSources, and autoEnableSkills are still refused.
Tests: runner level (a stored agent with a tool directory and a stored helper with a single tool, run end to end, across a restart, and after replacing the helper; unknown helper, cycles, paths); server (files in and out, unknown_helper, helper_cycle, agent_invalid_paths, agent_in_use, a session that runs the tool); Postgres (files round-trip).

Remaining later work

Per-owner API keys and isolation: build agents per owner, including sub-agent tools; default policy runIsolated for bash and MCP tools, backed by dockerRunner. Partly done:
- Keys: HostConfig.hcOwnerApiKeysFiles (--owner-api-keys OWNER=FILE), loaded into Host.hostOwnerApiKeys. The runner already builds one agent per session, so it picks the keys there (Runner.newAgent), from the owner of the session’s root. Sub-agents needed one fix to be covered: createSessionForNodeWith now always builds the agent from the node it was given. Before, a helper that was not also a root agent failed lookupAgent and silently fell back to the in-tool path, which is built at load time and knows no owner. That fallback remains for real failures, and gets no key at all when per-owner keys are in use.
- Isolation: System.Agents.Tools.Isolated, AgentDeps.adToolIsolation, HostConfig.hcToolIsolation (--isolate-tools docker:IMAGE|process:PATH). Enforced in buildAgent, around the tool execution rather than through ToolCallPolicy: a policy is the agent’s own (it would let an agent file opt out, and replacing it would drop defer rules), and a RunIsolated disposition without a runner runs in-process.
- Still open (owner decisions): isolation is opt-in, not the default, as there is no worker image to default to; it covers bash and MCP tool calls only; MCP servers still start on the host; no worker ships with the repository; unlisted owners use the shared keys.
~~Several server processes on one Postgres database: live-run ownership through a lease column (run_owner, run_lease_until).~~ Done:
- System.Agents.Host.Coordination: what a host needs from a shared database (acquire, renew, release a run lease; who holds it; which running sessions lost their owner; signals from the other processes). noCoordination is the single-process case (SQLite, the TUI) and changes nothing there. HostStores.hsCoordination, Host.hostCoordination.
- agents-postgres: migration 4 adds run_owner and run_lease_until to sessions; lease times are the database’s now(). Each process gets a random instance name. Session writes and accepted mail are announced with pg_notify on agents_sessions; one connection outside the pool listens and reconnects.
- The runner takes the lease in startRun and releases it wherever a run ends. A heartbeat (every third of rcLeaseTtl, default 30 s) renews, stops runs whose lease was taken (abandonRun, nothing stored), syncs the mail of running sessions, and takes over expired sessions (takeOverExpired, same recovery as recoverOnStartup, nothing resumed). recoverOnStartup only recovers sessions without a live lease. A run held elsewhere is busy: postMessage and completeCall go to mail, resume and deleteSession are refused, cancelRun posts StopRun, awaitRun waits on forwarded events.
- Writes are fenced by the versioned compare-and-store, not by the lease: a takeover stores a version, so the previous owner’s next write conflicts, and failRun stores nothing once the lease is gone.
- Mail: msAppend answers with the envelope as stored. Postgres settles seq under an advisory lock per session, since two processes each number from what they last loaded; mbSync merges what others appended.
- Events: only session.updated, run.stopped and session.deleted are forwarded to the other servers. Text deltas and tool events stay on the owner. A watch skips the forwarded copies (srRemoteEvents), so a registration known to two servers yields one mail per event.
- Not done: resuming a taken-over run by itself; sharing stored-agent edits and watch-session registrations between servers; an agents-server flag for the lease duration.
~~Database agents with files or sub-agents~~: done, see Database agents with tools and helpers.
Stopping a stored agent’s MCP servers when it is replaced or deleted.
Dispatching through servant, replacing the wai router (see Phase 13).

Decisions

Recorded 2026-09-18.

Waiting for a run is chosen per request with the wait and timeout query parameters (§5), on every endpoint that can start a run. The default is not to wait. The body’s run field still controls how far the run goes.
Deleting cascades to sub-sessions and continuation rows, and supports a dry run (?dry_run=true) that reports what would be deleted (§4.9).
Idle live sessions keep their engine for 15 minutes (hcLiveSessionTtl, configurable). A background call still running when its session is evicted ends up orphaned.
The MCP server over HTTP is wanted, but later (see Phase 8).

Recorded 2026-09-19 (defaults taken when the user asked to continue with the later work; see Milestone 2):

Authentication is a static bearer-tokens file mapping tokens (or their SHA-256) to owners. A session belongs to the owner of its root session; other owners get 404. Sessions without an owner are invisible once authentication is on.
MCP over HTTP answers each request with plain JSON and does not use MCP sessions.
agents-lib becomes the core by moving the TUI into a new agents-tui library, rather than creating an agents-core library.
Postgres lives in its own agents-postgres library; the host takes its stores from the caller.
Token streaming is opt-in (--stream-tokens).
Database agents cannot use file-based tools in their first version, nor extraAgents. Storing agents needs --admin-owners (and so authentication). A file agent hides a stored agent with the same slug.

Recorded 2026-09-20:

The OpenAPI document is generated from servant types, rather than hand-written or derived from a full servant rewrite of the dispatcher.
The chat page is loopback-only by default and off with --no-ui. It is plain HTML and JavaScript in one document, with no build step.

todos/durable-workflows.md, todos/durable-workflows.progress.md
todos/async-tool-calls.md
documentation/durable-workflows-howto.md, documentation/async-tool-calls.md, documentation/sessions.md

Internal Tool Mapping Documentation

2026-10-04T15:53:54Z

Generated from documentation/internal-tool-mapping.md, the repository is the canonical source and may be ahead of this page.

Internal Tool Mapping Documentation

Overview

This document describes how tools from various sources (OpenAPI, MCP, Bash, IO) are mapped to LLM-compatible tool names. This mapping is critical because different systems have different naming constraints.

The Problem

LLM APIs (like OpenAI) have strict requirements for function names:

Must start with a letter (a-z, A-Z)
Can only contain alphanumeric characters and underscores: [a-zA-Z0-9_]
Some providers have additional length constraints

Note: Dashes (-) and colons (:) are not allowed, even though some documentation may suggest otherwise.

However, tool identifiers from external sources may contain invalid characters:

OpenAPI operation IDs can contain dots (e.g., pet.findByStatus)
OpenAPI operation IDs can contain slashes (e.g., users/pets/get)
URLs may contain dots, colons, slashes, and other special characters (e.g., localhost:8001)
MCP tool names may contain dashes, colons, or other special characters

Current Naming Schemes

1. IO Tools

io2LLMName :: IOScript a b -> OpenAI.ToolName
io2LLMName io = OpenAI.ToolName ("io_" <> io.description.ioSlug)

Prefix: io_
Source: User-defined slug
Constraints: Assumed valid (user-controlled)

2. Bash Tools

bash2LLMName :: ScriptDescription -> OpenAI.ToolName
bash2LLMName bash = OpenAI.ToolName ("bash_" <> bash.scriptInfo.scriptSlug)

Prefix: bash_
Source: User-defined slug
Constraints: Assumed valid (user-controlled)

3. MCP Tools

mcp2LLMName :: Toolbox -> ToolDescription -> OpenAI.ToolName
mcp2LLMName box mcp = OpenAI.ToolName ("mcp_" <> box.name <> "_" <> mcp.getToolDescription.name)

Prefix: mcp_<toolboxName>_
Source: MCP tool name from server
Issue: MCP tool names may contain invalid characters (dashes, colons, etc.)

4. OpenAPI Tools

openapi2LLMName :: Text -> Text -> OpenAI.ToolName
openapi2LLMName tboxName operationId =
    OpenAI.ToolName ("openapi_" <> tboxName <> "_" <> operationId)

Prefix: openapi_<toolboxName>_
Source: OpenAPI operationId
Issue: operationId often contains dots (e.g., pet.findByStatus)
Issue: Toolbox names derived from URLs may contain colons (e.g., localhost:8001)

Error Manifestation

When an invalid tool name is sent to an LLM API, you may see errors like:

"Invalid request: function name is invalid, must start with a letter 
 and can contain only alphanumerics and underscores"

This commonly occurs with:

OpenAPI specs that use dot-notation for operation IDs:
- pet.findByStatus → Invalid (contains dot)
- users/pets/get → Invalid (contains slash)
Toolbox names derived from URLs:
- localhost:8001 → Invalid (contains colon)
- api.example.com → Invalid (contains dots)
MCP tool names with dashes or special characters:
- get-pet → Invalid (contains dash)

Proposed Solution: Normalization with Bi-directional Mapping

Core Principles

Normalization: Convert all tool names to LLM-valid format (alphanumeric + underscore only)
Bi-directional Mapping: Maintain ability to map back to original names
Consistency: Same normalization logic for registration and lookup
Collision Avoidance: Ensure unique normalized names

Normalization Function

-- | Normalize a tool name to be LLM-compatible.
-- 
-- Rules:
-- 1. Must start with a letter (prefix with 't' if needed)
-- 2. Replace all non-alphanumeric characters with underscores
-- 3. Collapse multiple consecutive underscores
-- 4. Preserve case (some providers are case-sensitive)
--
-- Examples:
-- >>> normalizeToolName "pet.findByStatus"
-- "pet_findByStatus"
-- >>> normalizeToolName "users/pets/get"
-- "users_pets_get"
-- >>> normalizeToolName "2.0_getPet"  -- starts with number
-- "t2_0_getPet"
-- >>> normalizeToolName "localhost:8001"  -- contains colon
-- "localhost_8001"
-- >>> normalizeToolName "get-pet"  -- contains dash
-- "get_pet"
normalizeToolName :: Text -> Text

Character Replacement Rules

Character	Replacement	Example
`.` (dot)	`_` (underscore)	`pet.find` → `pet_find`
`/` (slash)	`_` (underscore)	`users/pets` → `users_pets`
`:` (colon)	`_` (underscore)	`localhost:8001` → `localhost_8001`
`-` (dash)	`_` (underscore)	`get-pet` → `get_pet`
Other special chars	`_` (underscore)	`pet@home` → `pet_home`
Leading digit	Prefix with `t`	`2.0_api` → `t2_0_api`

Bi-directional Mapping Strategy

For OpenAPI tools, we need to maintain a mapping from normalized LLM names back to original operation IDs:

data ToolNameMapping = ToolNameMapping
    { originalName :: Text        -- ^ Original operationId
    , normalizedName :: Text      -- ^ LLM-safe name
    , fullLLMName :: Text         -- ^ With prefix (e.g., "openapi_pets_pet_findByStatus")
    }

-- Build mapping during tool registration
buildNameMapping :: Text -> [OpenAPITool] -> Map Text ToolNameMapping
buildNameMapping toolboxName tools = ...

-- Lookup during tool execution
findToolByLLMName :: Map Text ToolNameMapping -> OpenAI.ToolName -> Maybe OpenAPITool

Implementation in OpenAPI Toolbox

The OpenAPI toolbox should:

During Initialization:
- Normalize the toolbox name itself (may contain colons from URLs)
- Convert all operation IDs to normalized form
- Store mapping from normalized → original
- Generate LLM tool names with normalized IDs
During Tool Execution:
- Receive LLM tool call with normalized name
- Look up original operation ID from mapping
- Execute using original ID

Updated Naming Flow

OpenAPI Spec
    │
    ▼
Parse Operations
    │
    ▼
Extract operationId (e.g., "pet.findByStatus")
    │
    ▼
Normalize ──────────────────┐
    │                        │
    ▼                        │
"pet_findByStatus"          │
    │                        │
    ▼                        ▼
Add prefix:              Store mapping:
"openapi_pets_pet_findByStatus"  "pet_findByStatus" → "pet.findByStatus"
    │
    ▼
Register with LLM
    │
    ▼
LLM calls tool with name "openapi_pets_pet_findByStatus"
    │
    ▼
Lookup mapping → "pet.findByStatus"
    │
    ▼
Execute HTTP request

Implementation Plan

Phase 1: Normalization Function

Add to System.Agents.Tools.OpenAPI.Converter:

-- | Normalize a tool name for LLM compatibility.
-- Only allows alphanumeric characters and underscores.
normalizeForLLM :: Text -> Text
normalizeForLLM = collapseUnderscores . ensureLetterStart . replaceInvalid
  where
    replaceInvalid = Text.map replaceChar
    replaceChar c
        | isValidChar c = c
        | otherwise = '_'
    -- Only alphanumeric and underscore allowed (NO dashes, NO colons)
    isValidChar c = isLetter c || isDigit c || c == '_'
    
    ensureLetterStart t
        | Text.null t = "tool"
        | isLetter (Text.head t) = t
        | otherwise = "t" <> t
    
    collapseUnderscores = Text.intercalate "_" . filter (not . Text.null) . Text.splitOn "_"

Phase 2: Bi-directional Mapping

Add to System.Agents.Tools.OpenAPI.Converter:

data NameMapping = NameMapping
    { nmOriginal :: Text
    , nmNormalized :: Text
    }

buildToolNameMapping :: Text -> [OpenAPITool] -> Map Text NameMapping
buildToolNameMapping toolboxName tools = 
    Map.fromList [(nmNormalized m, m) | m <- mappings]
  where
    mappings = [ NameMapping orig (normalizeForLLM orig)
               | tool <- tools
               , let orig = getOperationIdOrFallback tool
               ]

Phase 3: Update OpenAPI Toolbox

Modify Toolbox to store name mapping:

data Toolbox = Toolbox
    { toolboxName :: Text
    , toolboxBaseUrl :: Text
    , toolboxTools :: [OpenAPITool]
    , toolboxNameMapping :: Map Text NameMapping  -- ^ normalized → original
    , httpRuntime :: HttpClient.Runtime
    , headerFunc :: Maybe (IO (Map Text Text))
    , staticHeaders :: Map Text Text
    }

Phase 4: Update Registration Functions

Update openapi2LLMName to use normalized names for both toolbox and operation:

openapi2LLMName :: Text -> Text -> OpenAI.ToolName
openapi2LLMName tboxName operationId =
    let normalizedBox = normalizeForLLM tboxName
        normalizedOp = normalizeForLLM operationId
    in OpenAI.ToolName ("openapi_" <> normalizedBox <> "_" <> normalizedOp)

Note: Both tboxName and operationId must be normalized because toolbox names may come from URLs containing colons.

Testing Strategy

Unit Tests:
- Test normalization of various edge cases (dots, slashes, colons, dashes)
- Test bi-directional mapping
- Test collision handling
Integration Tests:
- Test with real OpenAPI specs containing dots in operationIds
- Test with toolbox names from URLs containing colons
- Test end-to-end tool registration and execution
Property Tests:
- Round-trip property: normalize → denormalize should preserve original
- Validity property: normalized names always match [a-zA-Z][a-zA-Z0-9_]*

Edge Cases

1. Name Collisions

When two different operation IDs normalize to the same name:

-- Original names
"pet.find" and "pet_find" both normalize to "pet_find"

-- Solution: Append number suffix
"pet_find" and "pet_find_2"

2. Empty Operation IDs

When operationId is missing, we fall back to path/method:

-- From path "/pets/{id}" and method "GET"
-- Generated name: "get_pets__id_"

3. Very Long Names

Some providers have length limits. Strategy:

-- Truncate with hash suffix to maintain uniqueness
"openapi_very_long_toolbox_name_very_long_operation_id_..."
  → "openapi_very_long_toolbox_name_very_long_operat_a1b2c3"

4. Unicode Characters

Non-ASCII characters in operation IDs:

-- Replace with underscores
"获取宠物" → "____"
-- Then ensure letter start: "t____"

5. Toolbox Names from URLs

When toolbox names are derived from URLs, they may contain colons and dots:

-- Original URL-based name
"localhost:8001"
-- Normalized
"localhost_8001"

-- Original URL-based name
"api.example.com:8080"
-- Normalized
"api_example_com_8080"

src/System/Agents/Tools/OpenAPI/Converter.hs - Tool conversion logic
src/System/Agents/Tools/OpenAPI/Types.hs - Type definitions
src/System/Agents/Tools/OpenAPIToolbox.hs - Toolbox implementation
src/System/Agents/ToolRegistration.hs - LLM registration
src/System/Agents/LLMs/OpenAI.hs - LLM ToolName type

Future Considerations

Caching: Cache normalized names to avoid recomputation
Configuration: Allow users to provide custom normalization rules
Validation: Pre-validate specs during initialization
Reporting: Log warnings when names are normalized

CLI Reference

2026-10-04T15:53:54Z

Generated from documentation/cli-commands.md, the repository is the canonical source and may be ahead of this page.

CLI Reference

Complete reference for the Agents CLI commands and options.

Global Options

These options apply to all commands:

agents-exe [GLOBAL_OPTS] COMMAND [COMMAND_OPTS]

Option	Default	Description
`--api-keys FILE`	`~/.config/agents-exe/secret-keys`	Path to API keys JSON file
`--agent-file FILE`	(from config)	Agent configuration file(s)
`--agent SLUG`	-	Select agent by slug instead of file path
`--log-file FILE`	`agents-logfile`	Raw log output file
`--log-http URL`	-	HTTP endpoint for JSON logs
`--log-json-file FILE`	-	Local JSON log file

Commands

check

Validate agent configuration and display loaded tools.

agents-exe check [--tools MODE] [--show-config]

Options:

Option	Description
`--tools MODE`	Tool display mode: `none`, `list`, `agents-exe`, `openai` (default: `none`)
`--show-config`	Print each agent file's JSON before its check line; for a `.tramaj` [template](/docs-agent-templates.html), the JSON it evaluates to

Output:

my-agent: A helpful file assistant (12 tools)

Exit codes:

0 - All agents loaded successfully
1 - Configuration errors found

Example:

# Check default agents
agents-exe check

# Check specific agent
agents-exe check --agent-file ./custom-agent.json

# List tools in agents-exe format
agents-exe check --tools agents-exe

run

Execute a one-shot agent conversation.

agents-exe run [OPTIONS]

Options:

Option	Description
`--session-file FILE`	Resume from existing session
`--thinking TARGET`	Output thinking: `none`, `stdout`, `stderr` (default: `none`)
`-m, --media MEDIA`	Attach media file (can be specified multiple times)
`--prompt TEXT`	Initial prompt text
`--file FILE`	Read prompt from file
`--shell COMMAND`	Use shell command output as prompt
`--alias NAME`	Use predefined prompt alias
`--sep4 TEXT`	Short separator (4 chars)
`--sep40 TEXT`	Long separator (40 chars)
`--session-xs FILE`	Inject session at minimal verbosity
`--session-s FILE`	Inject session at low verbosity
`--session-m FILE`	Inject session at medium verbosity
`--session-l FILE`	Inject session at high verbosity
`--session-xl FILE`	Inject session at maximum verbosity

Asynchronous tool calls:

If the agent file enables asynchronous execution (see async-tool-calls.md), tool calls run concurrently and the model can be answered while they run. run still waits for them, since they cannot outlive the process.

It stops early only when a turn waits on deferred calls, which an external worker completes. It then prints a JSON report and stores the session:

{"status": "paused",
 "reason": "waiting for deferred tool calls",
 "session_id": "…",
 "deferred_calls": [{"tool": "approve_deploy", "tool_call_id": "call_9", "continuation_token": "…"}]}

Continue it with session complete and session resume.

Media Attachment Format:

The -m, --media option accepts file paths with optional MIME type:

# Auto-detect MIME type from extension
agents-exe run -m /path/to/image.png --prompt "Describe this image"

# Explicit MIME type (semicolon separator)
agents-exe run -m "image/jpeg;/path/to/photo.jpg" --prompt "Analyze this photo"

Supported Media Types:

Category	Extensions	MIME Types
Images	`.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.svg`	`image/png`, `image/jpeg`, `image/gif`, `image/webp`, `image/svg+xml`
Documents	`.pdf`, `.txt`, `.md`, `.json`, `.xml`	`application/pdf`, `text/plain`, `text/markdown`, `application/json`, `application/xml`
Audio	`.mp3`, `.wav`, `.ogg`, `.aac`, `.flac`	`audio/mp3`, `audio/wav`, `audio/ogg`, `audio/aac`, `audio/flac`
Video	`.mp4`, `.webm`, `.mov`, `.avi`	`video/mp4`, `video/webm`, `video/quicktime`, `video/avi`

Size Limit: 50MB per file

Examples:

# Simple prompt
agents-exe run --agent-file agent.json --prompt "Hello!"

# Read from file
agents-exe run --agent-file agent.json --file prompt.txt

# Shell command output as prompt
agents-exe run --agent-file agent.json --shell "git diff"

# Resume session
agents-exe run --agent-file agent.json --session-file session.json

# Use alias
agents-exe run --agent-file agent.json --alias code-review --file changes.patch

# Multi-part prompt with separators
agents-exe run \
  --prompt "Review this code:" \
  --sep4 "----" \
  --file code.py \
  --sep40 "========================================" \
  --prompt "What improvements can be made?"

# Inject previous session at medium verbosity
agents-exe run \
  --agent-file agent.json \
  --session-m previous-session.json \
  --prompt "Continue from where we left off"

# Attach single image
agents-exe run \
  --agent-file vision-agent.json \
  -m ./screenshot.png \
  --prompt "What do you see in this image?"

# Attach multiple images
agents-exe run \
  --agent-file vision-agent.json \
  -m ./image1.png \
  -m ./image2.jpg \
  -m "image/gif;./animation.gif" \
  --prompt "Compare these images"

# Attach document for analysis
agents-exe run \
  --agent-file document-agent.json \
  -m ./report.pdf \
  --prompt "Summarize the key findings"

# Combine media with text from file
agents-exe run \
  --agent-file multimodal-agent.json \
  --prompt "Analyze this diagram and code:" \
  -m ./diagram.png \
  --sep4 "----" \
  --file ./source-code.py

tui

Start the interactive Terminal UI.

agents-exe tui [--agent-file FILE...] [--keymap FILE] [--db PATH]
agents-exe tui --attach URL|PATH [--token TOKEN | --token-file FILE] [--keymap FILE]

The TUI is a client of an in-process SessionRunner it starts over the same Host config agents-exe serve uses, or, with --attach, of a running agents-exe serve/agents-server (see tui.md and agents-server.md).

Options:

Option	Default	Description
`--keymap FILE`, `-k FILE`	none	Path to a keymap configuration JSON file
`--db PATH`	next to the resolved sessions directory	SQLite database for the TUI's embedded session runner. Old `conv.<uuid>.json` history under the resolved sessions directories stays readable as a read-only fallback. Not with `--attach`.
`--attach URL\\|PATH`	none	Drive a running server instead of an embedded runner. `http://HOST:PORT` (or `https://`, optionally with a path prefix), `unix:///path/to.sock` for `serve --socket`, or a bare socket path (anything containing a `/` or ending in `.sock`). Nothing is loaded locally: agents, API keys and sessions are the server's.
`--token TOKEN`	none	Bearer token for `--attach`, when the server runs with `--auth-tokens`.
`--token-file FILE`	none	The same, read from a file (surrounding whitespace ignored); keeps the token out of the process list.

Features:

Real-time streaming responses
Multiple agent support (Tab to switch)
Tool call visualization
Session persistence
Turn navigation and forking
Message queue management
File attachments (Ctrl+F)
Clipboard paste support (Ctrl+V)

Keyboard Shortcuts:

Tab / Shift+Tab - Switch between agents
Enter - Send message
Ctrl+C - Quit
Up/Down - Scroll history
Ctrl+[ / Ctrl+] - Previous/Next tab
Ctrl+E - Pause/unpause conversation
Ctrl+D (when paused) - Clear queued messages
Ctrl+F - Attach file
Ctrl+Shift+F - Clear all attachments
Ctrl+V - Paste from clipboard

Examples:

# Single agent TUI
agents-exe tui --agent-file agent.json

# Multi-agent TUI
agents-exe tui \
  --agent-file coder.json \
  --agent-file reviewer.json \
  --agent-file tester.json

# Attach to a server started elsewhere with `agents-exe serve`
agents-exe tui --attach http://127.0.0.1:8080
agents-exe tui --attach unix:///run/agents/agents.sock
agents-exe tui --attach https://agents.example --token-file ~/.agents-token

spectate

Watch a running server: a read-only, live dashboard.

agents-exe spectate --attach URL|PATH [--token TOKEN | --token-file FILE]
                    [--panels SPEC] [--refresh SECONDS] [--layout-file FILE]

spectate is one more client of a running agents-exe serve/agents-server, like tui --attach, except that it only reads: it follows the server’s event feed and changes nothing. It is made for watching one long run, in three panels:

agents (tree; top left by default): the tree of sessions and of the sub-agent calls they make, indented by depth, each with its state and for how long.
tool calls (tools; bottom left): the calls running now, with elapsed time and the latest progress a background call reported, then the ones that finished recently.
text (text; right): what the model writes in one session, with the user queries it answers. By default the panel follows the session that wrote last; selecting a row of the tree pins it to that session.

Which panels show, where, how large, and how often the screen refreshes can be chosen on the command line and changed while running, as in top: see Layout below.

The events carry no timestamp, so every duration is measured by spectate from the moment it received the event; a run that started before spectate attached counts from the attach. Text arrives as the model writes it when the server runs with --stream-tokens, and turn by turn otherwise. A sub-agent call that runs inside its caller’s tool call (rather than as a session of its own) is in the tree, but has no event stream and so no text; and a session already running when spectate attaches shows its text from its next turn on.

Options:

Option	Default	Description
`--attach URL\\|PATH`	required	The server to watch. Same forms as `tui --attach`: `http://HOST:PORT`, `https://...`, `unix:///path/to.sock`, or a bare socket path.
`--token TOKEN`	none	Bearer token, when the server runs with `--auth-tokens`. `spectate` then sees that owner's sessions only.
`--token-file FILE`	none	The same, read from a file.
`--panels SPEC`	`tree:60+tools:40/45,text/55`	The panels shown, their places and sizes (see below).
`--refresh SECONDS`	`1`	Time between two refreshes of the screen, from `0.1` to `60`; decimals are accepted (`0.5`).
`--layout-file FILE`	`~/.config/agents-exe/spectate-layout`	The saved layout: read at start when the file exists, written by the `W` key.

Layout:

The screen is columns, left to right, each a stack of panels, top to bottom. --panels describes it in one string:

, separates columns, and + the panels stacked in one column;
a panel is tree (or agents), tools or text; one that is not named is hidden, and none can be named twice;
:N after a panel is its share of its column’s height, and /N after a column its share of the screen’s width. Shares go from 1 to 100, count relative to their neighbours, and are 50 when left out.

--panels tree,tools,text               # three equal columns
--panels text                          # only the text
--panels tree+tools+text               # one stack
--panels text/70,tree:30+tools:70/30   # text on the left, wide

Between two refreshes the events received wait, and the durations stand still: a longer interval makes a busy run calmer to read, a shorter one more live. Durations are still counted from when each event was received.

The layout is remembered only when asked, as top does with W: the W key writes it to the layout file, and the next spectate starts from that file. --panels and --refresh each go over what the file says, so a scripted layout (for a screenshot, say) is the same whatever was saved. A layout file that cannot be read is an error, not a layout silently dropped. The file holds what the flags take:

panels tree:60+tools:40/45,text/55
refresh 1

Keyboard Shortcuts:

Up / Down (or k / j) - Select a session in the tree; the text panel shows it
f - Follow the session that writes, again
PgUp / PgDn - Scroll the text
1 / 2 / 3 - Show or hide the agents, tool calls, text panel (the last one shown stays); a panel that comes back is a new last column
Tab / Shift+Tab - Choose the panel the next keys act on; its title is in brackets
< / > - Exchange it with the panel before or after
[ / ] - Narrow or widen its column
- / + - Shorten or heighten it in its column
s - Stack it under the column before, or give it a column of its own
0 - Back to the layout at start
d / D - Longer or shorter refresh interval (0.1, 0.25, 0.5, 1, 2, 5, 10, 30, 60 seconds)
W - Save the layout to the layout file
? or h - Show the keys, and the --panels/--refresh flags that reproduce the layout on screen
q, Esc or Ctrl+C - Quit (the runs go on)

No key sends anything to the server or to the agents.

Examples:

# In one terminal: a server, and a TUI or any other client driving it
agents-exe serve --socket /run/agents/agents.sock --stream-tokens
agents-exe tui --attach /run/agents/agents.sock

# In another: watch
agents-exe spectate --attach /run/agents/agents.sock

# The same, as three columns refreshed twice a second
agents-exe spectate --attach /run/agents/agents.sock --panels tree,tools,text --refresh 0.5

A TUI started without --attach runs its agents in its own process and exposes no endpoint, so there is nothing for spectate to attach to: run agents-exe serve and attach both.

mcp-server

Start an MCP (Model Context Protocol) server.

agents-exe mcp-server [--agent-file FILE...]

Description: Exposes loaded agents as MCP tools for integration with MCP clients like Claude Desktop.

Examples:

# Single agent MCP server
agents-exe mcp-server --agent-file agent.json

# Multi-agent MCP server
agents-exe mcp-server \
  --agent-file research-agent.json \
  --agent-file writing-agent.json

echo-prompt

Process and echo the prompt without calling the LLM.

agents-exe echo-prompt [OPTIONS]

Options: Same as run command.

Use case: Verify prompt construction before sending to LLM.

Example:

agents-exe echo-prompt \
  --prompt "Context:" \
  --file context.txt \
  --prompt "Question:" \
  --file question.txt

session-print

Display a session file in markdown format.

agents-exe session-print [OPTIONS] SESSIONFILE

Options:

Option	Description
`--show-tool-call-results MODE`	Display mode: `hidden`, `shown`, `elided` (default: `hidden`)
`--show-tool-call-arguments MODE`	Display mode: `hidden`, `shown`, `elided` (default: `hidden`)
`--n-turns N`	Limit to last N turns
`--repeat-system-prompt`	Show system prompt each turn
`--repeat-tools`	Show available tools each turn
`--antichronological`	Newest first (default: oldest first)
`--no-funny-stamp`	Skip the ASCII art logo

Examples:

# Print full session
agents-exe session-print session.json

# Print with tool results visible
agents-exe session-print --show-tool-call-results shown session.json

# Elide long outputs (show first/last 10 lines)
agents-exe session-print --show-tool-call-results elided session.json

# Last 5 turns only
agents-exe session-print --n-turns 5 session.json

# Reverse order
agents-exe session-print --antichronological session.json

session-edit

Edit a session file (reads JSON from STDIN, writes JSON to STDOUT).

agents-exe session-edit [OPTIONS] < input.json > output.json

Options:

Option	Description
`--take`	Take first N turns (use with `--count`)
`--take-tail`	Take last N turns (use with `--count`)
`--drop`	Drop first N turns (use with `--count`)
`--drop-tail`	Drop last N turns (use with `--count`)
`--count N`, `-n N`	Number of turns for take/drop operations
`--censor-tool-calls`	Remove all tool calls
`--censor-thinking`	Remove all thinking content

Examples:

# Keep only first 10 turns
agents-exe session-edit --take --count 10 < session.json > trimmed.json

# Keep only last 5 turns
agents-exe session-edit --take-tail --count 5 < session.json > recent.json

# Remove first 2 turns
agents-exe session-edit --drop --count 2 < session.json > dropped.json

# Remove all tool calls
agents-exe session-edit --censor-tool-calls < session.json > no-tools.json

# Remove thinking content
agents-exe session-edit --censor-thinking < session.json > no-thinking.json

session-index

Manage the SQLite FTS5 search index for session files.

agents-exe session-index [OPTIONS]

Options:

Option	Description
`--build`	Build the search index from scratch
`--update`	Incrementally update the search index
`--status`	Show index status (default)
`--clean`	Remove the search index
`--db-path PATH`	Path to search index database (default: `.agents-search.db`)
`--include-tool-outputs`	Include tool outputs in the index

Description: The session-index command manages a SQLite FTS5-based search index for fast fuzzy text search across session files. The index uses trigram tokenization to enable fuzzy matching (e.g., “error” matches “errors”, “erroring”).

Index Schema:

session_index - Session metadata cache (path, mtime, agent, turn count)
search_content - FTS5 virtual table with trigram tokenizer
tool_index - Tool call index for filtering by tool name

Examples:

# Build the search index
agents-exe session-index --build

# Check index status
agents-exe session-index --status

# Update index incrementally
agents-exe session-index --update

# Remove the index
agents-exe session-index --clean

# Build with tool outputs included
agents-exe session-index --build --include-tool-outputs

# Custom database location
agents-exe session-index --build --db-path ~/.config/agents-exe/search.db

session-search

Search session files with fuzzy text matching and filtering.

agents-exe session-search [OPTIONS] "QUERY"

Arguments:

Argument	Description
`QUERY`	Search query text (supports fuzzy matching)

Options:

Option	Description
`--db-path PATH`	Path to search index database
`--after DATE`	Filter sessions after date (YYYY-MM-DD)
`--before DATE`	Filter sessions before date (YYYY-MM-DD)
`--tool TOOLNAME`	Filter by tool name (can specify multiple)
`--agent SLUG`	Filter by agent slug
`--include-tool-outputs`	Include tool outputs in search
`--json`	Output results as JSON
`--preview N`	Show N lines of context around matches
`--limit N`	Limit to N results
`--auto`	Auto-update index if stale before searching

Description: Performs fast fuzzy text search across session files using an SQLite FTS5 index. The search supports trigram-based fuzzy matching, allowing queries like “error” to match “errors”, “erroring”, “terror”, etc.

Search Features:

Fuzzy matching: Trigram tokenizer handles typos and variations
Metadata filtering: Filter by date, tool usage, or agent
Incremental indexing: Only re-indexes changed sessions
JSON output: Machine-readable format for scripting

Examples:

# Basic fuzzy search
agents-exe session-search "database error"

# Search with auto-update
agents-exe session-search "migration" --auto

# Include tool outputs in search
agents-exe session-search "config.yaml" --include-tool-outputs

# Filter by date and tool
agents-exe session-search "auth" --after 2024-01-01 --tool write-file

# Filter by agent
agents-exe session-search "refactor" --agent my-coder

# JSON output for scripting
agents-exe session-search "TODO" --json

# Show context lines
agents-exe session-search "deploy" --preview 5

# Combined filters
agents-exe session-search "fix" --after 2024-01-01 --before 2024-12-31 --tool bash_write-file --json

JSON Output Format:

{
  "resultItems": [
    {
      "resultMetadata": {
        "resultSessionId": "550e8400-e29b-41d4-a716-446655440000",
        "resultFilePath": "/path/to/session.json",
        "resultAgentSlug": "my-agent",
        "resultTurnCount": 15,
        "resultRank": 0.5
      },
      "resultPreview": "...context around match...",
      "resultMatchedTerms": ["error", "database"]
    }
  ],
  "resultTotalMatches": 42,
  "resultQueryTimeMs": 15.3,
  "resultIndexWasUpdated": false
}

list-tool-calls

Extract and list all tool calls from a session file.

agents-exe list-tool-calls [OPTIONS] SESSIONFILE

Options:

Option	Description
`-f, --format FORMAT`	Output format: `human`, `json`, `brief` (default: `human`)

Description: Parses a session file and extracts all tool calls made during the conversation. Useful for debugging, auditing, and replaying specific tool calls.

Output formats:

human - Detailed human-readable format with indices, turn numbers, and arguments
json - JSON array for machine processing
brief - Compact tabular format (index, name, arguments preview)

Examples:

# List tool calls in human-readable format
agents-exe list-tool-calls session.json

# Output as JSON
agents-exe list-tool-calls session.json --format json

# Brief format for quick overview
agents-exe list-tool-calls session.json --format brief

Example output (human format):

Found 3 tool call(s):

[0] bash_read-file
  Turn: 1
  Arguments: 
    {"filepath":"./src/Main.hs"}

[1] bash_grep-files
  Turn: 2
  Arguments: 
    {"pattern":"TODO","filepath":"./src"}

[2] bash_write-file
  Turn: 2
  Arguments: 
    {"filepath":"./src/Main.hs","content":"..."}

To replay a tool call:
  agents-exe replay-tool-call --session <file> --tool-call <index> --tool <tool-path>

replay-tool-call

Replay a specific tool call from a session file, with validation.

agents-exe replay-tool-call --session FILE --tool-call INDEX --tool TOOLPATH [OPTIONS]

Options:

Option	Description
`-s, --session FILE`	Path to the session file containing the tool call
`-i, --tool-call INDEX`	Index of the tool call to replay (0-based)
`-t, --tool TOOLPATH`	Path to the tool script to execute
`--validate-only`	Only validate arguments, don't execute
`--raw`	Show raw output instead of formatted

Description: Extracts a tool call from a session file, validates its arguments against the tool’s schema, and optionally executes the tool with the same arguments. This is useful for:

Debugging failed tool calls
Reproducing tool execution for testing
Validating historical tool calls against updated schemas

Exit codes:

0 - Validation passed (and tool executed if not –validate-only)
1 - Validation failed or tool execution failed

Examples:

# First, list tool calls to find the index
agents-exe list-tool-calls session.json

# Validate and execute a tool call
agents-exe replay-tool-call \
  --session session.json \
  --tool-call 0 \
  --tool ./tools/read-file.sh

# Only validate without executing
agents-exe replay-tool-call \
  --session session.json \
  --tool-call 1 \
  --tool ./tools/grep-files.sh \
  --validate-only

# Execute and show raw output
agents-exe replay-tool-call \
  --session session.json \
  --tool-call 2 \
  --tool ./tools/write-file.sh \
  --raw

Example output:

✓ Tool call validation passed
Tool: read-file
Arguments: {"filepath":"./src/Main.hs"}

Executing tool...

=== Tool Output ===
module Main where

main :: IO ()
main = putStrLn "Hello, World!"
===================

tool-call

Call a tool from the first loaded agent with JSON payload from stdin.

echo '{}' | agents-exe tool-call TOOLNAME [OPTIONS]

Arguments:

Argument	Description
`TOOLNAME`	Name of the tool to call

Options:

Option	Description
`-l, --log-file FILE`	Optional log file for tracing tool execution

Description: Executes a single tool directly without involving the LLM. Useful for:

Testing tool implementations
Scripting tool usage
Debugging tool behavior

Examples:

# Call a tool with arguments
echo '{"filepath": "./README.md"}' | \
    agents-exe tool-call read-file --agent-file ./agent.json

# Call with logging
echo '{"command": "ls -la"}' | \
    agents-exe tool-call bash --log-file ./tool-debug.log

serve

Run agents over HTTP, like agents-server, loading agents-exe.cfg.json like the TUI does.

agents-exe serve [--agent-file FILE...] [--agent SLUG] [OPTIONS]

Description:

agents-exe serve is agents-server’s code (sessions in SQLite or Postgres, the HTTP API and SSE stream, MCP over HTTP, the chat page) run behind agents-exe’s own config loading, so it resolves agent files the same way every other agents-exe command does: --agent-file (repeatable), else agents-exe.cfg.json’s agentsFiles plus every .json and .tramaj file under agentsDirectories, else ~/.config/agents-exe/default; --agent SLUG narrows to one agent by slug, failing with the list of available slugs if it does not match. It shares agents-exe’s global --api-keys, --set/--set-json/--pin/--pin-json and --params-file, so those are not repeated below.

Options:

Option	Default	Description
`--db FILE\\|URL`	next to the resolved sessions directory	SQLite file, or `postgresql://` URL, for sessions and continuations
`--bind HOST`	`127.0.0.1`	Address to listen on
`--port PORT`	`8080`	Port to listen on
`--live-session-ttl SECONDS`	`900`	Idle time before a session's in-memory state is dropped
`--shutdown-grace SECONDS`	`10`	Time open requests get to finish on shutdown
`--auth-tokens FILE`	(none)	Bearer tokens and their owners; without, no authentication
`--stream-tokens`	off	Stream LLM answers as `text.delta` events
`--admin-owners OWNER,…`	(none)	Owners allowed to store and delete agents over the API (needs `--auth-tokens`)
`--no-ui`	off	Do not serve the chat page at `/`
`--cors-origin ORIGIN`	(none)	Allow this origin to call the server cross-origin; repeatable, or `*` for any (needs no `--auth-tokens`)
`--socket PATH`	(none)	Also listen on this Unix domain socket, in addition to `--bind`/`--port`; created `0600`, a stale file removed at start, closed and unlinked on shutdown. The socket is the local trust boundary: no `Origin`, no bearer token beyond `--auth-tokens`.
`--owner-api-keys OWNER=FILE`	(none)	This owner's sessions call the LLM with the keys in `FILE` instead of the shared ones; repeatable (needs `--auth-tokens`)
`--isolate-tools docker:IMAGE\\|process:PATH`	(none)	Run bash and MCP tool calls outside the server, in a worker you provide

See agents-server.md for everything the running server does (the HTTP API, sessions, deferred calls, CORS, authentication, running it as a service).

Examples:

# Explicit agent files, like agents-server
agents-exe serve --agent-file ./weather.json --api-keys ./keys.json --port 8080

# From a directory with an agents-exe.cfg.json
agents-exe serve --port 8080

# One agent from a multi-agent config, over a local socket only auth-tokens gate
agents-exe serve --agent architect --socket /run/agents-exe/agents.sock --auth-tokens ./tokens.json

check-tool-call

Validate a tool call payload against a tool schema (reads JSON from stdin).

cat payload.json | agents-exe check-tool-call --tool TOOLPATH

Options:

Option	Description
`-t, --tool TOOLPATH`	Path to the tool script to validate against

Description: Validates that a JSON payload from stdin matches the tool’s declared schema. Returns exit code 0 if valid, 1 if invalid with detailed error messages.

Example:

# Create a test payload
echo '{"filepath": "/path/to/file"}' | agents-exe check-tool-call --tool ./tools/read-file.sh

# Validate from file
cat payload.json | agents-exe check-tool-call --tool ./tools/my-tool.sh

Example output (valid):

✓ Tool call payload is valid

Example output (invalid):

Tool call validation failed for './tools/my-tool.sh (my-tool)' with 2 errors:

1. filepath: Required property missing

2. content: Required property missing

Please correct these issues and try again.

init

Initialize a new agent configuration.

agents-exe init [--agent-file FILE]

Creates:

agent.json - Agent configuration
tools/ - Tool directory
secret-keys - API keys file (if doesn’t exist)

Example:

agents-exe init --agent-file ./my-agent.json

new

Create scaffolding for new agents or tools.

agents-exe new (agent|tool) [OPTIONS]

new agent

Create a new agent configuration file from a template.

agents-exe new agent SLUG FILE [MODEL] [OPTIONS]

Arguments:

Argument	Description
`SLUG`	Unique identifier for the agent
`FILE`	Output file path
`MODEL`	Model name (e.g., gpt-4o, mistral-large). The provider preset is inferred from the model catalog.

Options:

Option	Description
`--add-to-config`	Add the agent to the `agentsFiles` of `agents-exe.cfg.json` when it is not listed there, without asking
`--no-add-to-config`	Never modify `agents-exe.cfg.json`
`-f, --force`	Overwrite existing file (given before the subcommand: `agents-exe new --force agent ...`)

Provider presets (URL, default model, API key ID) are selected automatically based on the model name. Use agents-exe new models list to see the built-in model patterns and agents-exe new models init to customize them locally.

Examples:

# Create agent with the default OpenAI preset
agents-exe new agent my-assistant ./my-assistant.json

# Create agent with custom model (preset inferred from the catalog)
agents-exe new agent coder ./coder.json gpt-4o

# Create agent in a subdirectory and list it in agents-exe.cfg.json
agents-exe new agent my-assistant ./agents/assistant.json --add-to-config

# Overwrite existing
agents-exe new --force agent my-assistant ./my-assistant.json

What a new agent can do:

A new agent works on the directory it is started from:

Toolbox	Gives the agent
`developer` (DeveloperToolbox)	Reading and editing files (`read-file-range`, `write-file-range`, `patch-file`), and the agent/tool scaffolding helpers
`system` (SystemToolbox)	The working directory and directory listings (`working-directory`, `list-directory`)
`memory` (SqliteToolbox)	A read-write SQLite database, `./{slug}-memory.sqlite`, created on first use

File access is limited by one file sandbox, workspace, declared in the agent’s fileSandboxes and shared by the developer and system toolboxes. It allows reading and writing ./ and everything below it, where ./ is the directory the agent runs in (not the directory of the agent file). Edit its fsbPredicate to narrow or widen what the agent may touch; see Tools for the predicates.

Generated agent file:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "my-assistant",
    "apiKeyId": "main-key",
    "flavor": "OpenAIv1",
    "modelUrl": "https://api.openai.com/v1",
    "modelName": "gpt-4-turbo-preview",
    "announce": "a helpful assistant powered by gpt-4-turbo-preview",
    "systemPrompt": [
      "You are my-assistant, a helpful AI assistant.",
      "You provide clear, accurate, and concise responses.",
      "When using tools, you explain your actions to the user."
    ],
    "toolDirectory": "tools",
    "mcpServers": [],
    "fileSandboxes": {
      "workspace": {
        "fsbPredicate": {"tag": "DirectoryRecursive", "contents": "./"},
        "fsbMaxFileSize": 52428800,
        "fsbName": null
      }
    },
    "builtinToolboxes": [
      {
        "tag": "DeveloperToolbox",
        "contents": {
          "Name": "developer",
          "Description": "Tools for developing agents and tools",
          "Capabilities": ["show-spec", "validate-agent", "create-agent", "create-tool", "read-file-range", "write-file-range", "patch-file"],
          "FileSandbox": {"ref": "workspace"}
        }
      },
      {
        "tag": "SystemToolbox",
        "contents": {
          "Name": "system",
          "Description": "Working directory and directory listings",
          "Capabilities": ["working-directory", "list-directory"],
          "FileSandbox": {"ref": "workspace"}
        }
      },
      {
        "tag": "SqliteToolbox",
        "contents": {
          "Name": "memory",
          "Description": "Notes and facts to remember across conversations",
          "Versioning": {"tag": "SqliteReadWrite", "path": "./my-assistant-memory.sqlite"}
        }
      }
    ]
  }
}

Listing the agent in agents-exe.cfg.json:

After writing the file, new agent looks for an agents-exe.cfg.json (in the current directory, then upward) and reports whether the new agent is loaded by it, that is, named in agentsFiles or sitting directly in one of the agentsDirectories:

listed: nothing else to do;
not listed, on an interactive terminal: it asks whether to add the file to agentsFiles (default: no);
not listed, not interactive: it prints the entry to add. Pass --add-to-config to add it without being asked, --no-add-to-config to never be asked.

Adding the entry rewrites the config file as formatted JSON; every other field is kept. Without any agents-exe.cfg.json, pass the file with --agent-file or create a config with agents-exe config local init.

new tool

Create a new tool script from a template.

agents-exe new tool SLUG LANGUAGE FILE [OPTIONS]

Arguments:

Argument	Description
`SLUG`	Unique identifier for the tool
`LANGUAGE`	Programming language: `bash`, `python`, `haskell`, `node`
`FILE`	Output file path (without extension)

Options:

Option	Description
`-f, --force`	Overwrite existing file

Examples:

# Create bash tool (default)
agents-exe new tool my-tool bash ./tools/my-tool

# Create Python tool
agents-exe new tool my-tool python ./scripts/my-tool

# Create Haskell tool
agents-exe new tool my-tool haskell ./tools/my-tool

# Create Node.js tool
agents-exe new tool my-tool node ./scripts/my-tool

Generated bash tool:

#!/bin/bash
# my-tool - A bash tool for agents-exe

set -euo pipefail

# Agents-exe tool protocol: describe|run
# Environment variables available during 'run':
#   AGENT_SESSION_ID      - UUID of the current session
#   AGENT_CONVERSATION_ID - UUID of the conversation
#   AGENT_TURN_ID         - UUID of the current turn
#   AGENT_AGENT_ID        - UUID of the executing agent (if available)

case "${1:-}" in
  describe)
    cat <<'DESCRIBE_EOF'
{
  "slug": "my-tool",
  "description": "Tool my-tool - describe what this tool does",
  "args": [],
  "empty-result": {
    "tag": "AddMessage",
    "contents": "--no output--"
  }
}
DESCRIBE_EOF
    ;;
  run)
    # TODO: Implement tool logic
    # Access arguments via environment or command line
    echo "Tool my-tool executed"
    ;;
  *)
    echo "Usage: my-tool <describe|run>" >&2
    exit 1
    ;;
esac

Next steps after creating a tool:

Edit the args array in the describe function
Implement the run function logic
Test with: agents-exe describe-tool ./tools/my-tool

config

Configure agents-exe itself: the project config file, the TUI keymap and the API keys file, without writing their JSON by hand.

agents-exe config [-f|--force] (local|keymap|api-key) ...

Options:

Option	Description
`-f, --force`	Overwrite an existing file or API key entry. It belongs to `config`, so it goes last: `agents-exe config local init --force`

config local init

Create an agents-exe.cfg.json in the current directory.

agents-exe config local init

The file is minimal: agentsConfigDir pointing at ~/.config/agents-exe and an empty agentsFiles. Add agents to agentsFiles or agentsDirectories (see Configuration File); agents-exe new agent offers to do it for the agents it creates. Only the current directory is checked for an existing file, not its parents; an existing file is kept unless --force is given.

config keymap init

Write a keymap file holding the default TUI key bindings and input settings, as a starting point for customization.

agents-exe config keymap init FILE

Argument	Description
`FILE`	Path where the keymap file is created (parent directories are created as needed)

Point the keymap field of agents-exe.cfg.json at the file to use it. An existing file is kept unless --force is given.

config api-key list

List the names of the configured API keys (never their values).

agents-exe config api-key list

config api-key create

Add an entry to the API keys file with a placeholder value, to be replaced by the real key in an editor.

agents-exe config api-key create NAME

Argument	Description
`NAME`	Name of the key, as referenced by an agent's `apiKeyId`

An existing entry of that name is kept unless --force is given, in which case it is reset to the placeholder.

Both api-key subcommands work on the secret-keys file of the config directory (~/.config/agents-exe/secret-keys by default), and create it empty if it does not exist. The global --api-keys option does not redirect them.

Examples:

# Start a project: config file, then an agent listed in it
agents-exe config local init
agents-exe new agent helper ./helper.json --add-to-config

# Declare the key the agent's preset refers to, then fill in its value
agents-exe config api-key create main-key
agents-exe config api-key list

# Customize the TUI key bindings
agents-exe config keymap init ./keymap.json

spec

Display embedded specification documentation.

agents-exe spec TOPIC

Topics:

Topic	Description
`bash-tools`	Binary tool protocol specification

Examples:

# Display bash-tools specification
agents-exe spec bash-tools

Use case: Learn the protocol for writing bash tools without leaving the CLI.

describe-tool

Display information about a tool script.

agents-exe describe-tool TOOL_PATH [OPTIONS]

Options:

Option	Description
`-f, --format FORMAT`	Output format: `json` or `pretty` (default: `json`)
`--check-only`	Only check validity, don't output full description

Description: Loads and displays the tool’s description (the output of tool describe).

Example:

agents-exe describe-tool ./tools/my-tool.sh

# Pretty format
agents-exe describe-tool ./tools/my-tool.sh --format pretty

# Only check validity
agents-exe describe-tool ./tools/my-tool.sh --check-only

Output:

{
  "slug": "my-tool",
  "description": "What this tool does",
  "args": [...],
  "empty-result": {...}
}

paths

Show important configuration paths.

agents-exe paths [OPTIONS]

Options:

Option	Description
`--json`	Output in JSON format

Description: Displays configuration paths including config directory, agent files, API keys file, and session storage configuration.

Session Storage: The session storage configuration supports multiple read locations:

Write location: Where new sessions are saved
Read locations: Directories searched for existing sessions (includes write location by default)

Examples:

# Human-readable output
agents-exe paths

# JSON output
agents-exe paths --json

Example output:

Configuration:
  File: /home/user/project/agents-exe.cfg.json
  Directory: /home/user/project
  Default: /home/user/.config/agents-exe

Agent Files:
  - ./main-agent.json
  - ./secondary-agent.json

API Keys:
  /home/user/.config/agents-exe/secret-keys

Session Storage:
  Write location: ./sessions/
  Read locations:
    1. ./sessions/
    2. ./.agents-sessions/
    3. ~/.config/agents-exe/sessions/

export

Export agent/tool configurations to archive or git.

agents-exe export [OPTIONS]

Source Options:

Option	Description
`--all`	Export all loaded agents
`--agent-slug SLUG`	Export specific agent by slug
`--tools-only`	Export only tools, not agent config
`--tool TOOLNAME`	Export specific tool

Destination Options:

Option	Description
`-o, --output FILE`	Output file path
`--git-url URL`	Git remote URL
`--git-branch BRANCH`	Git branch
`--git-message MESSAGE`	Commit message
`--git-push`	Push after commit
`--git-tag TAG`	Create git tag

Format Options:

Option	Description
`--format FORMAT`	Archive format: tar, tar.gz, zip
`--namespace NAMESPACE`	Namespace for export (e.g., "team.project")
`--no-tools`	Exclude tools from export
`--no-mcp`	Exclude MCP servers from export

Examples:

# Export to tar.gz
agents-exe export --output ./my-agent.tar.gz

# Export to git
agents-exe export \
  --git-url https://github.com/user/agents-repo \
  --git-branch main \
  --git-message "Update agent" \
  --git-push

# Export tools only
agents-exe export --tools-only --output ./tools.tar.gz

# Export specific agent
agents-exe export --agent-slug my-agent --output ./agent.tar.gz

# With namespace
agents-exe export --namespace team-a.project-1 --output ./export.tar.gz

import

Import agent/tool configurations from archive or git.

agents-exe import [OPTIONS]

Source Options:

Option	Description
`-f, --from-file FILE`	Import from archive file
`--git-url URL`	Import from git URL
`--git-ref REF`	Git ref (branch, tag, commit)
`--namespace NAMESPACE`	Namespace to import from

Destination Options:

Option	Description
`--to-current`	Import to current directory
`--to PATH`	Import to specific path
`--to-config-dir`	Import to config directory
`--install-to-agent FILE`	Install tools to agent's tool directory
`--install-to-tooldir DIR`	Install tools to specific directory

Mode Options:

Option	Description
`--overwrite`	Overwrite existing files
`--merge`	Merge with existing files
(default)	Fail on conflict

List Options:

Option	Description
`--list-namespaces`	List available namespaces (git only)
`--list-tools`	List available tools (git only)

Examples:

# Import from archive
agents-exe import --from-file ./agent.tar.gz

# Import from git
agents-exe import \
  --git-url https://github.com/user/agents-repo \
  --git-ref main

# Import tools to agent
agents-exe import \
  --from-file ./tools.tar.gz \
  --install-to-agent ./my-agent.json

# Import with overwrite
agents-exe import --from-file ./agent.tar.gz --overwrite

# List namespaces
agents-exe import --git-url https://github.com/user/repo --list-namespaces

# Import specific namespace
agents-exe import \
  --git-url https://github.com/user/repo \
  --namespace team-a.tools \
  --to ./tools/

cowsay

Display a fun message with the agents-exe mascot.

agents-exe cowsay [MESSAGE] [OPTIONS]

Options:

Option	Description
`-W, --width WIDTH`	Maximum width of speech bubble (default: 40)

Description: If no message is provided, reads from stdin.

Example:

agents-exe cowsay "Hello, agents!"

# With custom width
agents-exe cowsay "A much longer message that needs more space" --width 60

# Pipe from another command
echo "System ready" | agents-exe cowsay

describe

Output self-describing schema for the agent.

agents-exe describe [OPTIONS]

Options:

Option	Description
`--self-describe-slug NAME`	Override the slug field
`--self-describe-description DESC`	Override the description

Output: JSON schema for tool calling.

Use case: Integration with external systems that need to understand the agent’s interface.

self-describe

Output information about the agents-exe binary itself.

agents-exe self-describe

Output: Version, build info, and available commands.

Configuration File

Project-level configuration in agents-exe.cfg.json:

{
  "agentsConfigDir": "/custom/config/path",
  "agentsDirectories": [
    "./agents",
    "./more-agents"
  ],
  "agentsFiles": [
    "./main-agent.json"
  ],
  "promptAliases": {
    "translate": {
      "systemPrompt": ["You are a translator..."],
      "userPromptPrefix": "Translate to English:"
    }
  },
  "selfDescribeSlug": "my-agent",
  "selfDescribeDescription": "A custom agent",
  "agentsLogs": {
    "logJsonHttpEndpoint": "http://localhost:8080/log",
    "logJsonPath": "./logs/agents.json",
    "logRawPath": "./logs/agents.log"
  },
  "sessions": {
    "writeLocation": "./sessions/",
    "readLocations": [
      "./sessions/",
      "./.agents-sessions/",
      "~/.config/agents-exe/sessions/"
    ]
  }
}

Search order:

Current directory
Parent directories (upward search)

Sessions Configuration

The sessions section configures multi-location session storage:

Field	Type	Description
`writeLocation`	`FilePath`	Directory where new sessions are written
`readLocations`	`[FilePath]`	Directories to search for existing sessions

Key behaviors:

Deduplication: First location in readLocations has highest priority for duplicate session IDs
Auto-prepending: If writeLocation is not in readLocations, it’s automatically prepended
Tilde expansion: Paths like ~/.config/agents-exe/sessions/ are resolved to the user’s home directory
Backwards compatibility: If sessions is not present, falls back to agentsLogs.logSessionsJsonPrefix (deprecated)

Environment Variables

Variable	Description
`AGENTS_API_KEY`	Default API key (overrides file)
`AGENTS_CONFIG_DIR`	Config directory path
`AGENTS_LOG_LEVEL`	Logging verbosity
`AGENT_MD_VIEWER`	External markdown viewer for TUI session viewing

Exit Codes

Code	Meaning
`0`	Success
`1`	General error
`2`	Invalid arguments
`3`	Configuration error
`4`	Agent loading error
`5`	Tool execution error
`10`	Export error
`11`	Import error

Examples

Complete Workflow

# 1. Initialize new agent
agents-exe new agent my-assistant

# 2. Create a custom tool
agents-exe new tool file-reader python ./tools/file-reader

# 3. Edit tool and agent configurations
# (edit files as needed)

# 4. Check configuration
agents-exe check --agent-file ./my-assistant.json

# 5. Test with one-shot
agents-exe run --agent-file ./my-assistant.json --prompt "Test"

# 6. Start interactive session
agents-exe tui --agent-file ./my-assistant.json

# 7. Export for sharing
agents-exe export --agent-file ./my-assistant.json --output ./my-assistant.tar.gz

# 8. Import elsewhere
agents-exe import --from-file ./my-assistant.tar.gz --to ./new-location/

Multi-Agent Setup

# Start multi-agent TUI
agents-exe tui \
  --agent-file ./router.json \
  --agent-file ./coder.json \
  --agent-file ./reviewer.json

# Start multi-agent MCP server
agents-exe mcp-server \
  --agent-file ./router.json \
  --agent-file ./coder.json

Session Management Workflow

# Run an agent session
agents-exe run \
    --agent-file ./my-agent.json \
    --session-file ./session.json \
    --prompt "Let's work on a project"

# Later, inspect the session
agents-exe session-print ./session.json

# List all tool calls from the session
agents-exe list-tool-calls ~/.config/agents-exe/sessions/session-*.json

# Replay a specific tool call for debugging
agents-exe replay-tool-call \
  --session ./session.json \
  --tool-call 0 \
  --tool ./tools/read-file.sh \
  --validate-only

# Trim to last 20 turns
agents-exe session-edit --take-tail --count 20 < ./session.json > ./trimmed.json
mv ./trimmed.json ./session.json

# Continue the session
agents-exe run \
    --agent-file ./my-agent.json \
    --session-file ./session.json \
    --prompt "Where were we?"

Session Search Workflow

# Build the search index
agents-exe session-index --build

# Search for error-related sessions
agents-exe session-search "error" --json

# Find sessions using specific tool after a date
agents-exe session-search "config" --after 2024-01-01 --tool write-file

# Search with context preview
agents-exe session-search "TODO" --preview 3

# Auto-update index before searching
agents-exe session-search "refactor" --auto --limit 10

# Analyze a screenshot with a vision-capable agent
agents-exe run \
    --agent-file ./vision-agent.json \
    -m ./screenshot.png \
    --prompt "What UI issues do you see in this screenshot?"

# Analyze multiple images
agents-exe run \
    --agent-file ./design-agent.json \
    -m ./mockup-v1.png \
    -m ./mockup-v2.png \
    --prompt "Compare these two design mockups and recommend which to use"

# Analyze a PDF document
agents-exe run \
    --agent-file ./document-agent.json \
    -m ./report.pdf \
    --prompt "Summarize the key findings and recommendations"

# Combine image with code for debugging
agents-exe run \
    --agent-file ./debug-agent.json \
    --prompt "This error appears in the UI:" \
    -m ./error-screenshot.png \
    --sep4 "----" \
    --prompt "Here's the relevant code:" \
    --file ./src/Main.hs

Automation

# Process all files in directory
for file in *.txt; do
    agents-exe run \
        --agent-file ./processor.json \
        --prompt "Process this file:" \
        --file "$file" \
        --prompt "End of file."
done

# Git pre-commit hook
agents-exe run \
    --agent-file ./reviewer.json \
    --shell "git diff --cached" \
    --prompt "Review these changes."

# Daily summary from logs
agents-exe run \
    --agent-file ./summarizer.json \
    --shell "cat /var/log/app.log | tail -100" \
    --prompt "Summarize any errors or warnings"

# Automated image analysis with timestamp
agents-exe run \
    --agent-file ./monitor-agent.json \
    -m ./screenshots/$(date +%Y%m%d-%H%M%S).png \
    --prompt "Check for any alerts or errors in this status dashboard"

Developer Workflow

# Learn the tool protocol
agents-exe spec bash-tools

# Create and validate a tool
agents-exe new tool my-validator bash ./tools/my-validator
# (edit the tool)
agents-exe describe-tool ./tools/my-validator --format pretty

# Validate a payload
echo '{"input": "test"}' | agents-exe check-tool-call --tool ./tools/my-validator

# Test the tool directly
echo '{"input": "test"}' | agents-exe tool-call my-validator

# Create agent with dev tools
agents-exe new agent dev-assistant
# (add DeveloperToolbox to builtinToolboxes)
# Now the agent can use developer file-editing tools and show specs!

# Create a tool that outputs media
agents-exe new tool chart-generator bash ./tools/chart-gen
# Add to describe output:
#   "output-media-type": "image/png"
# Now the tool's output is treated as a PNG image

Debugging Tool Calls

# Run an agent session
agents-exe run --agent-file ./my-agent.json --prompt "Read the README"

# Find the session file
agents-exe paths

# List all tool calls
agents-exe list-tool-calls ~/.config/agents-exe/sessions/session-*.json

# Replay with validation only
agents-exe replay-tool-call \
  --session ~/.config/agents-exe/sessions/session-xxx.json \
  --tool-call 0 \
  --tool ./tools/read-file.sh \
  --validate-only

# Actually replay the tool call
agents-exe replay-tool-call \
  --session ~/.config/agents-exe/sessions/session-xxx.json \
  --tool-call 0 \
  --tool ./tools/read-file.sh

tools.md - Tool system details
tui.md - Terminal UI documentation
sessions.md - Session management
architecture.md - System architecture

MCP Protocol

2026-10-04T15:53:54Z

Generated from documentation/mcp.md, the repository is the canonical source and may be ahead of this page.

MCP Protocol

The Model Context Protocol (MCP) integration allows agents to connect to external servers that provide dynamic tool listings and execution capabilities.

Overview

MCP is a protocol for model context exchange that enables:

Dynamic tool discovery: Servers advertise available tools at runtime
Structured tool calls: JSON-RPC based communication
Process isolation: MCP servers run in separate processes
Standardized interface: Common protocol across different tool providers

┌─────────────┐      JSON-RPC       ┌─────────────┐
│   Agent     │<------------------->│ MCP Server  │
│  Runtime    │   (stdio/stdin)     │  Process    │
└─────────────┘                     └─────────────┘
       │                                   │
       │ Tool descriptions                 │ Tool execution
       │                                   │
       ▼                                   ▼
┌─────────────┐                     ┌─────────────┐
│     LLM     │                     │  External   │
│  (tool use) │                     │   Systems   │
└─────────────┘                     └─────────────┘

Architecture

Client Architecture (`System.Agents.MCP.Client`)

-- Client runtime managing MCP server process
data Runtime = Runtime
    { procHandle :: ProcessHandle
    , stdinHandle :: Handle
    , stdoutHandle :: Handle
    , toolsList :: TVar [ToolDescription]
    , callResults :: TVar (Map CallId Value)
    , serverName :: Text
    }

Server Architecture (`System.Agents.MCP.Server`)

The framework can also act as an MCP server, exposing agents as tools to other MCP clients:

multiAgentsServer :: 
    McpServerConfig -> 
    [AgentTree.Props] -> 
    IO ()

MCP Server Configuration

Simple Binary Configuration

{
  "mcpServers": [
    {
      "tag": "McpSimpleBinary",
      "contents": {
        "name": "filesystem",
        "executable": "/usr/bin/mcp-server-filesystem",
        "args": ["--root", "/home/user/projects"]
      }
    }
  ]
}

Environment variables (`env`)

An MCP server can receive configuration, including secrets, through environment variables. env maps a variable name to a binding value: a literal, or a reference to an agent parameter (see parameters-and-bindings.md). The variables are added to the inherited environment (and win on a clash).

{
  "tag": "McpSimpleBinary",
  "contents": {
    "name": "github",
    "executable": "/usr/bin/mcp-server-github",
    "args": [],
    "env": {
      "GITHUB_TOKEN": {"tag": "Param", "contents": "github_token"},
      "GITHUB_HOST": {"tag": "Literal", "contents": "github.com"}
    }
  }
}

The server starts once per agent tree, so a Param must have a value at load time (process scope: --set, --params-file, or a default). A parameter with no such value, such as a session-scope one, is a load error. The values are not put on the command line, and the traced process description shows variable names only, so a secret parameter does not leak through ps or traces.

Haskell Configuration Type

data McpServerDescription
    = McpSimpleBinary McpSimpleBinaryConfiguration

data McpSimpleBinaryConfiguration = McpSimpleBinaryConfiguration
    { name :: Text           -- Display name
    , executable :: FilePath -- Path to server binary
    , args :: [Text]         -- Command line arguments
    , env :: Maybe (Map Text BindingValue) -- Extra environment variables
    }

Protocol Flow

1. Server Initialization

Agent Runtime                    MCP Server
     │                                │
     │── Spawn process ──────────────>│
     │   (executable with args)       │
     │                                │
     │<─ Initialize request ──────────│
     │   (protocol version, etc.)     │
     │                                │
     │── Initialize response ────────>│
     │   (server capabilities)        │
     │                                │

2. Tool Discovery

-- Request tools list
sendRequest :: Runtime -> Method -> Value -> IO CallId

-- Receive tool descriptions
receiveTools :: Runtime -> IO [ToolDescription]

3. Tool Execution Loop

Agent Runtime                    MCP Server        LLM
     │                                │             │
     │<───────────────────────────────│<── Tool call│
     │   (Tool call from server)      │   request   │
     │                                │             │
     │── Execute tool locally ────────│             │
     │   (if agent-as-tool)           │             │
     │                                │             │
     │── Tool result ────────────────>│             │
     │                                │             │
     │<───────────────────────────────│─── Forward ─>│
     │                                │    result   │

JSON-RPC Protocol

Message Format

data JsonRpcMessage
    = JsonRpcRequest 
        { jsonrpc :: Text
        , method :: Text
        , params :: Maybe Value
        , id :: Maybe CallId
        }
    | JsonRpcResponse
        { jsonrpc :: Text
        , result :: Maybe Value
        , error :: Maybe JsonRpcError
        , id :: CallId
        }
    | JsonRpcNotification
        { jsonrpc :: Text
        , method :: Text
        , params :: Maybe Value
        }

Tool List Method

Request:

{
  "jsonrpc": "2.0",
  "method": "tools/list",
  "id": 1
}

Response:

{
  "jsonrpc": "2.0",
  "result": {
    "tools": [
      {
        "name": "read_file",
        "description": "Read contents of a file",
        "inputSchema": {
          "type": "object",
          "properties": {
            "path": {
              "type": "string",
              "description": "Path to the file"
            }
          },
          "required": ["path"]
        }
      }
    ]
  },
  "id": 1
}

Tool Call Method

Request:

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "read_file",
    "arguments": {
      "path": "/home/user/README.md"
    }
  },
  "id": 2
}

Response:

{
  "jsonrpc": "2.0",
  "result": {
    "content": [
      {
        "type": "text",
        "text": "# Project README\n\nThis is the content..."
      }
    ],
    "isError": false
  },
  "id": 2
}

Client Runtime

Starting a Client

startMcpClient :: 
    Tracer IO Trace ->
    McpSimpleBinaryConfiguration -> 
    IO (Either McpError Runtime)
startMcpClient tracer config = do
    -- Spawn process
    let procConfig = (proc (unpack config.executable) 
                            (map unpack config.args))
           { std_in = CreatePipe
           , std_out = CreatePipe
           , std_err = Inherit
           }
    (stdinH, stdoutH, _, procH) <- createProcess_ config.name procConfig
    
    -- Initialize communication
    runtime <- Runtime procH stdinH stdoutH 
                <$> newTVarIO []
                <*> newTVarIO Map.empty
                <*> pure config.name
    
    -- Query tools
    tools <- queryTools runtime
    atomically $ writeTVar (toolsList runtime) tools
    
    return $ Right runtime

Tool Execution

callTool :: 
    Runtime -> 
    Text ->       -- Tool name
    Value ->      -- Arguments
    IO (Either McpError Value)
callTool rt toolName args = do
    callId <- nextCallId
    
    -- Send call request
    sendRequest rt "tools/call" $ object
        [ "name" .= toolName
        , "arguments" .= args
        ]
    
    -- Wait for response
    waitForResponse rt callId

Server Mode

The framework can expose agents as MCP servers:

Starting MCP Server Mode

agents-exe mcp-server --agent-file agent.json

Server Capabilities

data McpServerConfig = McpServerConfig
    { serverName :: Text
    , serverVersion :: Text
    , supportedProtocols :: [ProtocolVersion]
    }

defaultMcpServerConfig :: McpServerConfig
defaultMcpServerConfig = McpServerConfig
    { serverName = "agents-mcp-server"
    , serverVersion = "1.0.0"
    , supportedProtocols = ["2024-11-05"]
    }

Agent-as-Tool Mapping

Each loaded agent is exposed as an MCP tool:

agentToMcpTool :: AgentTree.AgentTree -> ToolDescription
agentToMcpTool tree = ToolDescription
    { toolName = agentSlug tree
    , toolDescription = agentAnnounce tree
    , toolInputSchema = object
        [ "type" .= ("object" :: Text)
        , "properties" .= object
            [ "prompt" .= object
                [ "type" .= ("string" :: Text)
                , "description" .= ("Prompt to send to the agent" :: Text)
                ]
            ]
        , "required" .= (["prompt"] :: [Text])
        ]
    }

Error Handling

MCP Errors

data McpError
    = ProcessStartError Text
    | ProtocolError Text
    | ToolNotFound Text
    | ToolCallError Text
    | ParseError Text
    | TimeoutError

Error Response Format

{
  "jsonrpc": "2.0",
  "error": {
    "code": -32600,
    "message": "Invalid Request",
    "data": "Additional error details"
  },
  "id": null
}

Tracing

MCP operations are traced for debugging:

data Trace
    = McpClientClientTrace ClientTrace
    | McpClientRunTrace RunTrace
    | McpClientLoopTrace LoopTrace
    
data ClientTrace
    = SendingRequest CallId Method Value
    | ReceivedResponse CallId Value
    | ReceivedError CallId JsonRpcError

data RunTrace
    = RunCommandStart Text
    | RunCommandStopped Text ExitCode
    | RunBufferMoved ByteString ByteString

data LoopTrace
    = ToolsRefreshed [ToolDescription]
    | StartToolCall Text Value
    | EndToolCall Text Value Value
    | ExitingToolCallLoop

Best Practices

Server Implementation

Idempotency: Tools should be safe to call multiple times
Timeouts: Set reasonable timeouts for operations
Validation: Validate all inputs before execution
Error messages: Return clear, actionable error messages
Resource cleanup: Properly cleanup on process exit

Client Usage

Reconnect logic: Handle server crashes gracefully
Tool caching: Cache tool lists but refresh periodically
Concurrent calls: Be aware of server concurrency limits
Input validation: Validate arguments before sending

Example: Filesystem MCP Server

{
  "tag": "McpSimpleBinary",
  "contents": {
    "name": "filesystem",
    "executable": "npx",
    "args": ["-y", "@modelcontextprotocol/server-filesystem", "/home/user/docs"]
  }
}

Example: Custom MCP Server

A minimal MCP server in Python:

#!/usr/bin/env python3
import json
import sys

def send_message(msg):
    print(json.dumps(msg))
    sys.stdout.flush()

def handle_request(request):
    method = request.get("method")
    
    if method == "initialize":
        return {
            "protocolVersion": "2024-11-05",
            "capabilities": {},
            "serverInfo": {"name": "example", "version": "1.0.0"}
        }
    
    elif method == "tools/list":
        return {
            "tools": [
                {
                    "name": "echo",
                    "description": "Echo back the input",
                    "inputSchema": {
                        "type": "object",
                        "properties": {
                            "message": {"type": "string"}
                        },
                        "required": ["message"]
                    }
                }
            ]
        }
    
    elif method == "tools/call":
        params = request.get("params", {})
        name = params.get("name")
        args = params.get("arguments", {})
        
        if name == "echo":
            return {
                "content": [{"type": "text", "text": args.get("message", "")}],
                "isError": False
            }
    
    return None

def main():
    while True:
        line = sys.stdin.readline()
        if not line:
            break
        
        request = json.loads(line)
        result = handle_request(request)
        
        if "id" in request:
            send_message({
                "jsonrpc": "2.0",
                "result": result,
                "id": request["id"]
            })

if __name__ == "__main__":
    main()

Debugging

Enable MCP tracing with verbose logging:

agents-exe run --agent-file agent.json --log-http http://localhost:8080/log

View MCP communications in the trace output:

{
  "e": {
    "server": "filesystem",
    "val": {"x": "tool-call-start", "name": "read_file"}
  }
}

Export/Import

2026-10-04T15:53:54Z

Generated from documentation/export-import.md, the repository is the canonical source and may be ahead of this page.

Export/Import

Status: Removed

The export and import commands have been removed from agents-exe. They were wonky and hard to use, so we’re taking another approach (see issue #339).

Alternatives

For sharing agents and tools, consider these alternatives:

1. Direct File Copy

Tools and agent configurations are just files. You can copy them directly:

# Copy tools to another agent
cp ./source-agent/tools/* ./target-agent/tools/

# Copy entire agent configuration
cp -r ./my-agent ./shared-agents/

2. Git Submodules

For team sharing, use git submodules:

# Add shared tools as submodule
git submodule add https://github.com/company/shared-tools.git tools/shared

# Update when needed
git submodule update --remote

3. Package Managers

Use standard package managers for your platform:

Nix: Package tools as nix derivations
Homebrew: Tap for macOS/Linux
Apt/Yum: System packages for tools

4. Container Distribution

Package agents with their tools in containers:

FROM agents-exe:latest
COPY ./tools /app/tools/
COPY ./agent.json /app/

Historical Context

The original export/import system supported:

Exporting agents, tools, and MCP servers to archives (tar.gz, zip)
Git-based distribution with namespacing
Tool installation with conflict resolution
Namespace-based organization

This functionality was removed because:

Complexity: The namespace and packaging system added significant complexity
Overlap: Functionality overlapped with standard tools (tar, git, cp)
Usage patterns: Users preferred direct file management
Maintenance burden: High effort to maintain relative to usage

Migration

If you were using export/import, migrate to:

Simple archives: Use tar directly

# Instead of: agents-exe export --output pkg.tar.gz
tar czf pkg.tar.gz ./my-agent/

# Instead of: agents-exe import --from-file pkg.tar.gz
tar xzf pkg.tar.gz -C ./destination/

Git workflows: Use standard git workflows

# Clone and copy what you need
git clone https://github.com/company/agents /tmp/agents
cp /tmp/agents/tools/* ./my-tools/

Configuration management: Use tools like Ansible, Chef, or simple shell scripts

Web server embedding — Implementation Progress

2026-10-04T15:53:54Z

Generated from todos/web-server-embedding.progress.md, the repository is the canonical source and may be ahead of this page.

Web server embedding — Implementation Progress

Progress tracker for todos/web-server-embedding.md.

Phase 1 — Storage wiring fix ✅ COMPLETE

Combinators.StoreSessionProgress:
- SessionSink (SinkBackend / SinkFiles / SinkNone), chosen when the agent is built.
- agentPersistSession and sinkStoreCallback.
- agentStoreSession and agentStoreSessionWithCallback keep their behaviour (backend read from the wrapped agent, else files) on top of sinkStoreCallback.
CLI/SessionDurable.handleStart derives the conversation ID from the session ID, so session start --step no longer writes a second conv.<random>.json next to conv.<session-id>.json.

Phase 2 — Single agent factory ✅ COMPLETE

New System.Agents.AgentFactory:
- buildAgent :: Tracer IO Trace -> AgentDeps -> AgentRole -> ConversationId -> OSAgentNode -> IO Agent.
- AgentDeps (API keys, session sink, continuation store, tool cache, completion override), defaultAgentDeps, fileAgentDeps.
- AgentRole = RootAgent | SubAgent parentConv callStack.
- Trace (moved from OneShot, which re-exports it) and mapProgressiveDisclosureTrace.
Callers moved onto it, and their copies of the agent record, the OpenAI completion setup, API key lookup, and toolRegistrationToSystemTool removed:
- OneShot.nodeToAgent / nodeToAgentWithThinking (also used by the TUI): thinking output, media injection, and the --session-file copy are now decorators on the factory’s agent.
- AgentTree.OneShotTool.turnAgentRuntimeIntoIOTool takes AgentDeps instead of a SessionStore and API keys. OneShotTool.Trace is now a single OneShotTrace constructor.
- MCP.Server.runAgentWithQuery.
- CLI.SessionDurable.buildAgentForFile (drops its own JSON re-parse).
toolRegistrationToSystemTool has one definition, exported from Combinators.ProgressiveDisclosure.
Session.Types.newSessionFromPrompt, used by session start.

Behaviour changes

Progressive disclosure now applies to the MCP server and the durable session commands too; before, only run, the TUI, and sub-agents had it.
The MCP server applies the agent’s executionMode / toolCallPolicyConfig and runs with runUntilBlocked. A turn waiting on deferred calls returns an error rather than looping forever.
A sub-agent’s session file is named after the conversation ID used for its call-stack entry and OS World entity, not an unrelated random ID.

Verification

cabal build all --enable-tests succeeded. (It did not check warnings: the library was not built with -Wall then, see the Phase 4 section.)
agents-tests: 902 tests pass, including 8 new AgentFactoryTests.
Smoke test of session start, session start --step, and session step with an isolated HOME: exactly one conv.<session-id>.json per session.

Phase 3 — Metadata, versions, migrations, catalog ✅ COMPLETE

Session.Types: SessionStatus (idle / ready / running / waiting_external / failed) with text and JSON forms, and sessionStatusOf. isBlockedOnDeferredCalls (from Session.Loop) and hasBackgroundCalls / backgroundCalls (from Session.Step) moved here; Session.Loop still re-exports isBlockedOnDeferredCalls.
SessionStore:
- SessionLabels, SessionMeta, SessionQuery, VersionConflict.
- SessionBackend gains sbStoreLabelled, sbLoadMeta, sbCompareAndStore, and sbQuery. Every write increments the version; unconditional stores derive the status, except that a running status is kept.
- SQLite: runMigrations with a component-scoped schema_migrations table; migration 2 adds the metadata columns and indexes and derives the status of existing rows. Compare-and-store is a single conditional statement with RETURNING.
- File backend: meta.<uuid>.json sidecars.
- Composite: writes and queries on the primary, loads with fallback.
- SessionCatalog / CatalogEntry, fileCatalog, backendCatalog, isFileBusy.
StoreSessionProgress: sinkStoreCallback and agentPersistSession take SessionLabels; backendStoreCallbackWith.
AgentFactory labels sessions with the agent slug and, for sub-agents, the parent session. OneShotTool gives a sub-agent’s session the ID of its conversation.
Session tools (SystemToolbox/Session) read through introspectionCatalog; getSessionModTime removed. AgentTree.Props takes sessionCatalog; the CLI and TUI pass fileCatalog.
agents.cabal: the library’s import: warnings moved above visibility. Cabal ignored it in its old position, so the library was built without -Wall -Werror since the library was made public.

Not done here

The search index (Session/Search) stays on the file store (CLI-only).
PRAGMA settings are left to withHost (Phase 5).

Verification

agents-tests: 923 tests pass, including 21 new SessionMetadataTests.
The “forced full rebuild shows no warnings” claim made here (and in the Phase 3 commit message) was wrong: that rebuild did not recompile with -Wall -Werror. See the Phase 4 section for the real check and the fixes.

Warning fixes after re-enabling `-Wall -Werror`

A rebuild of the library from an empty build directory under -Wall -Werror reported warnings, now fixed:

From Phases 2 and 3 (this branch): unused imports in Tools/SystemToolbox/Session.hs, TUI/Event/Conversation.hs, CLI/SessionDurable.hs, and MCP/Server.hs.
Already present before this branch: a name binding shadowing a record field in Session/Step.hs (lateResultsQuery), and a missing signature on resetQuitConfirmation in TUI/Event.hs.

Phase 4 — Continuation consistency ✅ COMPLETE

Session.Async:
- ContinuationStore.csFindSession: the session of a pending or completed token.
- The continuation schema goes through runMigrations (component continuations); migration 2 adds a (session_id, completed_at) index.
- csComplete detects the update with RETURNING instead of SELECT changes().
Session.Wake:
- wakeSessionWith returns a WakeOutcome (applied / already completed / unknown tokens) and marks applied tokens completed in the store. wakeSession and wakeSessionWithCache wrap it.
- findSessionForToken (index first, then a scan of the backend) and sessionHasToken.

Not done here

The CLI complete command keeps scanning the file store: CLI agents have no continuation store.

Verification

Library rebuilt from an empty build directory under -Wall -Werror: clean.
cabal build all --enable-tests succeeded.
agents-tests: 930 tests pass, including 7 new ContinuationConsistencyTests.

Phase 5 — Session runner ✅ COMPLETE

System.Agents.Host: Host, HostConfig / defaultHostConfig, HostTrace, HostError, withHost (WAL, busy timeout, migrations, agent files, sub-agent tools storing into the database).
System.Agents.Host.Runner: SessionRunner with createSession, postMessage, resume, completeCall, cancelRun, getSession, awaitRun, deleteSession (dry run and cascade), subscribe, recoverOnStartup, runnerStats, and an idle-session reaper.
Session.Types: DeferredCallView and pendingDeferredCalls; the CLI’s extractDeferredCalls wraps them.
Session.Async: csCountSession and csDeleteSession.

Differences from the first version of the spec (spec updated)

Results completed during a run are queued and applied by the run, instead of being written by completeCall, which would make the run’s next versioned store conflict.
Events use one runner-wide broadcast channel; subscribe returns a blocking “next event” action for one session, which survives eviction.
NoActiveRun error; csCountSession for dry runs; deletion also refused while an ancestor runs; runner agents always run asynchronously.
After a cancel, background calls below the head turn are reported as cancelled by the next run (late results), not by cancelRun itself.

Verification

Library under -Wall -Werror: clean. cabal build all --enable-tests succeeded.
agents-tests: 940 tests pass, including 10 new RunnerTests. The runner tests passed 15 repeated runs. The test suite is not built with -threaded; the bundled SQLite is THREADSAFE=1 (serialized), which the threaded server in Phase 6 relies on.

Phase 6 — `agents-server` executable ✅ COMPLETE

examples/agents-server/:
- src/AgentsServer/Api.hs: the wai application (routing, JSON views, wait/timeout, SSE with keepalives and a shutdown flag).
- src/AgentsServer/Log.hs: JSON-line logs on stderr, traces summarised field by field.
- src/AgentsServer/Server.hs: CLI options, startup (host, runner, startup recovery), warp settings, SIGTERM/SIGINT handling.
- app/Main.hs, test/Main.hs.
agents.cabal: private library agents-server-internal, executable agents-server, test suite agents-server-tests (all threaded where it matters). wai 3.2.4 and warp 3.4.12 were already in the build plan, so nothing new is downloaded.
Host.Runner:
- Fixed subscribe: it filtered with retry after readTChan, which rolls the read back, so a subscriber blocked forever on the first event of another session. The Phase 5 tests only ever had one session emitting. New regression test a subscriber skips other sessions' events (fails on the old code: no events after 5 s).
- New subscribeSTM, used by the events stream.
documentation/agents-server.md; links from documentation/durable-workflows-howto.md and README.md; added to extra-doc-files.

Differences from the first version of the spec (spec updated)

The app code is a private sub-library so the test suite can run it.
SessionMetaView is the SessionMeta JSON, so it includes owner (always null for now).
Status codes: 202 exactly when the stored status is running at the time of answering, 200 otherwise; creation always 201, with a Location header.
In the events stream, a run’s session.updated (running version) comes just before its run.started: the runner stores, then announces.
Additions: --live-session-ttl, --shutdown-grace, filename on media, mode on resume, 404 not_found, 405, 413 (32 MiB bodies), list limit 1–500 (default 50), comma-separated status.
On shutdown, waiting requests answer at once and event streams end, so the grace period is only for other requests.

Verification

cabal build all --enable-tests under -Wall -Werror: clean.
agents-tests: 941 tests pass (one new runner test).
agents-server-tests: 7 tests pass; 30 repeated runs plus 10 with +RTS -N4, no failures.
Smoke tests of the binary against a fake OpenAI endpoint:
- health, agents, and a created session answered by the LLM;
- a deferred call completed over HTTP, with the events stream;
- a restart on the same database, with the session still there;
- SIGTERM with an open events stream: exit 0 in 0.01 s;
- the API key reached the LLM but not the log.
The guide’s Haskell snippet typechecks.

Phase 7 — Authentication and owners ✅ COMPLETE

Spec: new “Milestone 2” section planning Phases 7–12, with the defaults taken recorded as decisions 5–10.
SessionStore: SessionQuery.sqOwner (SQLite, file, composite), and migration 3 adding an (owner, updated_at) index.
Host.Runner: createSessionAs (with an owner) and sessionOwner, which returns the owner of the root session.
agents-server:
- AgentsServer.Auth: loads the tokens file (sha256 or plain token entries), compares tokens by SHA-256 digest, parses Bearer headers.
- --auth-tokens: 401 unauthorized without a valid token (/healthz stays open). Every session endpoint checks the owner, answering 404 for another owner’s session. Continuations check the owner of the token’s session, answering 404 unknown_token. Listing filters by owner, or by an owned parent.
- server.started logs authentication: bearer|none.
documentation/agents-server.md: an Authentication section.

Not done here

Per-owner API keys and a default isolation policy (see “Remaining later work” in the spec).

Verification

cabal build all --enable-tests under -Wall -Werror: clean.
agents-tests: 941 pass. The delete test now also checks that a sub-session has its root’s owner. The migrations test expects migrations 1–3.
agents-server-tests: 9 pass, including the two-owner flow and loading a tokens file.

Phase 8 — MCP over HTTP ✅ COMPLETE

AgentsServer.Mcp: the Streamable HTTP transport, answering with JSON, one ask_<slug> tool per root agent, runs through the session runner.
AgentsServer.Api:
- routes POST /mcp (405 for other methods);
- refuses non-loopback Origins when authentication is off (403 forbidden_origin), as the MCP transport requires against DNS rebinding;
- waitForRun is shared by the REST and MCP handlers.
documentation/agents-server.md: an MCP over HTTP section, and the origin rule.

Verification

agents-server-tests: 12 pass. New tests: an MCP flow (initialize with version negotiation, the notification’s 202, tools/list, tools/call with _meta.session_id, unknown tool, unknown method, batch, GET 405), a deferred call reported through MCP then completed over REST, and origin checks with and without authentication.
The official Python MCP SDK (2.2.0, streamable_http_client) against the binary and a fake OpenAI endpoint: initialize (2025-06-18), list tools, call ask_weather, which reported the deferred call and its token.

Phase 9 — TUI out of the core library ✅ COMPLETE

git mv of 25 modules from src/ to tui/: TUI.* except ToolCallActivity, plus CLI.TUI, CLI.Config, and CLI.
agents.cabal: new public library agents-tui. agents-lib loses those modules and brick, vty, text-zipper, data-clist. agents-exe adds agents-tui.
documentation/architecture.md: a “Libraries and executables” table.

Verification

cabal build all --enable-tests: clean. The build plan shows brick, vty, and text-zipper only under agents-tui, and nowhere in agents-lib’s transitive closure.
cabal test all: 941 + 12 pass. agents-exe --help runs.

Phase 10 — Postgres backend ✅ COMPLETE

agents-postgres library and agents-postgres-tests suite (see the spec).
Host.withHostStores / HostStores; withHost uses it.
agents-server --db postgresql://…; redactDatabase for the log.
Docs: a Postgres section in documentation/agents-server.md, and the library in the table in documentation/architecture.md.

Verification

agents-postgres-tests: 5 pass against a throwaway Postgres 16 cluster.
Smoke test of the binary on a Postgres 16 cluster:
- created a session blocked on a deferred call, restarted the server, and completed the call: the session reached idle, with consistent rows;
- the URL’s password is absent from the log, which holds only JSON lines.

Phase 11 — Token streaming ✅ COMPLETE

LLMs/OpenAIStream.hs (new), HttpClient.postStream, OpenAI.callLLMPayloadStreaming and withOverloadedRetry, OpenAICompletionConfig.cfgOnTextDelta, AgentDeps.adOnTextDelta, HostConfig.hcStreamTokens, SessionEvent.TextDelta, agents-server --stream-tokens.
Docs: a “Streaming answers” section and the text.delta event in documentation/agents-server.md.

Verification

agents-tests: 5 new OpenAIStreamTests.
agents-server-tests: 13 pass, including streaming against a fake streaming endpoint.
The non-streaming path through the binary still works (fake endpoint, deferred call reported).

Phase 12 — Agents from the database ✅ COMPLETE

AgentStore (new), AgentTree.loadAgentTreeFromConfig, stored agents in Host, mkPostgresAgentStore, /v1/agents/:slug endpoints, --admin-owners. See the spec for the differences from the plan.
Docs: a “Storing agents” section in documentation/agents-server.md.

Verification

agents-tests: 947 pass (a new runner test for stored agents).
agents-server-tests: 15 pass.
agents-postgres-tests: 6 pass.
Smoke test of the binary with authentication:
- a non-admin PUT is refused (403); an admin PUT stores an agent whose MCP server is agents-exe mcp-server, and its tool (mcp_weather_ask_weather_000) is listed;
- after a restart the agent is reloaded from SQLite with its tool;
- no tokens in the logs, and no MCP server process left after shutdown.

Async Tool Calls Implementation Progress

2026-10-04T15:53:54Z

Generated from todos/async-tool-calls-progress.md, the repository is the canonical source and may be ahead of this page.

Async Tool Calls Implementation Progress

Plan

See todos/async-tool-calls.md.

Phase Status

[x] Phase 1: ECS Tool-Call Entities
[x] Phase 2: Async Engine
[x] Phase 3: System Capability (building blocks only — see review below)
[x] Phase 3.5: End-to-End Correctness (added after review; blocks Phase 4)
[x] Phase 4: TUI / OneShot UX (TUI rendering not yet checked by hand)
[x] Phase 5: Cleanup & Hardening (tracing not done — see notes)

Success Criteria Status

Revised after the 2026-09-16 review, then after Phase 3.5 and Phase 4. The criteria are met in the library and tests, and async execution is now enabled from the agent JSON in the TUI, OneShot, sub-agents, and session commands. The TUI rendering has not been checked by hand.

Criterion	Status	Notes
Two long-running tool calls issued in one LLM turn execute concurrently	Met	`asyncEngineTests`; async calls now start before inline calls so they overlap.
LLM receives a partial user turn as soon as the first call finishes (`YieldOnAnyProgress`)	Met (stepper)	Running calls get a placeholder tool message; late results are delivered in the next user turn. `AsyncToolCallsTests`.
LLM can call `get_tool_call_status` and see structured progress or final result	Met	Accepts the provider id. Bash tools report output lines; sub-agent tools report their steps.
LLM can call `cancel_tool_call` to stop a running call	Met	Interrupts the engine thread via `ctxCancelToolCall`; a running subprocess is terminated (`processKilledOnCancel`).
Tool calls appear as ECS entities in the OS `World`	Met	Covered by `sessionStepEntityTests` and `toolCallEntityTests`.
Existing synchronous behavior unchanged when `ExecutionMode = Synchronous`	Met	Same turns as before. Sync steps only differ when a session already holds background calls from an async run.
`get_tool_call_status` returns `orphaned` for calls referenced in history that no longer exist	Met	Capability returns `orphaned` with `is_final: true`; the stepper fails such calls. Verified across a save/reload round-trip in a fresh world (`orphanedAcrossRestart`).

Phase 1 Completion Notes

Phase 1 is complete. The following was done:

ECS Tool-Call Components

System.Agents.OS.Conversation.Types already had ToolCallConfig/ToolCallState extended with:
- tcSessionId, tcConversationId on ToolCallConfig.
- tcProgress :: [ToolCallProgress], TcCancelled on ToolCallState.
- ToolCallProgress / ProgressKind for structured progress.

Tracked Tool Calls

System.Agents.Session.Types already had tcEntityId :: Maybe EntityId added to TrackedToolCall.

Tool-Call Entity Helpers

Extended System.Agents.OS.Conversation.ToolCalls:
- Added ensureToolCallComponents / ensureToolCallComponentsIO for idempotent component-store registration (safe to call on every step).
- createToolCallEntity, startToolCall, completeToolCall, failToolCall, cancelToolCall, addToolCallProgress.
- Query helpers findToolCallEntityBySessionId and listToolCallsBySessionAndConversation.

Session Step Integration

Updated System.Agents.Session.Step:
- prepareAgentWorld ensures tool-call component stores are registered idempotently and threads the registered world through the agent/context.
- ensureTrackedCallEntities promotes every TrackedToolCall to an OS entity when a World is available.
- executeTrackedCallWithEntity updates the OS entity state to TcExecuting before running a call and TcCompleted (with the JSON result) after.
- Both runStepMSync and runStepMAsync now create/update OS entities for every tool call.
- Existing behavior is unchanged when no World is configured.

Tests

Added sessionStepEntityTests to test/OS/ConversationTests.hs covering:
- Sync step creates completed tool-call entities.
- Async step creates entities for both sync-completed and deferred calls.
- Async step with all-sync policy completes all calls and creates entities.

Baseline

Full test suite passes.

Phase 2 Completion Notes

Phase 2 is complete. The async engine now runs RunAsync tool calls concurrently, keeps OS entity state in sync, and supports structured progress callbacks.

Async Engine (`System.Agents.Session.Async.Engine`)

AsyncEngine manages a shared semaphore for max concurrency and a registry of active batches keyed by ToolCallId.
startAsyncBatch spawns each call in a background Async thread, injects a per-call progress callback into ToolExecutionContext, and records the call in the registry.
waitForProgress / waitForProgressTimeout block until at least one call reaches a final state.
finalizeCompleted collects finished calls and removes them from the running set and engine registry.
cancelToolCall / cancelAsyncBatch send async exceptions to background threads and mark OS entities as TcCancelled.

Yield Strategy

AsyncYieldStrategy (YieldOnAnyProgress, YieldWhenAllDone, YieldOnTimeout) is defined in System.Agents.Session.Types.
waitAccordingToStrategy in System.Agents.Session.Step implements the three strategies.
The default strategy is YieldWhenAllDone (set by withAsyncConfig).

Progress & Lifecycle

When a background call starts, the engine emits a ToolCallProgress entry with ProgressStarted (payload {"started":true}) alongside setting the entity status to TcExecuting.
The progress callback writes ProgressPartial entries with arbitrary JSON payloads to the entity.
completeToolCall and failToolCall in System.Agents.OS.Conversation.ToolCalls now leave the state unchanged if it is already TcCancelled, preventing late engine completions from overwriting a cancellation.

Helpers

Added isToolCallCompletedIO and findToolCallEntityByToolCallId to System.Agents.OS.Conversation.ToolCalls.

Tests

Added asyncEngineTests to test/OS/ConversationTests.hs covering:
- Two RunAsync calls execute concurrently (total time < 300 ms for two 200 ms sleeps).
- Progress callback emits ToolCallProgress entries, including ProgressStarted.
- cancelToolCall marks an entity as TcCancelled and a later engine completion does not overwrite it; a second cancellation attempt returns False.

Baseline

Full test suite passes.

Phase 3 Completion Notes

Phase 3 is complete. Agents can now inspect and cancel async tool calls through the system toolbox.

Capability Constructors

Added to SystemToolCapability in System.Agents.Base:
- SystemToolGetToolCallStatus serialized as get-tool-call-status.
- SystemToolListRunningToolCalls serialized as list-running-tool-calls.
- SystemToolCancelToolCall serialized as cancel-tool-call.

Types (`System.Agents.Tools.SystemToolbox.Types`)

GetToolCallStatusParams — tool_call_id, include_progress, wait_for_completion, timeout_seconds.
ToolCallStatusResult — status, tool name, timing, result, progress, is_final.
ListRunningToolCallsResult / RunningToolCallInfo.
CancelToolCallParams / CancelToolCallResult.

Implementation (`System.Agents.Tools.SystemToolbox.ToolCallStatus`)

getToolCallStatus looks up the OS entity by ToolCallId and translates ToolCallStatus to LLM-facing strings (pending, running, completed, failed, cancelled).
- Supports wait_for_completion with a configurable timeout (polls every 50 ms).
- If the entity is missing, searches the session’s PartialUserTurn history and returns orphaned when the call is referenced but no longer exists.
listRunningToolCalls returns all non-final tool-call entities for the current session/conversation.
cancelToolCallById marks a non-final entity as TcCancelled and reports the previous status; returns cancelled: false if already final.

Registration (`System.Agents.ToolRegistration`)

Updated capabilityToText and buildSystemToolParams to expose the new capabilities and their parameters (tool_call_id, include_progress, wait_for_completion, timeout_seconds, reason).
Routed get-tool-call-status, list-running-tool-calls, and cancel-tool-call in the system_info tool handler.

Exports

System.Agents.Tools.SystemToolbox re-exports the new types and functions.
agents.cabal exposes System.Agents.Tools.SystemToolbox.ToolCallStatus.

Tests

Added toolCallStatusTests to test/OS/ConversationTests.hs covering:
- get-tool-call-status returns pending then running.
- get-tool-call-status returns completed after completion.
- get-tool-call-status returns orphaned for a tracked call missing an OS entity.
- list-running-tool-calls returns only non-final calls.
- cancel-tool-call marks a running entity as cancelled and a second attempt reports already final.

Baseline

Full test suite passes (870 tests).

Review (2026-09-16)

State at review time: branch asynctools, build green, 870 tests pass. Phases 1–3 delivered the building blocks (entities, concurrent engine, status/list/cancel capabilities), but the feature does not work end-to-end in a real session. Findings:

R1. The LLM cannot address its own tool calls

get-tool-call-status / cancel-tool-call take the internal ToolCallId UUID generated in mkReadyTrackedCall (Session/Step.hs, newToolCallId).
The model only ever sees the provider’s id (e.g. call_abc123). There is no mapping between the two and the UUID is never surfaced to the model.

R2. Partial answers never reach the LLM correctly

turnToMessages (PartialUserTurn …) in Session/OpenAI.hs emits tool messages for completed calls only. A running call has no tool message, which OpenAI-compatible APIs reject.
naiveTilNoToolCallStep (Session/Step.hs) only treats Ready / Deferred as outstanding. With only Running calls left, it asks for an LLM completion with a partial set of responses, then pushes an LlmTurn on top. The running calls’ results are buried and never delivered.
continuePartialTurn → executeTrackedCalls prepends a new PartialUserTurn instead of replacing the head (unlike Session/Wake.hs, which does newTurn : drop 1 turns). Every resume duplicates the turn in history. Pre-existing bug, but running calls now trigger it on every resume.

R3. `cancel-tool-call` does not stop anything

cancelToolCallById (Tools/SystemToolbox/ToolCallStatus.hs) only calls TCT.cancelToolCall, i.e. flips the entity to TcCancelled.
The background thread keeps running. Engine.cancelToolCall does kill the thread, but the capability cannot reach the engine from ToolExecutionContext.
The capability test only asserts the status flag.

R4. The engine does not survive between steps

runAsyncWithProgress (Session/Loop.hs) returns only the Session; the agent carrying ctxAsyncEngine is dropped. The caller resumes with an agent that has no engine.
ensureAsyncEngine then creates a fresh engine on each step, so defaultMaxConcurrency = 4 is per step rather than global, and the cancellation registry is lost.

R5. The stepper does not handle orphaned calls

pollRunningCall leaves a call Running forever when its entity is missing (e.g. after a restart), so the partial turn never resolves.
Only the capability reports orphaned, and it returns is_final: false for a call that can never finish.

R6. Smaller issues

Crash on malformed calls: requireEntityId in Session/Async/Engine.hs calls error when a call has no entity id. That happens whenever parseToolCallFromLlmToolCall fails, so a malformed call under an async policy crashes the step instead of degrading to inline.
No real overlap: in executeTrackedCalls, inline calls run sequentially before the async batch is started.
No progress emitters: nothing calls ctxProgressCallback. Also, mkSubcallContext propagates the callback into sub-agent contexts, which would write their progress into the parent call’s entity.
Hard to enable: the agent JSON has executionMode and toolCallPolicyConfig, but only the durable session CLI applies them (applyAgentDurableConfig in CLI/SessionDurable.hs). The TUI and OneShot ignore them, and there is no config for yield strategy or concurrency.

Phase 3.5 Completion Notes

Done 2026-09-16. Build green, 875 tests pass (5 new end-to-end tests).

Semantics

Placeholders. partialToolMessages (Session/Types.hs) emits one tool message per call. Final calls carry their result. Non-final calls, and calls whose result was delivered late, get a JSON placeholder (status, tool_call_id, a hint to use get-tool-call-status / cancel-tool-call). Used by OpenAI.hs, both naive step functions, and Wake.hs. Failed calls now also get their tool message (previously dropped).
Late delivery. collectLateResults (Session/Step.hs) finds Running calls in partial turns below the head, polls them, marks finished ones tcDeliveredLate = True (new field on TrackedToolCall, optional in JSON), and lateResultsQuery renders them into the next user turn’s userQuery (media is attached). The partial turn keeps showing the placeholder, so history matches what the model saw. Runs in both sync and async steps.
Stopping. naiveTilNoToolCallStep no longer stops on an LLM turn without tool calls while hasBackgroundCalls. It asks for a user turn, and the stepper blocks until the results arrive (all under YieldWhenAllDone, otherwise at least one).
Refresh. runStepMAsync refreshes the head partial turn before each step (refreshHeadPartialTurn). Background completions are picked up, and the head becomes a full UserTurn once every call is final.
Resume. executeTrackedCalls takes replaceHead; continuePartialTurn replaces the head instead of stacking. It also waits, per yield strategy, on calls carried over from earlier steps together with newly started ones. Finalization accepts Completed or Failed.
Message order. OpenAI.hs now emits tool messages before the user message in a turn (required once a user turn carries both).

Engine / ids / cancellation

ToolCallConfig.tcProviderCallId stores the provider’s id (createToolCallEntityWithProviderId); findToolCallEntityByProviderCallId resolves it within a session and conversation, preferring in-flight then most recently started calls (Kimi-style ids repeat).
Capabilities accept the provider id or the internal UUID; list-running-tool-calls returns the provider id. Session fallback reports final calls from history, Running as orphaned (is_final: true), and deferred as pending. waitForFinal blocks on STM instead of polling.
runStepMAsync installs one engine on the agent (prepareAsyncEngine) and keeps it on Evolve. buildContext exposes Engine.cancelToolCall as ctxCancelToolCall. cancel-tool-call uses it, falling back to marking the entity.
Engine: asynchronous exceptions are no longer swallowed (cancel really interrupts the tool), each call unregisters itself when it finishes, cancelToolCall reports whether the entity actually ended up cancelled, and TCT.cancelToolCall never overwrites a final state. startAsyncBatch ignores calls without an entity; the stepper runs those inline.
Orphans: pollRunningCall fails a Running call whose entity (or world) is missing.
withAsyncEngine registers tool-call stores first and stores the resulting world on the agent, so engine and agent share the same world value.
Waits use System.Timeout rather than registerDelay, which needs the threaded RTS (the test suite is not threaded).

Tests (`test/AsyncToolCallsTests.hs`)

Fast and gated slow call with YieldOnAnyProgress: partial turn, placeholder seen by the LLM, stepper blocks and delivers the late result exactly once, no duplicated partial turn, every completion pairs each tool call with exactly one tool message, session stops afterwards.
cancel-tool-call by provider id interrupts a sleeping tool (its onException handler runs, it never finishes), entity is cancelled, next step finalizes the turn.
Running call with a missing entity resolves as orphaned.
Partial turn with a deferred and a running call resumed twice stays a single head turn.
Malformed RunAsync call runs inline instead of crashing.
OS.ConversationTests orphan capability test now uses a Running call and expects is_final: true.

Known gaps carried forward

runAsync / runAsyncWithProgress still return only the session. Resuming in-process with a fresh agent loses the engine registry unless withAsyncEngine was installed up front (documented on both).
Placeholders do not include progress (history is pure; the model can call get-tool-call-status).
If the LLM fetches a result via get-tool-call-status, the late-delivery notice still repeats it once.
A head partial turn with only Deferred calls still spins under run (pre-existing; runAsync pauses correctly).
Cancel against a call whose engine is gone only marks the entity; the thread (if any) keeps running.

Phase 4 Completion Notes

Enablement

Base.Agent gains asyncYieldStrategy and maxConcurrency; the runtime Agent gains ctxMaxConcurrency (used when the engine is created).
applyAgentDurableConfig / buildToolCallPolicy moved to System.Agents.Session.AgentConfig (re-exported by CLI.SessionDurable) and are applied in OneShot.nodeToAgentWithThinking (OneShot, TUI, session commands) and AgentTree.OneShotTool.nodeToAgent (sub-agents).
prepareAgentWorld gives an asynchronous agent without a World a private one; otherwise RunAsync calls silently ran inline.
llmToolCallName now lives in Session.Types.

Example agent JSON:

"executionMode": "asynchronous",
"asyncYieldStrategy": "yieldOnAnyProgress",
"maxConcurrency": 4,
"toolCallPolicyConfig": {"default": {"tag": "runAsync"}, "rules": []}

Progress events

OSEvent_ToolCallActivity ToolCallActivity (OS.Events) with phases started / progressed / completed / failed / cancelled, keyed by session id, conversation id, tool-call id and provider id.
The engine publishes them on the context’s ctxEventQueue. The TUI already bridges its OS event queue to the Brick channel (TUI.Core.startOSEventBridge), so convertOSEvent maps them to AppEvent_ToolCallActivity; no RuntimeBridge change was needed.
Tool-call progress entries on the entity are capped at 50 (newest first).

TUI

TUI.ToolCallActivity keeps a ToolCallViews map (per session, per call) in UIState._toolCallViews: latest phase, start time, last progress. Views are pruned when a session update shows the call is no longer running.
Rendering: a “Background tool calls running: …” line above the turns, and one line per tracked call in partial turns (pending / deferred / running with latest progress / completed / delivered later / failed). No spinner.
While background calls run and the LLM has nothing to do, the TUI asks for user input instead of blocking: the stepper’s new askUserQuery races usrQuery against the background calls (per the yield strategy) and cancels usrQuery if the calls finish first. usrQuery must tolerate cancellation (the TUI reads a BChan, which is STM).

OneShot / loops

--wait-for-async was not added. run already waits for in-process background calls (they cannot outlive the process). Stopping early would only orphan them.
New Loop.runUntilBlocked / isBlockedOnDeferredCalls: returns the session when the head partial turn only waits on deferred calls instead of spinning. OneShot uses it, stores the session under its session id too, and prints a JSON report (status: paused, session_id, deferred calls with continuation tokens) for session complete / session resume.
Markdown export (SessionPrint) lists each call’s status in partial turns and shows finished results. The search index includes failed results and names of all calls in partial turns.

Progress emitters

Bash tools: when ctxProgressCallback is set (async calls), the script runs through Bash.runProcessReportingOutput, which reports {stream, line, lines, bytes} at most every 0.5s per stream. The process is terminated if the call is cancelled. Synchronous calls keep the old path.
Sub-agent tools (OneShotTool) report {message, turns} after each sub-agent step, and now rethrow async exceptions so cancelling them works (they were caught and turned into failures).
mkSubcallContext no longer copies the parent’s progress callback or cancel hook.

Tests (`test/AsyncToolCallsTests.hs`, now 17)

Async agent without a World; ctxMaxConcurrency bound; engine activity events; background results vs user query race (both ways); TUI view apply/prune/summaries; markdown partial turn; runUntilBlocked; subprocess output reports; subprocess terminated on cancel.
SessionDurableTests: yield strategy / max concurrency application and JSON parsing. Full suite: 890 tests.

Known gaps carried forward

TUI rendering and the input race were not exercised by hand.
Bash progress is tested on runProcessReportingOutput directly, not through a registered bash toolbox end to end.
terminateProcess sends SIGTERM to the script only; its children may survive (Phase 5: process groups).
run itself still spins on deferred-only partial turns; only runUntilBlocked users (OneShot) avoid it. The TUI still uses run.
Earlier gaps (engine not returned by runAsync, repeated result after get-tool-call-status) remain.

Phase 5 Completion Notes

Shutdown and cancellation

Engine.shutdownAsyncEngine cancels every batch the engine still owns. Loop.run / runWithProgress / runUntilBlocked shut the engine down when the run ends or throws (tracking the evolving agent in an IORef); runAsync* only on failure, since pausing with calls running is the point.
Bash scripts run in their own process group and are killed with SIGTERM, then SIGKILL after 100ms, so cancelling kills what the script started. Verified: the test fails without the group kill.
The TUI kills conversation threads when quitting (bounded to 2s), which runs the same cleanup.
The TUI uses runUntilBlocked, so a conversation waiting only on deferred calls stops with a status message instead of spinning.

Stale calls

asyncCallTimeoutSeconds (agent JSON) → ctxAsyncCallTimeout → aeCallTimeout. A call that outlives it is interrupted (killing its subprocess) and reported as failed: “async tool call timed out after Ns”.

Orphans across a restart

get-tool-call-status used to need includeFullSession, which is off by default (it also pushes the whole session into every bash tool’s environment). Contexts now carry ctxSessionToolCalls: the tracked calls of the session’s partial turns only. Without it, a reloaded session answered “tool call not found” instead of “orphaned”.
orphanedAcrossRestart writes a session with a running call to disk, reloads it in a fresh World, and checks both the capability and the stepper.

Concurrency

maxConcurrency stays per agent. newAsyncConcurrencyLimit + mkAsyncEngineSharing let a host share one limit across agents (each agent needs its own engine, since the executor is tied to its tools). No rate limiting (calls per minute) was added.

Docs

New documentation/async-tool-calls.md covers configuration, placeholders and late delivery, the capabilities, progress, cancellation, the TUI, one-shot runs, session files and the limits. Linked from documentation/README.md, documentation/durable-workflows-howto.md.
documentation/tools.md lists the three capabilities; documentation/tui.md documents background call display and input while calls run; documentation/cli-commands.md documents the paused JSON report of run.
The agent JSON in the new doc was checked by parsing it with Base.Agent.

MCP / OpenAPI streaming (question from the plan)

OpenAPI tools wait for one complete HTTP response: nothing to stream.
Done: MCP.Client advertises a _meta.progressToken on tools/call when the call has a progress callback, and forwards the server’s notifications/progress for that token to ctxProgressCallback (payload {progress, total?, message?}), so MCP tools report progress like bash tools (tool.progressed events, get_tool_call_status).

Not done

Tracing. Agent carries no tracer (tracers live in the builders’ closures), so Prod.Tracer support would mean threading one through the agent and the engine. The activity events (OSEvent_ToolCallActivity) and the progress entries on the entity are the observability path for now.
Rate limiting beyond a concurrency cap.
A pre-existing docs bug found on the way, now fixed: JSON examples used key spellings the parsers reject. Every builtin toolbox (Name, Description, Capabilities, FileSandbox, …), file sandboxes (fsbPredicate, fsbMaxFileSize, fsbName), bash toolboxes (Path, BasenameFilter), OpenAPI/PostgREST servers (SpecUrl, BaseUrl, Token), the removed Lua allowedPaths field, the old SQLite path/access fields (now Versioning), and the kebab-case agent keys in documentation/advanced-configuration.md (api-key-id → apiKeyId, …). Fixed in documentation/tools.md, documentation/file-loader.md, documentation/advanced-configuration.md and the Agent/bashToolboxes Haddock in System.Agents.Base. Every JSON block in documentation/ now decodes into the real types; the throwaway checker used for this is in the session scratchpad.

Tests (`test/AsyncToolCallsTests.hs`, now 21)

Failing run cancels background calls; call timeout; orphan across a save/reload; subprocess killed on cancel; process group killed on cancel.
Full suite: 894 tests, async group stable over repeated runs.

Completion Plan

Phase 3.5: End-to-End Correctness (done — see completion notes above)

Call ids (R1). Accept the provider’s tool-call id in get-tool-call-status / cancel-tool-call / list-running-tool-calls, either by storing it on ToolCallConfig or by resolving it through the tracked calls. Keep the internal UUID as a fallback. list-running-tool-calls should return the id the model knows.
Placeholder responses (R2). For calls still Running (or Deferred) when the LLM is asked for a completion, emit a tool message such as {"status":"running","tool_call_id":…,"progress":[…]} so the request is valid and the model knows the call is pending.
Delivering late results (R2). A finished result cannot be attached to a tool message the model already answered. Inject it into the next user turn as a notice (e.g. “tool call X completed: …”). Update naiveTilNoToolCallStep so Running calls count as outstanding and so completed-late calls are surfaced.
Partial turns (R2). Replace the head PartialUserTurn on resume instead of prepending a new one (match Session/Wake.hs).
One engine (R3, R4). Either return the agent from runAsync / runAsyncWithProgress, or keep the engine alongside the OS World so every step shares it. Add a cancel hook to ToolExecutionContext so cancel-tool-call actually kills the thread via Engine.cancelToolCall.
Orphaned calls (R5). In pollRunningCall, mark a Running call whose entity is missing as Failed with an “orphaned” response. Return is_final: true for orphaned calls in the capability.
Malformed calls (R6). Replace the error in requireEntityId with a fallback that executes the call inline.
End-to-end tests.
- Two slow calls with YieldOnAnyProgress → PartialUserTurn → resume → full UserTurn, with a valid OpenAI message list at every step (every tool call has exactly one tool message, no duplicated turns).
- cancel-tool-call stops a real running tool (e.g. a sleeping bash command) and the thread is gone.
- A Running call whose entity is missing is resolved as orphaned.

Phase 4: TUI / OneShot UX (done — see completion notes above)

Enablement. executionMode and toolCallPolicyConfig already exist in the agent JSON; apply them in the TUI and OneShot agent builders (reuse applyAgentDurableConfig), and add asyncYieldStrategy and maxConcurrency. Without this nothing in the UI can be exercised.
Progress events. Add AppEvent_ToolCallProgress to System.Agents.TUI.Types. The engine emits through ctxEventQueue when present. Decide how the event reaches the TUI BChan given the TUI still goes through the legacy RuntimeBridge.
Rendering. Show running calls with a spinner and latest progress in TUI/Render/Conversation.hs; handle partial turns in SessionPrint / formatSessionAsMarkdown.
OneShot. Add --wait-for-async. Without it, exit with JSON listing the pending calls and the session to resume.
A real progress emitter. Make one tool call ctxProgressCallback (bash output lines are the obvious candidate). Stop propagating the callback into sub-agent contexts, or give sub-calls their own.

Phase 5: Cleanup & Hardening (done except tracing — see completion notes above)

Cancel running batches on session abort, TUI quit, or exception; shut the engine down with the agent. Kill whole process groups on cancel. Stop run (and the TUI) from spinning on deferred-only partial turns.
Expire stale calls (timeout plus a check that the thread is still alive).
Global max concurrency / rate limiting, configurable (builds on Phase 3.5 step 5).
Tracing for async calls and progress in Prod.Tracer.
Verify orphan handling across a real process restart + session reload.
Update documentation/tools.md, documentation/tui.md, documentation/cli-commands.md, and document the agent JSON schema for async.
Clarify whether MCP / OpenAPI tools can stream into the progress callback.

Agents - AI Agent Framework

2026-10-04T15:53:54Z

Generated from documentation/README.md, the repository is the canonical source and may be ahead of this page.

Agents - AI Agent Framework

A Haskell-based framework for building and orchestrating AI agents with support for multi-agent hierarchies, tool systems, and LLM integrations.

Overview

The Agents framework provides a complete infrastructure for:

Agent Definition: JSON-based agent configurations with system prompts, tool directories, and LLM settings
Multi-Agent Orchestration: Hierarchical agent trees with parent-child relationships and cross-agent references
Tool System: Extensible tool registration with support for bash scripts, MCP servers, and OpenAPI integrations
Session Management: Persistent conversation sessions with turn-based interactions
Durable Workflows: Asynchronous, resumable execution with deferred tool calls and isolated deployments
Multiple Interfaces: CLI, TUI (Terminal UI), MCP server, and an HTTP server

Quick Start

Installation

# Build the project
cabal build

# Run the durable-workflow demonstrator (no API key needed)
cabal run durable-workflow-demo

# Run tests
cabal test

Creating an Agent

Create an agent.json file:

{
  "slug": "my-agent",
  "apiKeyId": "openai",
  "flavor": "openai",
  "modelUrl": "https://api.openai.com/v1",
  "modelName": "gpt-4",
  "announce": "A helpful assistant",
  "systemPrompt": ["You are a helpful assistant."],
  "toolDirectory": "tools",
  "mcpServers": [],
  "extraAgents": []
}

Running the Agent

# Check agent configuration
agents-exe check --agent-file agent.json

# Run in one-shot mode
agents-exe run --agent-file agent.json --prompt "Hello!"

# Start interactive TUI
agents-exe tui --agent-file agent.json

# Start MCP server
agents-exe mcp-server --agent-file agent.json

Configuration

API Keys

Store API keys in ~/.config/agents-exe/secret-keys:

{
  "openai": "sk-...",
  "openrouter": "sk-..."
}

Project Configuration

Create agents-exe.cfg.json in your project root:

{
  "agentsDirectories": ["./agents"],
  "agentsFiles": ["./main-agent.json"],
  "agentsLogs": {
    "logJsonHttpEndpoint": "http://localhost:8080/log",
    "logJsonPath": "./logs/agents.json",
    "logRawPath": "./logs/agents.log"
  }
}

Project Structure

agents/
├── app/                      # Application entry point
│   └── Main.hs              # CLI argument parsing and command routing
├── examples/                # Runnable example programs
│   └── durable-workflow-demo/
│       └── Main.hs          # Self-contained durable workflow demonstrator
├── src/
│   └── System/Agents/
│       ├── Base.hs          # Core types (Agent, AgentId, ConversationId)
│       ├── Runtime.hs       # Agent runtime and execution
│       ├── AgentTree.hs     # Multi-agent hierarchy management
│       ├── Session/         # Session management
│       ├── Tools/           # Tool system
│       ├── MCP/             # Model Context Protocol
│       ├── TUI/             # Terminal UI
│       ├── CLI/             # Command implementations
│       ├── ExportImport/    # Tool sharing
│       └── FileLoader/      # File loading utilities
├── documentation/                    # Documentation
└── test/                    # Test suite

Key Features

Multi-Agent Hierarchies

Agents can reference other agents via:

Tool Directory: Child agents in a subdirectory
Extra Agents: Explicit references via extraAgents field

Tool Types

Bash Tools: Executable scripts in the tool directory
MCP Tools: Model Context Protocol servers
OpenAPI Tools: REST API endpoints via OpenAPI specs
IO Tools: Haskell-based tool implementations

Session Persistence

Sessions are automatically saved and can be resumed:

# Resume a session
agents-exe run --agent-file agent.json --session-file session.json

# Print session history
agents-exe session-print session.json

Durable Workflows

Turns can pause mid-execution, persist, and resume when external results arrive:

# Pause after one async step
agents session pause <session-id> --agent-file agent.json

# List deferred calls
agents session pending <session-id>

# Inject an external result
agents session complete <token> result.json

# Resume until completion or next yield
agents session resume <session-id> --agent-file agent.json

See Durable Workflows How-To for a full walkthrough, including a runnable mock-LLM demonstrator.

Documentation

Architecture - Runtime and core architecture
Tool System - Tool registration and execution
MCP Protocol - Model Context Protocol integration
Session Management - Session lifecycle and persistence
Agent Templates - Write an agent file as a tramaj program: shared sandboxes and toolboxes, values picked by the operator
Parameters, Bindings & Narrowing Sub-Agents - Bind tool arguments to declared parameters, keep secrets out of the model, and narrow sub-agents at call time
Durable Workflows How-To - Resumable async execution
agents-server - Agents over HTTP: sessions, events, OpenAPI, and a chat page
Asynchronous Tool Calls - Background tool calls, progress and cancellation
Terminal UI - TUI interface
CLI Reference - Command reference
Export/Import - Tool sharing and distribution
File Loader - File loading utilities

Architecture Overview

┌─────────────────────────────────────────────────────────────┐
│                        CLI / TUI / MCP                       │
└──────────────────────┬──────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────────┐
│                      AgentTree                               │
│         (multi-agent hierarchy management)                   │
└──────────────────────┬──────────────────────────────────────┘
                       │
┌──────────────────────▼──────────────────────────────────────┐
│                     Runtime                                  │
│    (agent execution, tool registration, LLM calls)           │
└──────────────────────┬──────────────────────────────────────┘
                       │
        ┌──────────────┼──────────────┐
        ▼              ▼              ▼
┌──────────────┐ ┌──────────┐ ┌──────────────┐
│   Session    │ │  Tools   │ │     LLM      │
│  Management  │ │  System  │ │  Integration │
└──────────────┘ └──────────┘ └──────────────┘

Contributing

Fork the repository
Create a feature branch
Make your changes
Run tests: cabal test
Submit a pull request

License

See the project LICENSE file for details.

Adding New Executable-Program Tools

2026-10-04T15:53:54Z

Generated from documentation/binary-tool.md, the repository is the canonical source and may be ahead of this page.

Adding New Executable-Program Tools

Executable-program tools (also called “bash tools”) are external programs that agents-exe can invoke as part of agent execution. These tools extend agent capabilities by performing side effects like reading files, installing packages, probing the network, or interacting with external APIs.

Tool Protocol Overview

Bash tools must adhere to a simple protocol with two commands:

1. The `describe` Command

When called with a single describe argument (no other parameters), the tool must return a JSON description of its interface:

$ ./my-tool describe
{
  "slug": "my_tool",
  "description": "What this tool does",
  "args": [
    {
      "name": "arg_name",
      "description": "Argument description",
      "type": "string",
      "backing_type": "string",
      "arity": "single",
      "mode": "dashdashspace"
    }
  ],
  "empty-result": { "tag": "AddMessage", "contents": "No results found" }
}

2. The `run` Command

When called with run followed by arguments, the tool executes its function and must write output to stdout:

$ ./my-tool run --arg-name value
Tool output here

JSON Description Format

Top-Level Fields

Field	Type	Required	Description
`slug`	String	Yes	Unique identifier for the tool (no spaces)
`description`	String	Yes	Human-readable description for the LLM
`args`	Array	Yes	List of argument definitions
`empty-result`	Object	No	Behavior when tool returns empty output

Argument Definition

Each argument in the args array has the following fields:

Field	Type	Required	Description
`name`	String	Yes	Argument name (used in JSON input)
`description`	String	Yes	Description for the LLM
`type`	String	Yes	Semantic type (e.g., "string", "number")
`backing_type`	String	Yes	Implementation type (e.g., "string")
`arity`	String	Yes	"single" or "optional"
`mode`	String	Yes	How to pass the argument (see below)

Argument Modes

The mode field determines how arguments are passed to the script:

Mode	Behavior	Example
`positional`	Added as consecutive arguments	`script run value`
`dashdashspace`	Added as `--name value` (two args)	`script run --name value`
`dashdashequal`	Added as `--name=value` (one arg)	`script run --name=value`
`stdin`	Concatenated to stdin with blank separators	`echo "value" \\| script run`

Empty Result Behavior

The optional empty-result field controls what happens when the tool produces no output:

// Do nothing (default behavior)
{ "tag": "DoNothing" }

// Add a message when output is empty
{ "tag": "AddMessage", "contents": "No results found" }

Session Context via Environment Variables

When a tool is executed within a session, agents-exe passes session context via environment variables. This allows tools to access session metadata without requiring the LLM to provide it as explicit arguments.

Available Environment Variables

Variable	Description	Example Value
`AGENT_SESSION_ID`	UUID of the current session	`550e8400-e29b-41d4-a716-446655440000`
`AGENT_CONVERSATION_ID`	UUID of the conversation	`6ba7b810-9dad-11d1-80b4-00c04fd430c8`
`AGENT_TURN_ID`	UUID of the current turn	`7c8b9d20-beef-22e2-91c5-11d15fe541d9`
`AGENT_AGENT_ID`	UUID of the executing agent (if available)	`a1b2c3d4-e5f6-7890-abcd-ef1234567890`
`AGENT_SESSION_JSON`	Full session serialized as JSON (when requested)	`{...}`

Use Cases for Context

Tools can use session context for:

Audit logging: Include session IDs in external API calls for traceability
Context-aware processing: Access conversation history to make smarter decisions
Correlation: Tag external telemetry with session identifiers
Agent-specific behavior: Adjust behavior based on which agent is executing the tool

Example: Tool with Session Context

#!/bin/bash

if [ "$1" == "describe" ]; then
    cat <<'EOF'
{
  "slug": "session-aware-logger",
  "description": "Logs operations with session context for audit trails",
  "args": [
    {
      "name": "operation",
      "description": "The operation being logged",
      "type": "string",
      "backing_type": "string",
      "arity": "single",
      "mode": "dashdashspace"
    }
  ]
}
EOF
    exit 0
fi

# Access session context from environment (with defaults)
SESSION_ID="${AGENT_SESSION_ID:-unknown}"
CONVERSATION_ID="${AGENT_CONVERSATION_ID:-unknown}"
TURN_ID="${AGENT_TURN_ID:-unknown}"
AGENT_ID="${AGENT_AGENT_ID:-unknown}"

# Parse the operation argument (skipping 'run')
OPERATION=""
while [[ $# -gt 0 ]]; do
    case $1 in
        --operation)
            OPERATION="$2"
            shift 2
            ;;
        *)
            shift
            ;;
    esac
done

# Log with context
echo "[session=$SESSION_ID turn=$TURN_ID] Executing: $OPERATION"

# Optionally access full session JSON for complex operations
if [ -n "$AGENT_SESSION_JSON" ]; then
    # Parse session JSON for additional context
    TURN_COUNT=$(echo "$AGENT_SESSION_JSON" | jq '.turns | length')
    echo "[session=$SESSION_ID] Turn $TURN_COUNT"
fi

Note on Session JSON

The AGENT_SESSION_JSON variable is only set when the full session context is available and requested. It contains the complete serialized session, which can be large. Tools should:

Check if the variable is set before using it
Handle potentially large JSON payloads efficiently
Consider using jq or similar for parsing

Complete Example Tool

Here’s a complete example of a file-reading tool:

#!/bin/bash
# File: tools/read-file.sh

if [ "$1" == "describe" ]; then
    cat <<'EOF'
{
  "slug": "read_file",
  "description": "Reads the contents of a file at the specified path",
  "args": [
    {
      "name": "path",
      "description": "Absolute or relative path to the file to read",
      "type": "string",
      "backing_type": "string",
      "arity": "single",
      "mode": "dashdashspace"
    }
  ],
  "empty-result": { "tag": "AddMessage", "contents": "File is empty or could not be read" }
}
EOF
    exit 0
fi

# Extract the path argument
FILE_PATH=""
while [[ $# -gt 0 ]]; do
    case $1 in
        --path)
            FILE_PATH="$2"
            shift 2
            ;;
        *)
            shift
            ;;
    esac
done

if [ -z "$FILE_PATH" ]; then
    echo "Error: No path specified" >&2
    exit 1
fi

if [ ! -f "$FILE_PATH" ]; then
    echo "Error: File not found: $FILE_PATH" >&2
    exit 1
fi

cat "$FILE_PATH"

Tool Discovery

Tools are discovered from the tools directory relative to your agent.json file:

project/
├── agent.json          # Agent definition
└── tools/              # Tool directory
    ├── read-file.sh    # Executable tool script
    ├── git-helper      # Another tool
    └── ...

agents-exe:

Lists all executable files in the tools directory
Calls each with describe to load its interface
Makes successfully loaded tools available to the agent

Using `agents-exe` as a Tool

agents-exe itself adheres to the same protocol, allowing you to nest agents:

# This allows agents-exe to be used as a tool by another agents-exe instance
agents-exe describe
agents-exe run --prompt "Hello"

This is useful for:

Creating hierarchies of agents with different access rights
Running agents across container boundaries
Delegating to specialized sub-agents

Best Practices

Always validate arguments - Check for missing or invalid arguments before processing
Write to stdout - The LLM receives stdout; use stderr for errors
Return non-zero on errors - Use appropriate exit codes
Keep descriptions concise - LLMs work better with clear, brief descriptions
Use appropriate modes - Choose argument modes that make sense for your tool’s interface
Handle empty results gracefully - Consider using empty-result to provide helpful messages
Document context usage - If your tool uses environment variables, document it for users
Provide sensible defaults - Use bash parameter expansion for optional context variables: ${AGENT_SESSION_ID:-unknown}

Debugging Tools

Test your tool independently before using it with agents-exe:

# Test the describe command
./my-tool describe | jq .

# Test the run command
./my-tool run --arg-name value

# Verify exit codes
echo $?  # Should be 0 on success, non-zero on error

Use agents-exe check to validate all tools in your agent configuration.

Inspecting Tool Schemas

The check command supports an optional --tools flag that allows you to inspect how tools are serialized for LLM calls. This is useful for debugging tool definition issues:

# Default behavior - just validate and show tool counts
agents-exe check

# List tools with their names and descriptions
agents-exe check --tools=list

# Show the internal agents-exe tool representation as JSON (wrapped in markdown details tags)
agents-exe check --tools=agents-exe

# Show the OpenAI LLM tool representation as JSON (wrapped in markdown details tags)
agents-exe check --tools=openai

The --tools option accepts the following modes:

Mode	Description
`none`	Default. Only show agent names and tool counts.
`list`	Print a markdown list of tool names and descriptions.
`agents-exe`	Output the internal tool representation as JSON within `<details>` tags.
`openai`	Output the OpenAI LLM tool schema (as sent to the API) within `<details>` tags.

This feature is particularly useful when:

Debugging why a tool isn’t being called by the LLM
Verifying that tool schemas are correctly formed
Documenting available tools for users
Comparing internal vs. LLM-facing representations

SQQ Agent Flow Documentation

2026-10-04T15:53:54Z

Generated from documentation/sqq-flow.md, the repository is the canonical source and may be ahead of this page.

SQQ Agent Flow Documentation

Overview

The sqq-agent.sh is a task queue automation system that manages AI agent execution on Git repositories. It integrates with GitHub issues, uses git worktrees for isolated task execution, and automatically creates pull requests with the results.

Architecture Components

Core Configuration

Variable	Purpose
`SQQ_BIN`	Path to the `sqq` queue management binary
`QUEUE_DB`	SQLite database path for task queue storage (`tasks/queue.sql`)
`TASK_DIR`	Directory for instruction files (`tasks/`)
`SESSIONS_DIR`	Directory for agent session outputs (`tasks-sessions/`)
`BASE_BRANCH`	Base git branch for new worktrees (`main`)
`GITHUB_USERNAME`	GitHub username to filter issues by author (security filter)
`PROJECT_MAP`	Maps labels to relative paths in worktree
`AGENT_MAP`	Maps labels to agent configuration files

Data Flow

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│  GitHub Issues  │────▶│  Task Files     │────▶│  Queue (sqq)    │
│  (with labels)  │     │  (.md files)    │     │  (SQLite)       │
└─────────────────┘     └─────────────────┘     └─────────────────┘
         │                                               │
         │                                               ▼
         │                                      ┌─────────────────┐
         │                                      │  Process Loop   │
         │                                      └─────────────────┘
         │                                               │
         ▼                                               ▼
┌─────────────────────────────────────────────────────────────────┐
│                        WORKTREE EXECUTION                         │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────┐  │
│  │ Setup        │─▶│ Prepare      │─▶│ Run Agent            │  │
│  │ Worktree     │  │ Environment  │  │ (agents-exe)         │  │
│  └──────────────┘  └──────────────┘  └──────────────────────┘  │
│                                              │                  │
│  ┌──────────────┐  ┌──────────────┐         │                  │
│  │ Create PR    │◀─│ Push Branch  │◀────────┘                  │
│  └──────────────┘  └──────────────┘                            │
└─────────────────────────────────────────────────────────────────┘

Command Flows

1. Initialization (`init`)

sqq-agent.sh init

Creates TASK_DIR and SESSIONS_DIR directories
Initializes SQLite queue database via sqq init --queue

2. Manual Task Addition (`add`)

sqq-agent.sh add <label> <branch>

Validates the project label exists in PROJECT_MAP
Finds or creates a numbered task file (format: NNNN-<branch>.md)
Opens $EDITOR for user to write instructions
Enqueues job: sqq-agent.sh worktree_exec "<label>" "<branch>" "<taskfile>"

3. GitHub Issue Import (`from_github`)

sqq-agent.sh from_github

Queries GitHub for issues labeled agents/to-be-taken from the configured author only
For each issue:
- Matches project label from issue labels
- Creates task file with issue content (title, body, comments)
- Adds note to mention issue number in commit
- Enqueues worktree_exec job
- Updates issue labels: removes agents/to-be-taken, adds agents/taken

Security Note: The GITHUB_USERNAME filter ensures that only issues created by the trusted author are processed. This prevents arbitrary code execution from malicious issues filed by third parties on public repositories.

4. Queue Processing (`process`)

sqq-agent.sh process

Runs sqq process --action Exec in a loop
On failure: waits 60 seconds and retries
Continues until queue is empty or manually stopped

5. Worktree Execution (`worktree_exec`) - Core Engine

sqq-agent.sh worktree_exec <label> <name> <instruction_file>

Phase 1: Worktree Setup

Fetch latest origin/BASE_BRANCH
Remove existing worktree (if any) and prune
Create new git worktree at directory <name>

Change to project directory based on PROJECT_MAP[<label>]

Phase 3: Environment Preparation

Execute ./git-agent-task.sh prepare if present
This allows per-project dependency installation or setup

Phase 4: Agent Execution

Check agent configuration with agents-exe --agent-file check

Run agent with instruction file:

agents-exe --agent-file <config> run \
    --session-file <session>.json \
    -f <instruction_file>

Capture agent output as commit message
Export session to markdown: agents-exe session-print

Phase 5: Commit and PR

Create and checkout new branch <name>
If changes detected:
- Stage all changes: git add -A
- Commit with agent output as message (no verify)
- Push to origin
- Create Pull Request via gh pr create with label agents/agent-pr
For non-GitHub tasks: optionally run ./git-agent-task.sh preview

6. Automatic Merging (`merge-pr`)

sqq-agent.sh merge-pr

Identifies the default branch of the repository.
Lists all Pull Requests with the label agents/agent-pr.
For each PR:
- If the base branch is not the default branch, it triggers an automatic merge (gh pr merge --auto).
- If the base branch is the default branch, it skips the PR to ensure manual review for main branch changes.

File Naming Conventions

Task Files

Format: TASK_DIR/NNNN-<branch>.md
Sequential numbering starting from 0001
Preserves existing files when re-queueing same branch

Session Files

JSON: SESSIONS_DIR/<name>.session.json (agent state)
Markdown: SESSIONS_DIR/<name>.session.md (human-readable)

Error Handling and Retries

Queue processing loops indefinitely with 60s delay on failure
Failed worktrees are force-removed before retry
Agent execution errors don’t prevent commit/push if changes exist
Missing git-agent-task.sh generates warnings but continues

Integration Points

Tool	Purpose
`sqq`	Task queue management (enqueue/dequeue/process)
`gh`	GitHub CLI for issue/PR management
`agents-exe`	AI agent execution engine
`git worktree`	Isolated working directories per task

Security Considerations

Uses --no-verify on git commits to bypass hooks in worktrees
Force-removes worktrees (potential data loss if in-progress)
Executes ./git-agent-task.sh without sandboxing
Critical: GITHUB_USERNAME environment variable filters GitHub issues by author to prevent processing malicious issues from third parties on public repositories. Default is lucasdicioccio.

AI Assistant Guidelines - Agents Framework

2026-10-04T15:53:54Z

Generated from documentation/ai-assistant-guidelines.md, the repository is the canonical source and may be ahead of this page.

AI Assistant Guidelines - Agents Framework

This document provides guidance for AI assistants (including future instances of myself) working on the Agents framework codebase.

Quick Start for AI Assistants

First Steps

When invoked on this codebase, always:

Check the knowledge base - Query sqlite_knowledge_store_query for existing state
Review recent commits - Use bash_get_git_log to see what’s changed
Check existing docs - Review docs_index table for current documentation
Identify the task - Understand what changes or documentation are needed

Essential Queries

-- Get project overview
SELECT * FROM project_overview;

-- List all documented modules
SELECT * FROM code_index WHERE documented_in IS NOT NULL;

-- Find undocumented modules
SELECT filepath, module_name FROM code_index WHERE documented_in IS NULL;

-- Check existing documentation
SELECT * FROM docs_index;

Knowledge Base Structure

Tables

Table	Purpose
`commits_seen`	Git commits already processed
`project_overview`	High-level project information
`code_index`	All source modules with metadata
`docs_index`	Documentation files index

Code Index Schema

-- Track which modules are documented where
SELECT 
    filepath,
    module_name,
    purpose,
    documented_in,
    last_updated
FROM code_index
ORDER BY filepath;

Documentation Index Schema

-- Check documentation coverage
SELECT 
    doc_path,
    title,
    description,
    related_modules
FROM docs_index
ORDER BY doc_path;

Documentation Standards

File Organization

documentation/
├── README.md                 # Project overview (always up to date)
├── architecture.md           # System architecture
├── tools.md                  # Tool system
├── mcp.md                    # MCP protocol
├── sessions.md               # Session management
├── tui.md                    # Terminal UI
├── cli-commands.md           # CLI reference
├── export-import.md          # Tool sharing
├── file-loader.md            # File loading utilities
└── ai-assistant-guidelines.md # This file

Documentation Format

All docs should:

Use Markdown format
Include ASCII diagrams for architecture (Graphviz when complex)
Provide Haskell type definitions for key types
Include code examples
Have a Table of Contents for longer docs
Reference related modules in System.Agents.* namespace

Graphviz Guidelines

When creating diagrams:

Save dot files to documentation/*.dot
Generate PNGs with matching names
Keep diagrams focused on one concept
Use consistent styling

// documentation/example.dot
digraph Architecture {
    rankdir=TB;
    node [shape=box, style=rounded];
    
    Main -> AgentTree;
    AgentTree -> Runtime;
    Runtime -> Session;
    Runtime -> Tools;
}

Working with the Codebase

Module Categories

Category	Path Pattern	Doc Reference
Core	`System.Agents.Base`, `System.Agents.Runtime*`	architecture.md
CLI	`app/Main.hs`, `System.Agents.CLI.*`	cli-commands.md
Tools	`System.Agents.Tools.*`	tools.md
MCP	`System.Agents.MCP.*`	mcp.md
Sessions	`System.Agents.Session.*`	sessions.md
TUI	`System.Agents.TUI.*`	tui.md
Export/Import	`System.Agents.ExportImport.*`	export-import.md
File Loading	`System.Agents.FileLoader*`	file-loader.md

Key Types to Document

When encountering new types, always document:

Purpose - What problem does this solve?
Fields - What does each field represent?
Relationships - How does it connect to other types?
Usage examples - How is it used in practice?

Tracing Conventions

The codebase uses Prod.Tracer extensively:

-- All significant operations should be traced
data Trace
    = OperationStart Param
    | OperationComplete Result
    | OperationError Error

-- Use contramap for sub-tracers
subTracer :: Tracer IO ParentTrace -> Tracer IO ChildTrace
subTracer = contramap ParentConstructor

Maintenance Tasks

Keeping Documentation Current

When code changes:

Identify affected docs:

SELECT doc_path FROM docs_index 
WHERE related_modules LIKE '%ModuleName%';

Update code_index:

UPDATE code_index 
SET documented_in = 'documentation/file.md',
    last_updated = datetime('now')
WHERE module_name = 'System.Agents.Module';

Review and update the relevant doc file

Adding New Modules

When new modules are added:

Add to code_index:

INSERT INTO code_index (filepath, module_name, purpose, last_updated)
VALUES ('src/System/Agents/NewModule.hs', 
        'System.Agents.NewModule',
        'Description of purpose',
        datetime('now'));

Determine if new documentation needed
Update existing docs with cross-references

Handling Commits

Track commits in commits_seen:

-- After processing a commit
INSERT INTO commits_seen (commit_hash, commit_date, commit_message)
VALUES ('abc123...', '2024-01-15', 'message');

Common Patterns

Agent Configuration

Agents are configured via JSON:

{
  "slug": "agent-name",
  "apiKeyId": "key-ref",
  "flavor": "openai",
  "modelUrl": "https://api.openai.com/v1",
  "modelName": "gpt-4",
  "announce": "Description",
  "systemPrompt": ["Instructions"],
  "toolDirectory": "tools",
  "mcpServers": [...],
  "extraAgents": [...]
}

Tool Definition

Tools follow this pattern:

data ToolRegistration = ToolRegistration
    { toolName :: Text
    , toolDescription :: Text
    , toolParameters :: Value  -- JSON Schema
    , toolExecutor :: Value -> IO ToolResult
    }

Session Flow

User Input -> Session -> LLM Call -> Tool Execution -> Response -> Persist Session

Documentation Gaps to Watch For

Watch for these common omissions:

New CLI commands - Update cli-commands.md
New tool types - Update tools.md
API changes - Document breaking changes
Configuration options - Keep examples current
Error types - Document error conditions

Questions to Ask

When documentation is unclear:

What is the purpose of this module/component?
How does it relate to other parts of the system?
What are the key data types?
What is the typical usage flow?
Are there any gotchas or edge cases?

Iterative Documentation Workflow

For large documentation tasks:

Plan - List all modules/files to document
Chunk - Work on one subsystem at a time
Query - Check knowledge base for existing state
Write - Create/update markdown files
Index - Update docs_index and code_index
Review - Check for consistency and completeness
Commit - First line of response should be summary

Summary Line Format

The first line of every response should be a concise summary:

Brief description of what was done or discovered

Examples:

“Updated tool system documentation with OpenAPI toolbox details”
“Discovered undocumented MCP server configuration types”
“Created architecture diagram for agent tree system”

Self-Correction Checklist

Before completing work:

[ ] Knowledge base tables are consistent
[ ] All new modules indexed in code_index
[ ] Documentation links are valid
[ ] Code examples compile (if applicable)
[ ] ASCII diagrams render correctly
[ ] Summary line is present
[ ] Related modules cross-referenced

Emergency Recovery

If knowledge base is corrupted:

Rebuild code_index by scanning all .hs files
Rebuild docs_index by listing documentation/*.md files
Update project_overview with basic project info
Re-link modules to documentation manually

Useful One-Liners

# Count modules by subsystem
ghc -i src -e ":browse System.Agents.Tools" 2>/dev/null | wc -l

# Find undocumented exports
find src -name "*.hs" -exec grep -l "^module " {} \; | while read f; do
    mod=$(grep "^module " "$f" | head -1 | awk '{print $2}')
    # Check if in code_index...
done

# List all types in a module
ghc -i src -e ":browse System.Agents.Base" 2>/dev/null | grep "data "

Contact Points

Key files for understanding the system:

File	Why It's Important
`app/Main.hs`	Entry point, CLI commands
`src/System/Agents/Base.hs`	Core types
`src/System/Agents/AgentTree.hs`	Multi-agent orchestration
`src/System/Agents/Runtime/Runtime.hs`	Execution engine
`agents.cabal`	Dependencies and build config

Remember: This documentation is for AI assistants. Keep it practical, specific to this codebase, and actionable.

Asynchronous Tool Calls

2026-10-04T15:53:54Z

Generated from documentation/async-tool-calls.md, the repository is the canonical source and may be ahead of this page.

Asynchronous Tool Calls

By default an agent runs every tool call of a turn one after another and only answers the LLM once all of them are done. In asynchronous mode, calls run concurrently in the background, the LLM can be handed a partial answer while they run, and it can ask about, wait for, or cancel them.

This is useful when a turn issues several slow calls (builds, test runs, HTTP requests), and when you want the model (or the user) to keep working while a long call finishes.

For deferred calls completed by an external worker (rather than in this process), see durable-workflows-howto.md.

Enabling it

Async execution is configured per agent in the agent JSON:

{
  "slug": "my-agent",
  "executionMode": "asynchronous",
  "asyncYieldStrategy": "yieldOnAnyProgress",
  "maxConcurrency": 4,
  "asyncCallTimeoutSeconds": 900,
  "toolCallPolicyConfig": {
    "default": {"tag": "runSync"},
    "rules": [
      {"tool": "build_project", "disposition": {"tag": "runAsync"}},
      {"tool": "run_tests", "disposition": {"tag": "runAsync"}}
    ]
  }
}

Field	Meaning	Default
`executionMode`	`synchronous` or `asynchronous`	`synchronous`
`toolCallPolicyConfig`	Which tools run in the background (`runAsync`), inline (`runSync`), isolated (`runIsolated`), or deferred to an external worker (`defer`)	everything `runSync`
`asyncYieldStrategy`	When a step hands control back to the LLM while calls run	`yieldWhenAllDone`
`maxConcurrency`	Calls running at once for this agent	4
`asyncCallTimeoutSeconds`	Give up on a call that runs this long	no limit

Yield strategies:

Value	Behaviour
`yieldWhenAllDone`	Wait for every call of the turn (like synchronous mode, but concurrent)
`yieldOnAnyProgress`	Answer the LLM as soon as one call finishes
`{"tag": "yieldOnTimeout", "milliseconds": 500}`	Answer after the given delay, finished or not

Both executionMode and toolCallPolicyConfig are needed: the mode enables the machinery, the policy decides which calls actually go to the background.

The settings apply everywhere an agent runs: run (one-shot), the TUI, the session commands, and agents called as tools by other agents.

What the LLM sees

A call that has not finished when the LLM is asked for its next completion gets a placeholder tool message instead of a result:

{"status": "running",
 "message": "This tool call has not finished yet. Its result will be delivered in a later message. Use get-tool-call-status with this tool_call_id to inspect it, or cancel-tool-call to stop it.",
 "tool_call_id": "call_abc"}

Every tool call always gets exactly one tool message, as OpenAI-compatible APIs require.

When the call finishes, its result is delivered in the next user message, once, as a notice:

Background tool calls finished since their placeholder responses:

tool_call_id call_abc (run_tests) completed:
42 tests passed

If the LLM ends its turn while calls are still running, the session does not stop: it waits for the results and sends them.

Tool-call capabilities

Enable these in the System Toolbox so the model can manage its own background calls:

{"builtinToolboxes": [
  {"tag": "SystemToolbox",
   "contents": {
     "Name": "system",
     "Description": "System context and background tool calls",
     "Capabilities": ["get-tool-call-status", "list-running-tool-calls", "cancel-tool-call"]
   }}
]}

Capability	Use
`list-running-tool-calls`	The calls still running in this session
`get-tool-call-status`	Status, progress and (once final) result of a call; can block until it finishes with `wait_for_completion`
`cancel-tool-call`	Interrupt a running call

Calls are addressed by the tool_call_id the model itself used (e.g. call_abc); the internal UUID also works.

get-tool-call-status reports orphaned for a call that was running when its process went away (e.g. the session was saved and reloaded elsewhere). Such a call can never finish, so the stepper resolves it as failed rather than waiting for it.

Progress

Tools that run in the background can report progress, which the model reads through get-tool-call-status and the TUI shows next to the running call.

Bash tools report their latest output line, at most twice a second: {"stream": "stdout", "line": "compiling module 12/40", "lines": 12, "bytes": 480}.
Agents called as tools report each step of the sub-agent: {"message": "sub-agent calling read_file", "turns": 7}. This applies to a prompt_agent_<slug> call that still runs in-tool (narrowed with bindings/with/as, or outside the runner). Under the runner, a sub-agent call with no narrowing runs as its own session instead (todos/os-as-standalone-server.md, Phase 5): its progress is that session’s own session.updated/text.delta/tool.* events, not this per-step callback.
Your own tools can report anything JSON through ctxProgressCallback in the ToolExecutionContext.

MCP and OpenAPI tools do not report progress yet. An OpenAPI call waits for one complete HTTP response, so there is nothing to stream. MCP does define progress notifications (ProgressNotification in System.Agents.MCP.Base), but the client currently ignores them; forwarding them to ctxProgressCallback would make MCP tools report progress like bash tools.

The 50 most recent progress entries per call are kept.

Cancelling

cancel-tool-call interrupts the call’s thread. A bash tool runs its script in its own process group, so cancelling kills the script and whatever it started. The same happens when a call exceeds asyncCallTimeoutSeconds.

Background calls are cancelled when the agent that owns them finishes or fails, so nothing keeps running (or keeps a subprocess alive) after a run.

In the TUI

Running calls appear under the turn that issued them, with their latest progress:

[Partial] > run the tests and the linter
  ✓ lint (call_1): completed
  ⏳ run_tests (call_2): running: 128 tests passed

A line above the conversation lists what is still running. While background calls run and the LLM has nothing to do, the TUI accepts input, so you can keep talking; whichever comes first — your message or the results — is sent to the model.

In one-shot runs

agents-exe run waits for background calls: they cannot outlive the process.

It stops early only when a turn waits on deferred calls, which an external worker has to complete. It then prints a JSON report and stores the session so it can be continued:

{"status": "paused",
 "reason": "waiting for deferred tool calls",
 "session_id": "…",
 "deferred_calls": [{"tool": "approve_deploy", "tool_call_id": "call_9", "continuation_token": "…"}]}

Continue it with agents-exe session complete and agents-exe session resume (see durable-workflows-howto.md).

Session files

Partial turns are stored in the session, with each call’s state. session-print shows them:

## ⏸️ Step 4: Partial Turn (in progress)
_(Some tool calls had not finished when this turn was sent)_

### ⏳ Tool Call Status

- `lint` (`call_1`): completed
- `run_tests` (`call_2`): running (placeholder sent)

Limits

Background calls live in the process that started them. A session reloaded elsewhere reports them as orphaned.
A pause/resume in the same process keeps the running calls only if the agent was built with withAsyncEngine; runAsync does not hand the engine back.
If the model reads a result with get-tool-call-status, the delivery notice repeats it once.
maxConcurrency is per agent. To cap several agents together, share one limit built with newAsyncConcurrencyLimit between their engines.

Plan: Durable Workflows for agents-exe

2026-10-04T15:53:54Z

Generated from todos/durable-workflows.md, the repository is the canonical source and may be ahead of this page.

Plan: Durable Workflows for agents-exe

See todos/durable-workflows.progress.md for implementation progress.

Goal

Turn agents-exe into a system suitable for:

Durable executions: sessions can be stored in and resumed from databases.
Isolated deployments: tool calls can run inside Docker or as local processes, transparently.
Transparent synchronous/asynchronous tool calls: the runtime can pause a turn after any subset of tool calls, yield to the outside world, and wake up later when results arrive.

Requested example flow:

user-turn 1: some question
agent-turn 1: reply with 3 tool calls
user-turn: perform 1 tool call, mark 2 as deferred
system: yields (possibly even reboots)
system: wakes up, executes deferred tool calls and places results, then wakes up again
user-turn: provides 3 tool results to agent
agent: continues

This document lists the primitives and architectural changes needed to support that flow.

Current state

The codebase already has scaffolding:

ExecutionMode (Synchronous / Asynchronous)
PartialUserTurn
ContinuationToken, AsyncToolResponse (ToolComplete / ToolYield)
ContinuationStore / ToolContinuation
ToolCache
OS persistence layer with SQLite/Postgres backends
File-based SessionStore

What is missing:

The async scheduler does not actually use ctxAsyncToolCall or AsyncToolResponse; it just runs tools synchronously and pauses after each one.
There is no per-call decision primitive for sync vs. deferred vs. isolated execution.
There is no “wake up and inject external results” primitive.
ContinuationStore has stubbed load/list implementations.
ToolExecutionContext contains non-serializable fields (portal, world, event queue), making durable continuations fragile.
Session storage is file-only.
There is no abstraction for running a tool call outside the current process (Docker / subprocess).

Phase 1 — Core primitives (types & interfaces)

1.1 Rich tool-call state inside a turn

Introduce an explicit state machine for every LLM-issued tool call within a user turn:

Ready      -> Running | Deferred | Completed
Running    -> Completed | Failed
Deferred   -> Ready | Completed   (via external wake)
Completed  -> terminal

Add a TrackedToolCall type wrapping LlmToolCall with:

tcId :: ToolCallId — stable ID for matching external results
tcState :: ToolCallState
tcResult :: Maybe UserToolResponse
tcContinuation :: Maybe ContinuationToken
tcPolicy :: AppliedPolicy — record of why it ran this way

Refactor PartialUserTurnContent to hold [TrackedToolCall] instead of separate pCompletedResponses, pPendingCalls, and pPendingContinuations lists.

1.2 Tool-call policy / decorator

Add a pure decision primitive:

data ToolCallDisposition
  = RunSync
  | RunAsync                  -- yield a continuation token
  | RunIsolated IsolationSpec -- run outside the current process
  | Defer Reason              -- intentionally pause (e.g., approval)
  | Decorate [Decorator] ToolCallDisposition

type ToolCallPolicy =
  ToolExecutionContext -> LlmToolCall -> ToolCallDisposition

This is the requested decoration primitive. It lets the runtime decide, per call, whether to execute immediately, defer, or run in isolation.

Provide defaultToolCallPolicy = const $ const RunSync for backward compatibility.

1.3 Pluggable executor interface

Decouple how a tool call runs from the session loop:

data ToolExecutor = ToolExecutor
  { execSync  :: ToolExecutionContext -> LlmToolCall -> IO UserToolResponse
  , execAsync :: ToolExecutionContext -> LlmToolCall -> IO AsyncToolResponse
  }

Concrete executors to provide:

inProcessExecutor — current behavior
yieldingExecutor — always returns ToolYield
cachingExecutor :: ToolCache -> ToolExecutor -> ToolExecutor
isolatedExecutor :: DeploymentRunner -> ToolExecutor

Replace the single Agent.ctxAsyncToolCall with an optional ToolExecutor selected by the policy.

Phase 2 — Async scheduler in the session loop

2.1 Replace the one-at-a-time async loop

Current runStepMAsync executes exactly one pending call per step and then pauses.

New scheduler behavior, given a set of TrackedToolCalls:

Classify each call via ToolCallPolicy.
Run all RunSync calls and collect results.
For RunAsync / Defer calls, generate continuation tokens and move them to Deferred.
Yield a PartialUserTurn containing:
- completed calls,
- deferred calls with tokens,
- any remaining ready calls.

This matches the requested flow:

perform 1 tool call, mark 2 as deferred → system yields → wakes up, executes deferred calls, places results → user-turn provides 3 tool results.

2.2 Continuation snapshot

When a call is deferred, generate a ContinuationToken and store a serializable ToolContinuationSnapshot:

token
sessionId
toolCallId
llmToolCall
cacheKey
policy / isolationSpec
serializable context fields: ctxSessionId, ctxConversationId, ctxTurnId, ctxCallStack, ctxAllowedTools, ctxParentConversation

Do not store the non-serializable fields (ctxToolPortal, ctxWorld, ctxEventQueue). Re-hydrate those on wake.

Phase 3 — Durable session storage

3.1 Generalize `SessionStore`

The current SessionStore is file-only. Introduce a backend interface:

data SessionBackend = SessionBackend
  { sbStore  :: SessionId -> Session -> IO ()
  , sbLoad   :: SessionId -> IO (Maybe Session)
  , sbList   :: IO [(SessionId, UTCTime)]
  , sbDelete :: SessionId -> IO ()
  }

Backends:

FileSessionStore FilePath
SqliteSessionStore Connection
CompositeSessionStore [SessionBackend] for read fallback with a single write target

3.2 Use OS persistence (optional)

The OS layer already has SqliteBackend, component tables, and migrations. Consider adding a SessionComponent persisted via persistComponent/loadComponent. This reuses ECS migrations and unifies backends.

If session storage stays separate, at least align IDs: SessionId/TurnId/ConversationId are UUID-based, and OS EntityId is also UUID-based, so conversion is trivial.

Phase 4 — Wake / resume API

4.1 Inject external results

wakeSession ::
     Session
  -> [(ContinuationToken, UserToolResponse)]
  -> IO Session

Behavior:

Find the PartialUserTurn and matching deferred TrackedToolCalls.
Move them to Completed with the provided result.
Update the cache if configured.
If all calls are complete, convert the PartialUserTurn to a full UserTurn.
Otherwise leave remaining deferred/ready calls in place.

4.2 Resume execution

resumeSession ::
     ConversationId
  -> Agent r
  -> Session
  -> IO (Either r Session)

If the latest turn is PartialUserTurn, run the scheduler on remaining ready/deferred calls.
If the latest turn is UserTurn, continue to the LLM step.
Returns Left r on completion, Right Session when it needs to yield again.

4.3 Complete continuations from external workers

completeContinuation ::
     ContinuationStore
  -> ToolCache
  -> ContinuationToken
  -> UserToolResponse
  -> IO Bool

This already exists as resumeAsyncToolCall but the ContinuationStore implementation is stubbed. Finish the load/list implementations and ensure proper JSON round-tripping.

Phase 5 — Isolated deployment primitives

5.1 `DeploymentRunner` abstraction

data DeploymentRunner = DeploymentRunner
  { drName    :: Text
  , drExecute :: IsolationSpec -> LlmToolCall -> IO (Either IsolationError UserToolResponse)
  }

Implementations:

localProcessRunner — fork a worker process
dockerRunner — run inside a container
functionRunner — for serverless/FaaS (future)

5.2 Serialization contract for isolated calls

Define a stable JSON envelope consumed by any external runner:

{
  "token": "...",
  "toolCall": { ... },
  "contextSnapshot": { ... },
  "policy": { "tag": "RunIsolated", "spec": { "image": "...", "sandbox": "..." } }
}

Result envelope:

{ "token": "...", "status": "success", "result": { ... } }

This makes Docker vs. local execution transparent to the agent.

5.3 Tool-call decorator for isolation

Example policy:

isolateBash :: ToolCallPolicy
isolateBash ctx call
  | toolName call == "bash_command" = RunIsolated (Docker "agents-exe/bash-runner:latest")
  | otherwise                       = RunSync

Phase 6 — Integration & agent combinators

6.1 Extend `Agent` record

Add to Agent r:

ctxToolCallPolicy :: ToolCallPolicy
ctxSessionBackend :: Maybe SessionBackend
ctxContinuationStore :: Maybe ContinuationStore
ctxDeploymentRunner :: Maybe DeploymentRunner

Keep defaults so existing agents compile unchanged.

6.2 New combinators

withToolCallPolicy    :: ToolCallPolicy -> Agent r -> Agent r
withSessionBackend    :: SessionBackend -> Agent r -> Agent r
withContinuationStore :: ContinuationStore -> Agent r -> Agent r
withDeploymentRunner  :: DeploymentRunner -> Agent r -> Agent r

6.3 Update `agentStoreSession`

System.Agents.Combinators.StoreSessionProgress should store via the configured SessionBackend rather than only files, while still supporting file fallback.

Phase 7 — CLI / operator API

New commands for real operation:

agents session pause <session-id> — yield after current step
agents session resume <session-id> — continue execution
agents session pending <session-id> — list deferred calls / continuation tokens
agents session complete <token> <result-file> — inject an external result
agents session run-isolated <session-id> — poll and execute RunIsolated calls

Phase 8 — Testing strategy

Add tests for:

Policy classification: a policy can mark some calls sync and some deferred.
Partial turn serialization: a session with deferred calls round-trips through SQLite/file backends.
Wake/resume: wakeSession injects results and eventually produces a full UserTurn.
Cache integration: deferred calls whose result is later cached are resolved on resume.
Isolation contract: a DeploymentRunner can execute a tool call via a subprocess and return a result envelope.
Determinism: the same session state resumed twice produces the same outcome.

Open questions

Should PartialUserTurnContent keep its current shape or be rewritten around TrackedToolCall?
Rewriting is cleaner but touches JSON serialization and existing tests.
Should session storage move into the OS ECS persistence layer or stay separate?
Moving into ECS unifies migrations/backends; keeping it separate is less invasive.
How do we handle non-serializable ToolExecutionContext fields on resume?
Recommended: store a serializable snapshot and re-hydrate portal/world/event queue in the runtime.
Do we want true concurrency for sync calls or sequential execution?
Sequential is simpler and matches current cache semantics; concurrency can be added later behind a policy flag.
Should ToolCallId be a UUID or an index into the turn?
A UUID is safer for external workers and database keys; an index is smaller and deterministic. Hybrid: ToolCallId = TurnId + Int index.

Suggested first milestone

The smallest vertical slice that proves the design:

Add TrackedToolCall and refactor PartialUserTurnContent.
Implement ToolCallPolicy with RunSync / Defer decisions.
Update the async scheduler to batch sync calls and yield deferred ones.
Implement wakeSession.
Add a SQLite-backed SessionBackend.
Write a test exercising the full requested flow:
- 3 tool calls issued,
- policy executes 1 and defers 2,
- session persists,
- wake injects 2 results,
- session resumes and produces a UserTurn with all 3 results.

This milestone delivers the requested primitives without yet building Docker runners or Postgres backends.

Spec: the OS as a standalone server (TUI and web UI as clients)

2026-10-04T15:53:54Z

Generated from todos/os-as-standalone-server.md, the repository is the canonical source and may be ahead of this page.

Spec: the OS as a standalone server (TUI and web UI as clients)

Status: proposal, 2026-09-23, revised the same day after checking service readiness. Phases 0 and 1 done on branch feature/os-standalone-server (commits 8de2a64..dc01dc8); All phases landed (8de2a64..771c988), with the open items listed under “Remaining after Phase 5”. Builds on todos/web-server-embedding.md (done) and todos/session-mailbox.md (done).

Goal

One long-running agents-exe process hosts the agents and their sessions. It loads agents files the way the TUI does today (agents-exe.cfg.json, agentsFiles, agentsDirectories, --agent-file, --agent). Clients attach to it and drive it:

the TUI, in the same process or over a socket, with no loss of features;
the web chat page, as today;
any program, through the HTTP API: create and message sessions, wake them (continuations, mail), interrupt them, cancel them, follow their events.

The end state: agents-exe serve runs standalone; agents-exe tui either embeds the runner (today’s single-process experience) or attaches to a running server with --attach URL; both TUI modes are the same code.

The OS picture

OS	here
kernel	`Host` + `Host.Runner` (`SessionRunner`)
process table	the `sessions` table, `srLive` for the resident ones
syscall interface	the runner's operations, serialized as `Command`
kernel log / netlink	the `SessionEvent` broadcast, serialized as `Event`
terminal	the TUI: a client that only renders and issues commands
`init`	`agents-exe serve`: config loading, then the runner

The ECS System.Agents.OS.* layer (World, components, OS.Persistence) is not the kernel in this picture. In production it only tracks async tool-call entities and subcall visibility for the TUI. The kernel’s state model is sessions, mail and continuations, and its source of truth is the persisted session.

Current state

Two runtimes exist and share only the agent tree loader and AgentFactory.buildAgent.

	TUI (`tui/`)	agents-server (`examples/agents-server`, `src/System/Agents/Host*`)
loop	`Session.Loop.runUntilBlocked`, one forked thread per conversation (`TUI/Event/Conversation.hs`, `spawnConversationIO`)	`Host.Runner.runLoop`, one run at a time per session, under a lock
agent build	`OneShot.nodeToAgent`, then the TUI rewrites `step` and `usrQuery`	`Runner.newAgent` (`buildAgent … RootAgent`, durable mailbox, router, spawn and watch hooks)
mail	`newInMemoryMailbox` + one process-local `MailRouter` in `Core`	`newDurableMailbox`, `session_mail` table, `serverMailRouter`
user input	a `BChan` per conversation plus a `TVar` of buffered messages	`postMessage`, which becomes `UserMessage` mail when the session is busy
pause	`Control Pause` mail since b62e2a6	`pauseSession` → `Control Pause`, `POST /v1/sessions/:id/pause`
sessions	file store, `conv.<uuid>.json`, `SessionStore.listSessions` once at startup	`SessionBackend` (SQLite or Postgres), versioned `sbCompareAndStore`, `sbQuery`
config	`agents-exe.cfg.json`, `agentsDirectories`, `--agent`, `--params-file` (all in `app/Main.hs`)	flags only: `--agent-file` (repeatable), `--db`, `--set`/`--pin`
events out	`TQueue OSEvent` bridged to Brick; a whole-`Session` progress callback; heartbeat polling of TVars	`SessionEvent` broadcast `TChan`, per-session `subscribe`, SSE
identity	`ConversationId` generated independently of `SessionId`	`ConversationId` derived from `SessionId`

What we can reuse as-is

Host.withHost / withHostStores: multi-file loading keyed by slug, plus agents stored in the database.
Host.Runner: createSession[As[WithParent]], spawnSession, postMessage (with nmInterrupt), resume, completeCall, cancelRun, cancelAttachedCalls, pauseSession, getSession, awaitRun, deleteSession, recoverOnStartup, subscribe/subscribeSTM.
The HTTP API and SSE stream (AgentsServer.Api), MCP over HTTP, auth, /openapi.json, the chat page.
Mail: Envelope, MailBody, Sender, Priority, ControlMsg all have JSON. Every client-facing thing is addressed by a UUID (SessionId, ContinuationToken, ToolCallId, MessageId).
Session and SessionMeta have JSON; the TUI’s usage, signals and trajectory views are pure functions over a Session, so they run on a client unchanged.

Gaps

G1. The TUI is the host, not a client. spawnConversationIO builds the agent, overrides step (pause polling, progress callback, Stop → AskUserPrompt rewrite) and usrQuery (blocks on the BChan), forks Loop.runUntilBlocked, and owns the World, the OSEvent queue and the MailRouter. Quit is killThread on every conversation. None of this goes through the runner.

G2. The runner needs a first message. createSession takes a NewMessage; the TUI creates an empty conversation and waits for input.

G3. Event shape. The TUI renders from full Session snapshots (the OnSessionProgress callback) and from OSEvent_Subcall* and OSEvent_ToolCallActivity. The runner emits SessionEvent (meta plus head turn, deferred calls, run start/stop, text deltas, tool start/stop) and AgentFactory sets ctxEventQueue = Nothing, so subcall progress and tool-call activity never reach the server or its clients. Nine of the fifteen OSEvent constructors are never emitted.

G4. No JSON for commands and events. SessionEvent, NewMessage, RunMode, RunnerError, OSEvent, ToolCallActivity, ToolCallPhase, SessionProgress, Outgoing, Receipt, SendError have no codecs. The only encoder is eventFrame in the server’s Api.hs; the only decoders are its request parsers.

G5. No replay, no global feed. Events carry no sequence number and are not persisted; a reconnecting client gets a fresh snapshot. Subscriptions are per SessionId; there is no owner-wide or server-wide feed, so a live session list cannot be kept up to date.

G6. Runner and HTTP operations the TUI relies on are missing:

TUI feature	today	runner / HTTP
fork at a turn (`handleForkAtTurn`)	local `Session` copy with `forkedFromSessionId`	none
continue a stored session with the currently selected agent	`handleRestoredConversation`	`smAgent` is fixed per session
edit / delete queued messages (`TUI/Event/Queue.hs`)	shared `TVar` of discrete messages read by the runtime	not needed: becomes a TUI-local draft (§5)
list sessions	`SessionStore.listSessions`	`sbQuery` directly on the backend, not a runner call
generic mail (send `AgentMessage`, `StopRun`, `CancelCalls [ids]`, `Resume`)	n/a	`sendControlMail` is internal; no route
complete a deferred call	not supported in the TUI	`completeCall`, `POST /v1/continuations/:token`
agent details (model, prompt, activation flags) for the Agents tab	live `OSAgentNode` TVars	`GET /v1/agents` has slug, description, tool names only

G7. Config parity. locateAgentsExeConfig, initArgParserArgs, resolveAgentFiles, --params-file handling and the default-directory fallback live in app/Main.hs and are not in a library. The server cannot load agents “like the TUI does”.

G8. Identity. The TUI’s ConversationId is independent of its SessionId (see the comment in spawnConversationIO); the server derives one from the other. AgentId is random per load; the slug is the stable name. There are two ConversationId types (Base and OS.Core.Types).

G9. Spawn semantics differ. The TUI’s spawn-session hook resolves a slug against the roster of root agents; the server resolves helpers of the calling node.

G10. Sub-agent runs are not sessions. A prompt_agent_* call runs its child inside the tool call, so the child produces no SessionEvent and cannot be cancelRun’d or subscribed to on its own. Only the TUI sees it, through OSEvent_Subcall*.

G11. Volatile per-session state. Secret params (lsParams), watches and the run Async are lost on restart or eviction. A client protocol needs a “resupply secrets” step, or durable encrypted storage.

G12. No CORS. checkOrigin in AgentsServer.Api refuses non-loopback Origins when auth is off and lets everything through when it is on, but the server never emits Access-Control-Allow-* headers and does not answer preflight OPTIONS. A browser page served by another local web app (a different port is a different origin) cannot call the API or open the SSE stream directly; it has to proxy through its own backend.

G13. Service packaging. The server itself is service-ready: SIGTERM and SIGINT trigger a graceful stop with --shutdown-grace, logs are JSON lines on stderr, every path is a flag, recoverOnStartup runs at boot. But there is no unit file, no install target, no bundling/ entry and no documentation for running it under systemd.

Integration modes, as of 2026-09-23

Mode	Status
server-side web app calls the API and renders its own chat	works
browser chat UI on another origin talks to the service directly	blocked by G12
web app proxies the API under its own origin	works
MCP client (`ask_<slug>` tools, blocking, no session continuity)	works, coarse
another agents-exe agent uses it as an OpenAPI toolbox	works
deferred tool calls executed by the web app's own workers	works
TUI attached from elsewhere	not started

Design

1. The protocol module

A library module, System.Agents.Protocol, owning two sum types with JSON instances. The HTTP layer, the SSE encoder, the in-process client and the socket client all speak these; nothing else defines a wire format.

data Command
    = CreateSession   { agent :: Text, message :: Maybe NewMessage, run :: Maybe RunMode, params :: Map ParamName Value, parent :: Maybe SessionId }
    | PostMessage     { session :: SessionId, message :: NewMessage, run :: Maybe RunMode, params :: Map ParamName Value }
    | Resume          { session :: SessionId, mode :: RunMode, params :: Map ParamName Value }
    | CompleteCall    { token :: ContinuationToken, result :: UserToolResponse, resume :: Bool, params :: Map ParamName Value }
    | CancelRun       SessionId
    | CancelAttached  SessionId
    | Pause           SessionId
    | SendMail        { session :: SessionId, body :: MailBody, priority :: Priority }   -- StopRun, CancelCalls, Resume, AgentMessage, …
    | ForkSession     { session :: SessionId, atTurn :: Maybe Int, agent :: Maybe Text }   -- newest-first index; turns have no id
    | ListSessions    SessionQuery
    | GetSession      SessionId
    | ListAgents
    | GetAgent        Text
    | DeleteSession   { session :: SessionId, mode :: DeleteMode }
    | Subscribe       { scope :: SubscribeScope, after :: Maybe EventSeq }

data Event = Event
    { evSeq     :: EventSeq          -- total order per server
    , evSession :: Maybe SessionId   -- Nothing for server-wide events
    , evBody    :: EventBody
    }

data EventBody
    = RunStarted RunMode | RunStopped SessionStatus | SessionFailed Text
    | SessionUpdated SessionMeta (Maybe Turn)
    | CallsDeferred [DeferredCallView]
    | TextDelta Text
    | ToolCallStarted … | ToolCallProgressed ToolCallActivity | ToolCallCompleted …
    | SubcallStarted { parent :: SessionId, child :: SessionId, slug :: Text, depth :: Int }
    | SubcallCompleted … | SubcallFailed …
    | SessionCreated SessionMeta | SessionDeleted SessionId
    | MailAccepted Envelope | MailRejected SendError
    | AgentsChanged

Notes:

NewMessage gains Maybe semantics through CreateSession.message: Nothing creates an idle session with no turn (G2). The runner stores it with status ready and no run.
ForkSession is new runner work (G6): copy the session up to atTurn, set forkedFromSessionId, optionally rebind smAgent. This also covers “continue with another agent”: a fork at the head with a new agent.
SendMail generalizes sendControlMail. It is how the TUI posts StopRun on quit instead of killThread, and how a client sends AgentMessage mail on behalf of another session.
OSEvent is retired as a public type. ToolCallActivity and the Subcall* payloads move into EventBody; the unused constructors are dropped. Session.Async.Engine and OneShotTool emit through a runner hook (ctxEmit :: Maybe (EventBody -> IO ())) instead of ctxEventQueue, and the runner sets it (G3). SubcallProgress no longer carries a whole Session: a child that is a real session (see §4) emits its own SessionUpdated.

2. Sequence numbers and replay (G5)

The runner stamps every event with a monotonically increasing EventSeq and keeps a bounded in-memory ring (say the last 4096 events).
Subscribe { after = Just n } replays from the ring when n is inside it. When it is not, the server answers with a snapshot per session in scope (as today) followed by live events, and the client re-fetches GetSession for anything it cares about. This is the SSE Last-Event-ID contract.
SubscribeScope = OneSession SessionId | Owner (Maybe Text) | All. All needs an admin owner when auth is on.
Persisting the event log is deferred (see Decisions). The stored session plus session_mail already lets a client rebuild state; the ring only needs to cover reconnection hiccups.

3. `RunnerClient`: one interface, two implementations

data RunnerClient = RunnerClient
    { rcCommand   :: Command -> IO (Either RunnerError Reply)
    , rcSubscribe :: SubscribeScope -> Maybe EventSeq -> IO (IO Event, IO ())   -- next, close
    }

inProcessClient :: SessionRunner -> RunnerClient
httpClient      :: BaseUrl -> Maybe BearerToken -> IO RunnerClient   -- HTTP + SSE
socketClient    :: FilePath -> IO RunnerClient                       -- Unix socket, same framing

The TUI depends only on RunnerClient. agents-exe tui builds inProcessClient over a runner it starts itself; agents-exe tui --attach builds httpClient or socketClient. No TUI code knows which.

4. The TUI as a client (G1, G6, G8, G9, G10)

Conversation in the TUI becomes: SessionId, the latest Session and SessionMeta (from GetSession and SessionUpdated), derived views (tool-call views, signals, usage), unread markers. No ThreadId, no BChan, no callbacks, no TuiAgent with TVars.
Core loses coreWorld, coreOSEventQueue, coreMailRouter, coreBufferedMessages, corePausedConversations. The agent list is ListAgents plus AgentsChanged.
Send = PostMessage when the session is idle. While a run is active, send appends to a TUI-local draft (§5) rather than posting a discrete queued message; the draft is posted as one message when the run stops. “Send now” posts the draft at once (it becomes mail, folded at the next R1); an interrupt (PostMessage with nmInterrupt) bypasses the draft as today. Unread mail from other senders is shown read-only next to it (see Decisions, D3).
Pause = Pause; unpause = SendMail Control Resume. Paused becomes a SessionStatus detail the server reports, not a TUI-local flag.
Interrupt and hard cancel are already commands (PostMessage with nmInterrupt, CancelAttached). Quit = SendMail StopRun for every session the TUI started in embedded mode, and nothing when attached.
Fork = ForkSession. Continue with another agent = fork at the head.
Subcall visibility: a prompt_agent_* call spawns a real child session via spawnSession (the server already links smParent), so the child shows up through SessionCreated + its own events. The TUI’s spawn-roster question (G9) goes away: the server resolves helpers, and CreateSession with a parent covers the “spawn any root as a child” case the TUI has today.
Deferred calls: the TUI gains a pending-calls view and CompleteCall, which the server already supports.
Sessions history tab = ListSessions, refreshed by SessionCreated, SessionUpdated, SessionDeleted.
ConversationId in the TUI = sessionIdToConversationId sid, as on the server (G8).

The OS.* ECS layer is left in place for tool-call entities inside the engine; the TUI stops constructing a World.

5. The draft buffer (G6, queued messages): a UI concern

Three messages typed in a row while the LLM is busy are almost always one message being elaborated. So the TUI keeps, per conversation, one editable draft with append semantics instead of a queue of discrete messages, and posts it whole when it is the user’s turn. The kernel never sees a draft: it only ever receives real messages (PostMessage) and interrupts.

Where it lives: in the TUI’s Conversation state only. It is not persisted and does not survive a detach; a second client attached to the same session does not see it. That is fine: it is unsent text, like the contents of the composer.
Editing: the queue tab becomes a draft editor. Collapsed, it shows the draft’s first line and size; expanded, it is a text editor over the whole draft. Sending while busy appends a paragraph; the user can rewrite or clear it at will.
When it ships: on RunStopped with a status that accepts input, the TUI posts the draft as one PostMessage and clears it. “Send now” from the editor posts it immediately; the server turns it into UserMessage mail and R1 folds it into the next user turn, which fires after each round of tool results too, so an elaboration typed during a tool call reaches the model at the next completion. A paused session keeps the draft until it is resumed.
What the LLM sees: unchanged from the mailbox spec. R1 emits one tool message per call, then a user message with the folded mail. For a provider that rejects a user message directly after tool results, the agent opts into mailInToolResult: true: R1 then appends the folded mail as a trailing text block of the last tool result of that round, with the same [mail …] header. This is a property of mail folding, not of drafts, and applies to attached (sync) calls; detached and deferred results already arrive as mail of their own.

6. `agents-exe serve` and config parity (G7)

Move from app/Main.hs into a library module (System.Agents.CLI.Config or similar): locateAgentsExeConfig, config parsing, agentsFiles plus agentsDirectories expansion, the default-directory fallback, resolveAgentFiles for --agent, --params-file, and the sessions-store choice.
HostConfig gains a constructor from that config, so withHost can be fed either explicit --agent-files or the resolved config.
agents-exe serve = the current agents-server main, with config loading in front. agents-server keeps working as the same code with flags only. The SQLite session backend becomes the default for the TUI too; the file store stays readable through a composite backend for old conv.*.json history.
--socket PATH on serve for a local, token-free attach; --bind/--port as today for HTTP.

7. HTTP surface additions

Route	Command
`POST /v1/sessions` with no `prompt`	`CreateSession { message = Nothing }`
`POST /v1/sessions/:id/fork`	`ForkSession`
`POST /v1/sessions/:id/pause`	`Pause`
`POST /v1/sessions/:id/mail`	`SendMail`
`GET /v1/sessions/:id/mail?unread=`	inbox listing
`GET /v1/events?scope=&after=`	`Subscribe` across sessions, with `Last-Event-ID`
`GET /v1/agents/:slug`	extended with model, system prompt, tool activation

GET /v1/sessions/:id/events stays, gains Last-Event-ID.

Phases

Phases 0 and 1 need nothing and make the service usable from a local web app on their own. Phase 2 needs nothing either. Phase 3 needs Phase 2. Phase 4 needs 1, 2 and 3. Phase 5 needs only Phase 2.

Phase 0: service readiness (G12, G13) — done

--cors-origin ORIGIN on agents-server, repeatable, * allowed only without auth: emits Access-Control-Allow-Origin (echoing the matched origin), -Allow-Methods, -Allow-Headers (Authorization, Content-Type), -Expose-Headers (Location), -Max-Age, and answers preflight OPTIONS with 204 before auth and before checkOrigin. An origin listed here passes checkOrigin even with auth off. The SSE route keeps working cross-origin through ?access_token= since EventSource cannot set headers. Docs: a “Running as a service” section in documentation/agents-server.md with a systemd unit, and the CORS flag in the flag table and the authentication section. Also add the undocumented cancel-attached, pause, interrupt, tool.started and tool.completed to the same doc. Tests: preflight, allowed and refused origins, SSE with access_token from an allowed origin.

Phase 1: `agents-exe serve` and config parity (G7) — done

Landed as System.Agents.CLI.ConfigLoader (not CLI.Config: the TUI library already owns that module name), hostConfigFromResolved, agents-exe serve reusing the global --agent-file/--agent/--params-file flags, and --socket. The Unix listener stops accepting on SIGTERM and unlinks its file but does not drain in-flight requests through --shutdown-grace; only the TCP listener does.

Original scope:

Config loading into the library (§6); serve subcommand on agents-exe; --socket. The agents-server executable stays as the flags-only entry point over the same code. Docs: documentation/agents-server.md gains the config section; documentation/cli-commands.md gains serve.

Phase 2: the protocol, in the library

Split in three sequential steps: 2a the Protocol types with JSON, event sequence numbers, the ring and replay, the cross-session feed, and the server re-implemented on them; 2b the missing runner operations (listSessions, sendMail, create with no message, forkSession) and their routes; 2c ctxEmit replacing ctxEventQueue so subcall and tool-activity events reach the runner stream, and mailInToolResult.

2a landed as System.Agents.Protocol (Event { seq, session_id, owner, body }, EventBody with the old SessionEvent kinds plus session.created / session.deleted, JSON for NewMessage, RunMode, RunnerError, DeleteMode, DeletionPlan, SubscribeScope), a 4096-event ring in the runner, subscribe/subscribeSTM taking a scope and an after, Last-Event-ID and ?after= on the session stream, and GET /v1/events?scope=owner|all. Deviations from the sketch: evSession and evOwner are always set, also on the server-wide events, because the JSON merges them with the embedded SessionMeta fields; RunnerError’s decoder is lossy (only the code survives, by design of the existing {error, message} shape); the snapshot fallback on an unavailable replay keeps the pre-existing tiny gap between snapshot and re-subscribe.

2b landed: createSessionAs* take Maybe NewMessage (promptless POST /v1/sessions creates an idle ready session), listSessions on the runner, sendMail/listMail with POST/GET /v1/sessions/:id/mail, and forkSession with POST /v1/sessions/:id/fork. Deviation: Turn has no id of its own, so forks are addressed by a 0-based newest-first turn index (at_turn), exactly as the TUI’s handleForkAtTurn does, and the error is UnknownTurn SessionId Int (unknown_turn). The Command sketch’s atTurn :: TurnId is wrong until turns get ids.

2c landed: ctxEmit :: Maybe (OSEmission -> IO ()) next to ctxEventQueue (an intermediate OSEmission type avoids a Session.Base → Protocol import cycle; the runner converts it to EventBody), the four kinds subcall.started / subcall.completed / subcall.failed / tool.progressed, the nine dead OSEvent constructors deleted, and mailInToolResult. Deviations: subcall events are stamped with the parent as session_id (so a subscriber of the parent sees them) and carry the child as child_session_id. Open gap: mailInToolResult is implemented in the synchronous step path only; the server always runs agents asynchronously, so it has no effect on server sessions until the async path (runStepMAsync) folds mail the same way. Tracked for Phase 5.

System.Agents.Protocol with Command, Event, EventBody, JSON instances, and codecs for NewMessage, RunMode, RunnerError, UserToolResponse views. AgentsServer.Api re-implemented on top of it (no behaviour change; eventFrame goes away). Runner gains EventSeq stamping and the ring, listSessions, sendMail, createSession with no message, forkSession. ctxEmit replaces ctxEventQueue; Engine and OneShotTool emit through it. mailInToolResult folding at R1 (§5). Tests: JSON round trips, replay from the ring, fork, empty create, mailInToolResult folding.

Phase 3: the TUI on the in-process runner

Split in three sequential steps: 3a RunnerClient and inProcessClient in the library plus a serializable agent descriptor (ListAgents / GetAgent with model, prompt and tool activation, which also fixes the GET /v1/agents gap); 3b the TUI’s startup, Agents tab, History tab and conversation layer rewritten on RunnerClient (the large step); 3c subcall visibility and tool-call activity from runner events, the pending-calls view with CompleteCall, and retiring OSEvent, ctxEventQueue and the TUI’s event bridge.

3a landed (a78d280..bb4e003): AgentDescriptor (model, prompt, tools with Activation, parameters, helpers) served by GET /v1/agents; Command and Reply; System.Agents.Host.Client with RunnerClient, Subscription, inProcessClient owner runner and one typed helper per operation. The owner is the client’s identity, not a Command field, so a command cannot spoof another owner.

3b is itself split in four, because Brick’s single TuiState/AppEvent means old and new conversation models cannot coexist: 3b-i the new Conversation/Core/AppEvent types and the startup on Host + SessionRunner + inProcessClient, with handlers stubbed so the shape compiles; 3b-ii the event bridge (subscribeAll → AppEvent) and the command dispatch; 3b-iii the draft buffer and the History tab; 3b-iv render and keymap cleanup, unit tests, smoke test. Known regression to accept until Phase 5: AppEvent_SubcallProgress carried the child’s whole Session; the runner stream has no equivalent, so a subcall shows start, completion and failure only.

3b landed (f73c018..d2887c6). The TUI is a RunnerClient over an in-process runner: promptless createSession then postMessage; one subscribeAll bridge thread feeding Brick; pause/resume, interrupt, hard cancel, fork, restore (fork when the selected agent differs), F5 and quit (StopRun mail, no killThread) all go through the client; the draft buffer replaces the queue (Ctrl+A edit, Ctrl+G send now, Ctrl+D clear; “edit” pulls the draft back into the composer); History refreshes from events with a per-session cache and Loading/error states; HostConfig.hcLegacySessionDirs composites old conv.*.json directories as a read fallback for both the TUI and serve. Runner fix along the way: postMessage accepts the first message of an empty ready session. An end-to-end pty script against a fake OpenAI endpoint lives in checks/phase3b-iv-e2e/.

3c landed (c5f017f, 5eeb4b0): OSEvent and ctxEventQueue are gone; OSEmission + ctxEmit is the one emission mechanism, with queueEmitter for local consumers; new event kind hook.failed; the per-step subcall progress snapshot is gone with no successor until Phase 5. The TUI has a Pending panel and answer-pending (Ctrl+Y) that completes the oldest deferred call through completeCall with auto-resume. UserToolResponse has no error form, so there is no fail-pending. Open: no selection among several pending calls; no UI for hook.failed.

inProcessClient; the TUI conversation layer rewritten on RunnerClient (§4). agents-exe tui starts a Host + SessionRunner over the SQLite backend chosen from config (Phase 1), with the file store composited in for history. Feature parity checklist: new, continue, fork, send, draft editor (§5), pause, interrupt, hard cancel, quit, subcall visibility, tool-call activity, F5 tools refresh, export, attachments, plus the new pending-calls view. Remove coreWorld, coreOSEventQueue, coreMailRouter, coreBufferedMessages, corePausedConversations, Loop usage, OneShot.nodeToAgent usage in the TUI.

Phase 4: attach — done

Landed (20522f2..4e03ec9): System.Agents.Host.Client.Http.httpClient over http(s):// and unix:// URLs with bearer token, SSE parsing, transparent reconnect with Last-Event-ID (backoff 100 ms to 5 s, 45 s stall timeout, duplicates dropped), AllSessions falling back to the owner scope on 403; agents-exe tui --attach URL|PATH with --token / --token-file; POST /v1/sessions takes parent; GET /v1/sessions/:id takes wait/timeout; streams send an Agents-Replay header and flush headers at once (a warp header-buffering bug had hung the attached TUI); the chat page follows GET /v1/events. Open: with --attach, app/Main.hs still resolves local agent files, so --agent SLUG can fail when the slug is unknown locally.

Original scope:

httpClient (HTTP + SSE with Last-Event-ID) and socketClient; agents-exe tui --attach URL|PATH. The chat page moves to GET /v1/events for its session list. Tests: a TUI-less client test that runs the server, attaches, creates, interrupts, forks and replays after a dropped stream.

Phase 5: sub-agents as sessions — done, with two documented gaps

Landed (d7850d0..771c988): ctxRunSubagent hook, installed by Runner.newAgent, runs prompt_agent_* as a real child session (createSessionForNode, parent = the calling session, helper resolved by slug anywhere under the calling root) and awaits it; cancelling the parent’s call cancels the child; subcall.started carries the real child id, so live progress is the child’s own session.updated. mailInToolResult now folds on the async path too. tui --attach no longer resolves local agent files or creates the default config. Gaps: (1) a prompt_agent_* call with any narrowing (bindings, with, as, own or inherited) or whose helper is not declared under the root still runs in-tool, because ToolExecutionContext cannot name a narrowed OSAgentNode without a module cycle; (2) a PartialUserTurn shown mid-round does not yet carry that round’s mail in its placeholder when mailInToolResult is on. The fake endpoint in checks/ is a fixed responder, so no end-to-end test exercises a sub-agent through the TUI.

Original scope:

prompt_agent_* spawns through spawnSession and waits on the child’s RunStopped (or its mail), so children are cancellable and observable on their own and SubcallProgress with an embedded Session disappears. This touches OneShotTool and the async engine; it is the riskiest phase and is why it is last.

Remaining after Phase 5

G10 residue: done. Narrowed (bindings, with, as, own or inherited) and undeclared-helper prompt_agent_* calls now run as runner sessions: the hook gets a SubagentNarrowing and the tool’s own node (as a Dynamic, which avoids the module cycle). A narrowed child is pinned live (never evicted); the narrowing is not persisted, so after a restart a resumed narrowed child runs un-narrowed (bindings may be secret).
mailInToolResult: done – a partial turn holds the round’s mail out of its query (pMailInToolResult) and renders it into the last placeholder (partialTurnForLlm); folded once when the round completes.
G11: secret params, watches and the run handle are volatile (D7 stands); since 2026-09-25 recoverOnStartup fails the running calls of a session whose required params were lost, with a params_required detail. Since 2026-09-27, watches are also recovered: a session_watches table (SQLite via System.Agents.Session.WatchStore, Postgres via System.Agents.Postgres.mkPostgresWatchStore, both modeled directly on session_mail/MailStore) persists each active watch-session registration (watcher, WatchRequest, absolute deadline) alongside the in-memory WatchHandle; serverWatchSession writes the row before starting to forward, serverUnwatchSession and the forwarding loop’s own TTL expiry both delete it, and recoverOnStartup (via the new recoverWatches) re-registers every row whose deadline has not yet passed and drops the rest, exactly as an in-process TTL timeout would. A WatchRequest carries no secret, so unlike session parameters this needed no client resupply step – D7’s “durable storage” branch applies here, not its “resupplied by the client” one. Events raised while no process was watching are not replayed; a recovered watch only sees events from the moment it is re-registered, same as any other live subscriber. The run handle (lsRun/Async) turned out not to need new work: eviction already refuses to drop a LiveSession with an active run or a running background call (evictIdle’s lsRun/hasRunningCalls checks), so it is never actually lost to eviction; and a process restart was already covered by recoverOnStartup’s existing orphaned/params_required handling above. This is a deviation from the literal G11 wording (“the run Async… [is] lost on restart or eviction”) only in that the eviction half of that claim no longer held by the time this phase started – no behavior change was needed there, only watches. (feat/g11-watch-recovery, full cabal test suite green: 1248 tests in agents-tests, 32 in agents-server-tests, 7 in agents-postgres-tests including a migration-inventory test updated for the new table.)
TUI: selection among several pending calls (select-pending, Ctrl+O) and fail-pending (Ctrl+W) are done; a failed call completes with the text Error: <reason> rather than a new UserToolResponse variant. Still no view for hook.failed.
Service packaging: docs only, no unit file shipped, no bundling/ entry.
checks/lib/fake_llm.py scripts multi-turn answers (text and tool calls, matched on request index, system prompt, last message or offered tools); pty end-to-end checks now cover a sub-agent call as a child session (checks/phase5-subagent-e2e) and deferred calls answered, failed or picked from the Pending panel (checks/phase5-pending-e2e). Not covered yet: interrupts, and the same flows with tui --attach.
D8’s out-of-scope items: multi-server Postgres, mid-RunAsync durability, per-owner API keys.

Decisions (proposed)

D1. The runner is the OS. The ECS World is an engine-internal detail for tool-call entities. OS.Persistence, OS.Interfaces remnants and the unused OSEvent constructors are deleted rather than wired.

D2. One wire format, defined once, in the library. No JSON in examples/agents-server beyond what the servant routes need to reference the Protocol types.

D3. Drafts are a UI concern; the kernel only knows messages and interrupts. Mail stays append-only and immutable. The TUI keeps one editable draft per conversation with append semantics (§5), posted as one PostMessage when it is the user’s turn. Nothing about drafts crosses the wire or is stored. Discrete queued messages, and deleting or reordering them, are dropped: successive messages while the model is busy are treated as one message being elaborated. Mail from other sessions and tool results are shown read-only.

D4. SessionId is the conversation. ConversationId is always derived from it, in every front-end.

D5. The event ring is in memory; the event log is not persisted. Replay covers reconnects; a client that is behind by more than the ring refetches sessions. Revisit only if a use case needs an audit trail (then persist Event rows in the store, keyed by EventSeq).

D6. Embedded mode is the same code as attached mode. The TUI never gets a privileged path to runtime internals again; anything it needs is a Command or an Event, which keeps the web page and API clients at parity.

D7. Secrets are resupplied by the client. No durable secret storage in this spec; lsParams stays volatile and the TUI resupplies from its --params-file on CreateSession/PostMessage, as the web page does today.

D8. Out of scope: several servers on one Postgres, mid-tool-call durability for RunAsync calls, per-owner API keys. They are documented as unsupported in documentation/agents-server.md and are not needed for attach. (Several servers on one Postgres was done since, with run leases: see todos/web-server-embedding.md, Remaining later work.)

todos/web-server-embedding.md, .progress.md: the Host/Runner and the HTTP server this builds on.
todos/session-mailbox.md: mail, interrupts, control messages, watches.
documentation/agents-server.md: the current API; sections to update are marked in Phases 0 and 1.
documentation/tui.md: TUI features that form the parity checklist of Phase 2.
documentation/OS-API.md: the ECS layer; its OS.Interfaces section is stale and goes with D1.

Durable Workflows How-To

2026-10-04T15:53:54Z

Generated from documentation/durable-workflows-howto.md, the repository is the canonical source and may be ahead of this page.

Durable Workflows How-To

This guide shows how to exercise and extend the durable-workflow features in agents-exe. Durable workflows let an agent turn pause after some tool calls, persist its state, and resume later when external results arrive.

The canonical example flow is:

The user asks a question.
The LLM replies with three tool calls.
The runtime executes one call immediately and defers the other two.
The process yields (and can even be restarted).
External workers complete the deferred calls.
The runtime wakes the session with the results.
The LLM receives all three results and continues.

This guide covers:

A ready-to-run demonstrator that exercises the flow without calling a real LLM.
Core concepts behind durable execution.
CLI commands for operating durable sessions.
Building your own durable agent in Haskell.

To run durable agents behind an HTTP API, with sessions in SQLite, see agents-server.md.

Running the demonstrator

The repository contains a self-contained executable called durable-workflow-demo. It uses a mock LLM so you can run it without API keys.

Build and run:

cabal run durable-workflow-demo

You should see output similar to:

Durable workflow demonstrator
=============================

Step 1: run the async scheduler on the LLM turn.
  [executing in-process] fetch_local
Yielded partial turn:
  completed: 1
    - fetch_local: result from fetch_local
  deferred: 2
    - fetch_remote_a token: 550e8400-e29b-41d4-a716-446655440000
    - fetch_remote_b token: 6ba7b810-9dad-11d1-80b4-00c04fd430c8

Step 2: complete the deferred calls from the outside world.
  providing result for token 550e8400-e29b-41d4-a716-446655440000
  providing result for token 6ba7b810-9dad-11d1-80b4-00c04fd430c8
Turn is now complete.

Step 3: resume the session until the LLM produces a final answer.

=== Final LLM response ===
All three calls are complete. Results:
- fetch_local: result from fetch_local
- fetch_remote_a: external result for fetch_remote_a
- fetch_remote_b: external result for fetch_remote_b

The source is in examples/durable-workflow-demo/Main.hs and is heavily commented. It is the best starting point for adapting the workflow to your own agents.

Core concepts

Durable workflows are built from four composable layers.

1. Execution mode

Agent.ctxExecutionMode is either Synchronous (default) or Asynchronous. Only asynchronous agents yield partial turns.

asyncAgent = agent{ctxExecutionMode = Asynchronous}
-- or
asyncAgent = withExecutionMode Asynchronous agent

2. Tool-call policy

A ToolCallPolicy is a pure function that decides, for every tool call, whether to run it synchronously, defer it, run it asynchronously, or isolate it:

policy :: ToolExecutionContext -> LlmToolCall -> ToolCallDisposition
policy _ctx call
    | callName call == "fetch_local"  = RunSync
    | callName call == "bash_command" = RunIsolated (Docker "agents-exe/bash-runner:latest")
    | otherwise                       = Defer (Reason "waiting for external service")

Dispositions can be decorated with timeouts, retries, cache keys, or labels:

Decorate [WithTimeout 30, WithRetries 2] RunSync

Agents loaded from JSON can declare the same policy without Haskell code; see Configuring an agent for durable mode.

3. Pluggable executor

ToolExecutor decouples how a call runs from the session loop. The runtime provides inProcessExecutor, yieldingExecutor, cachingExecutor, isolatedExecutor, and mkDurableExecutor to compose them.

For most use cases you only need to set ctxToolCallPolicy; the session loop falls back to the agent’s native toolCall for synchronous calls.

4. Persistence and wake/resume

When a call is deferred, the runtime optionally stores a serialisable ToolContinuationSnapshot in a ContinuationStore and emits a PartialUserTurn. Later:

wakeSession injects external results.
resumeSession continues execution until completion or the next yield.
SessionBackend (file, SQLite, or composite) stores session state across process restarts.

CLI commands for operators

The agents-exe session command group creates and operates on stored sessions. These commands load sessions through the configured file-based SessionStore and use the first supplied --agent-file when execution needs to advance. Every command that advances execution forces the agent into asynchronous mode.

Configuring an agent for durable mode

To defer calls from the CLI, declare a tool-call policy in the agent JSON. The policy and execution mode are applied everywhere the agent runs: run, the TUI, the session commands, and sub-agents.

{
  "slug": "my-agent",
  "executionMode": "asynchronous",
  "toolCallPolicyConfig": {
    "default": {"tag": "runSync"},
    "rules": [
      {"tool": "fetch_remote_a", "disposition": {"tag": "defer", "reason": "external worker"}},
      {"tool": "fetch_remote_b", "disposition": {"tag": "defer", "reason": "external worker"}},
      {"tool": "build_project", "disposition": {"tag": "runAsync"}}
    ]
  }
}

Rules match on the exact tool name; calls that match no rule get default. Available disposition tags are runSync, runAsync, runIsolated (with a spec), defer (with a reason), and decorate (with decorators and inner). Without toolCallPolicyConfig, every call runs synchronously and nothing is ever deferred.

See async-tool-calls.md for the related asyncYieldStrategy, maxConcurrency, and asyncCallTimeoutSeconds fields.

Walkthrough

# 1. Create a session holding the initial prompt.  The LLM is not called yet.
agents-exe --agent-file ./my-agent.json session start \
    --prompt "Fetch the local and remote data, then summarise it"
# session-id: <session-id>

# 2. Advance until the turn yields on deferred calls.
agents-exe --agent-file ./my-agent.json session resume <session-id>

# 3. See what is waiting on the outside world.
agents-exe session pending <session-id>

# 4. Provide each deferred result.
agents-exe session complete <token> result.txt

# 5. Let the LLM continue with all results.
agents-exe --agent-file ./my-agent.json session resume <session-id>

agents-exe run can start the same flow: when a turn waits on deferred calls it stops, stores the session, and prints a JSON report with the session id and continuation tokens. Continue it with session complete and session resume.

The TUI (agents-exe tui) can also answer deferred calls directly, without the session CLI: a run that stops on deferred calls shows a Pending panel listing each call, and Ctrl+Y (answer-pending) puts the message editor into “answer mode” for the selected one (Ctrl+O moves the selection; Ctrl+W fails it instead) — type the result and send it, which calls the same completeCall mechanism (with autoResume set) the server’s own GET /v1/sessions/:id/pending workers use, so the run resumes on its own. See documentation/tui.md’s “Pending calls” section.

Start a new session

agents-exe --agent-file ./my-agent.json session start --prompt "..."
agents-exe --agent-file ./my-agent.json session start --step --prompt "..."

Creates a new asynchronous session whose first turn contains the prompt, stores it, and prints its session-id. By default no scheduling happens, which leaves every step to an external process or operator. With --step, it also runs one scheduling step right away. It accepts the same prompt options as run (--prompt, --file, media attachments, and so on).

Run exactly one step

agents-exe --agent-file ./my-agent.json session step <session-id>

Runs a single scheduling step and prints the resulting session state: a turn waiting for the LLM, the LLM’s tool calls, a yielded partial turn, or the final answer.

Pause after one async step

agents-exe --agent-file ./my-agent.json session pause <session-id>

Runs one asynchronous step and persists the resulting session, like step. If the turn yields, it prints the continuation tokens for deferred calls.

Resume until completion or next yield

agents-exe --agent-file ./my-agent.json session resume <session-id>

Calls resumeSession and persists after each yield.

List pending deferred calls

agents-exe session pending <session-id>

Shows each deferred call’s tool name, call id, continuation token, and disposition.

Inject an external result

Create a result file. JSON is parsed as a UserToolResponse; anything else is treated as plain text.

# JSON result
echo '{"type":"text","content":"42"}' > result.json
agents-exe session complete <token> result.json

# Plain-text result
echo "42" > result.txt
agents-exe session complete <token> result.txt

complete scans all sessions in the store to find the one containing the token.

Run deferred isolated calls

agents-exe --agent-file ./my-agent.json session run-isolated <session-id>

If the agent has ctxDeploymentRunner configured, this executes deferred RunIsolated calls through the runner and injects their results back into the session.

Building your own durable agent

The demo is a minimal Haskell program. The key pieces are reproduced below.

1. Make the agent asynchronous and set a policy

import System.Agents.Session.Base

myPolicy :: ToolCallPolicy
myPolicy _ctx call
    | callName call == "fetch_local" = RunSync
    | otherwise                      = Defer (Reason "external")

agent' = agent
    { ctxExecutionMode = Asynchronous
    , ctxToolCallPolicy = myPolicy
    }

2. Run the async scheduler

import System.Agents.Session.Step (runStepM)

(_agent, result) <- runStepM convId agent' session0
case result of
    Left final      -> putStrLn "Session completed immediately."
    Right session1  -> putStrLn "Session yielded."

3. Wake with external results

import System.Agents.Session.Wake (wakeSession)

let responses =
        [ (token, TextResponse "external answer")
        | (token, _call) <- deferredCalls
        ]
session2 <- wakeSession session1 responses

4. Resume

import System.Agents.Session.Wake (resumeSession)

final <- resumeSession convId agent' session2
case final of
    Left (llmTurn, _session) -> print (llmTurn.llmResponse.responseText)
    Right session3           -> putStrLn "Yielded again."

5. Add durable storage

import System.Agents.Session.Async (mkSqliteContinuationStore)
import System.Agents.SessionStore (mkSqliteSessionStore)
import Database.SQLite.Simple (open)

conn <- open ".agents-durable.db"
backend <- mkSqliteSessionStore conn
store   <- mkSqliteContinuationStore conn

let durableAgent = withDurableWorkflows backend store agent'

withDurableWorkflows is a convenience combinator that installs both a session backend and a continuation store.

6. Add a tool cache

import System.Agents.Tools.Cache (mkSqliteToolCache)

cache <- mkSqliteToolCache ".agents-cache.db"
let cachedAgent = withToolCache agent' cache

Cached synchronous calls are skipped on resume if their result is already in the cache.

Isolated execution

To run a tool call outside the current process, provide a DeploymentRunner:

import System.Agents.Session.Base (dockerRunner, localProcessRunner)

runner = localProcessRunner "./worker.sh"
-- or
runner = dockerRunner "agents-exe/bash-runner:latest"

isolatedAgent = withDeploymentRunner runner durableAgent

The worker receives a stable JSON envelope on stdin and must print a result envelope on stdout. See System.Agents.Session.Isolation for the envelope schema, or look at the worker script in test/DurableWorkflowTests.hs.

Testing

The durable-workflow implementation is covered by:

test/DurableWorkflowTests.hs — policy, wake/resume, cache, backends, isolation envelopes, and integration.
test/DurableWorkflowDeterminismTests.hs — resume-twice and edge-case tests.
test/SessionDurableTests.hs — CLI helper tests.

Run them with:

cabal test agents-tests

Specs

2026-10-04T15:53:54Z

Specs

The todos/ directory holds the design documents behind the larger features (durable workflows, asynchronous tool calls, the session mailbox, partial application of tool arguments, the standalone server) and their progress notes. Most start with a Status line saying what of it has shipped, which is reproduced here. These pages are generated from the agents-exe repository: the repository is the canonical source, and may be ahead of what is published here.

Asynchronous / Interruptible Tool Calls, Status: no status line yet
Async Tool Calls Implementation Progress, Status: no status line yet
Spec: bash tools v2, long-running tool processes, Status: proposal, 2026-09-25, from issue #539 and a robot-chat conversation
Plan: Durable execution from the CLI, Status: no status line yet
Plan: Durable Workflows for agents-exe, Status: no status line yet
Durable Workflows — Implementation Progress, Status: no status line yet
Spec: the OS as a standalone server (TUI and web UI as clients), Status: proposal, 2026-09-23, revised the same day after checking service
Spec: a salmon-driven preview deployment of agents-server, Status: proposal, 2026-10-03. Nothing implemented. A design pass with a
Spec: the session mailbox (agent-to-agent mail, suspended tool calls, interrupts, tool-call wrappers), Status: proposal, 2026-09-22. Nothing implemented. Supersedes the designs in
Spec: partial application of tool arguments, Status: Phases 1-7 and §8.4 (derive_agent) done as of 2026-09-21. Phase 1 done
Spec: embedding agents-exe in a web server, Status: proposed (2026-09-18)
Web server embedding — Implementation Progress, Status: no status line yet

OS Model API Reference

2026-10-04T15:53:54Z

Generated from documentation/OS-API.md, the repository is the canonical source and may be ahead of this page.

OS Model API Reference

Complete API reference for the Entity-Component-System (ECS) based OS architecture.

Core ECS Types
World Operations
Agent Operations
Toolbox Operations
Resource Management
Concurrent Access
Conversation Tracking
OS Events
Persistence Layer

Core ECS Types

EntityId

-- | Unique identifier for any entity in the system.
newtype EntityId = EntityId { unEntityId :: UUID }
    deriving (Eq, Ord, Show, Hashable, FromJSON, ToJSON)

-- | Generate a new unique EntityId.
newEntityId :: IO EntityId

Example:

eid1 <- newEntityId  -- Unique ID
eid2 <- newEntityId  -- Different unique ID
assert (eid1 /= eid2)  -- True

Phantom-Typed Entity IDs

Phantom types provide compile-time safety for entity operations:

newtype AgentId = AgentId { unAgentId :: EntityId }
newtype ToolboxId = ToolboxId { unToolboxId :: EntityId }
newtype ConversationId = ConversationId { unConversationId :: EntityId }
newtype TurnId = TurnId { unTurnId :: EntityId }
newtype ToolCallId = ToolCallId { unToolCallId :: EntityId }
newtype ResourceId = ResourceId { unResourceId :: EntityId }

Example:

eid <- newEntityId
let agentId = AgentId eid
let toolboxId = ToolboxId eid  -- Same underlying ID, different type

-- Type-safe: Can't mix up IDs
useAgent agentId       -- OK
useAgent toolboxId     -- Compile error!

Component Typeclass

-- | Type-level identifier for component types.
newtype ComponentTypeId = ComponentTypeId { unComponentTypeId :: Int }

-- | A component is any data type that can be attached to an entity.
class Component a where
    componentId :: Proxy a -> ComponentTypeId

Creating a custom component:

data MyComponent = MyComponent
    { mcValue :: Int
    , mcName :: Text
    } deriving (Show, Eq, Generic)

instance FromJSON MyComponent
instance ToJSON MyComponent

-- Choose a unique ID (check existing IDs to avoid collisions)
instance Component MyComponent where
    componentId _ = ComponentTypeId 100

World Operations

Creating and Managing Worlds

-- | The world contains all component stores.
newtype World = World { componentStores :: HashMap ComponentTypeId (TVar Any) }

-- | Create an empty world.
newWorld :: STM World

-- | Register a component store for a component type.
registerComponentStore :: (Component a, Typeable a) => World -> Proxy a -> STM World

Example:

import Data.Proxy (Proxy(..))

-- Create world with agent components
world <- atomically $ do
    w <- newWorld
    w' <- registerComponentStore w (Proxy @AgentConfig)
    w'' <- registerComponentStore w' (Proxy @AgentState)
    pure w''

Entity Operations

-- | Create a new entity.
createEntity :: IO EntityId

-- | Check if an entity exists (has any component).
entityExists :: World -> EntityId -> STM Bool

-- | Get all entities with a specific component.
allEntitiesWithComponent :: (Component a, Typeable a) => World -> STM [EntityId]

Example:

eid <- createEntity
exists <- atomically $ entityExists world eid  -- False

atomically $ setComponent world eid (AgentConfig "test" ...)
exists' <- atomically $ entityExists world eid  -- True

-- Find all agents
agentIds <- atomically $ allEntitiesWithComponent @AgentConfig world

Component Operations

-- | Get a component for an entity.
getComponent :: (Component a, Typeable a) => World -> EntityId -> STM (Maybe a)

-- | Set a component for an entity.
setComponent :: (Component a, Typeable a) => World -> EntityId -> a -> STM ()

-- | Modify a component for an entity.
modifyComponent :: (Component a, Typeable a) => World -> EntityId -> (a -> a) -> STM ()

-- | Remove a component from an entity.
removeComponent :: (Component a, Typeable a) => World -> EntityId -> STM ()

-- | Check if an entity has a specific component.
hasComponent :: (Component a, Typeable a) => World -> EntityId -> STM Bool

Example:

-- Set component
atomically $ setComponent world eid AgentConfig
    { agentName = "my-agent"
    , agentModel = ModelConfig "openai" "..." "gpt-4" "key1"
    , agentSystemPrompt = "You are helpful"
    , agentToolboxBindings = []
    }

-- Get component
mConfig <- atomically $ getComponent @AgentConfig world eid
case mConfig of
    Nothing -> putStrLn "No agent config"
    Just config -> putStrLn $ "Agent: " ++ show config.agentName

-- Modify component
atomically $ modifyComponent @AgentConfig world eid $
    \c -> c { agentName = "renamed-agent" }

-- Check and remove
hasIt <- atomically $ hasComponent @AgentConfig world eid
when hasIt $ atomically $ removeComponent @AgentConfig world eid

Agent Operations

Agent Configuration

data AgentConfig = AgentConfig
    { agentName :: Text                    -- ^ Human-readable name
    , agentModel :: ModelConfig            -- ^ LLM configuration
    , agentSystemPrompt :: Text            -- ^ System instructions
    , agentToolboxBindings :: [ToolboxBindingSpec]  -- ^ Bound toolboxes
    } deriving (Show, Eq, Generic)

data ModelConfig = ModelConfig
    { modelFlavor :: Text      -- ^ Provider (openai, mistral, etc.)
    , modelUrl :: Text         -- ^ API endpoint
    , modelName :: Text        -- ^ Model identifier
    , modelApiKeyId :: Text    -- ^ Key reference
    } deriving (Show, Eq, Generic)

Agent State

data AgentState = AgentState
    { agentStatus :: AgentStatus           -- ^ Current status
    , agentCurrentConversation :: Maybe ConversationId
    , agentCreatedAt :: UTCTime
    } deriving (Show, Eq, Generic)

data AgentStatus
    = AgentIdle                           -- ^ Available for work
    | AgentBusy TurnId                    -- ^ Executing a turn
    | AgentError Text                     -- ^ Error state
    deriving (Show, Eq, Generic)

Example: Creating an Agent

createAgent :: OS -> AgentConfig -> IO AgentId
createAgent os config = do
    eid <- createEntity
    now <- getCurrentTime
    
    atomically $ do
        setComponent (osWorld os) eid config
        setComponent (osWorld os) eid AgentState
            { agentStatus = AgentIdle
            , agentCurrentConversation = Nothing
            , agentCreatedAt = now
            }
    
    pure $ AgentId eid

-- Usage
let config = AgentConfig
    { agentName = "coder-agent"
    , agentModel = ModelConfig "openai" "https://api.openai.com/v1" "gpt-4" "openai-key"
    , agentSystemPrompt = "You are a helpful coding assistant"
    , agentToolboxBindings = []
    }
agentId <- createAgent os config

Toolbox Operations

Toolbox Configuration

data ToolboxConfig = ToolboxConfig
    { toolboxName :: Text           -- ^ Human-readable name
    , toolboxType :: ToolboxType    -- ^ Type of toolbox
    , toolboxSettings :: Value      -- ^ Type-specific settings
    } deriving (Show, Eq, Generic)

data ToolboxType
    = ToolboxTypeBash
    | ToolboxTypeMCP
    | ToolboxTypeOpenAPI
    | ToolboxTypePostgREST
    | ToolboxTypeSqlite
    | ToolboxTypeSystem
    | ToolboxTypeDeveloper
    | ToolboxTypeLua
    | ToolboxTypeSkills
    deriving (Show, Eq, Generic)

Toolbox State

data ToolboxState = ToolboxState
    { toolboxScope :: ResourceScope     -- ^ Resource lifetime scope
    , toolboxStatus :: ToolboxStatus    -- ^ Current status
    , toolboxResourceRef :: Maybe ResourceId  -- ^ Associated resource
    } deriving (Show, Eq, Generic)

data ToolboxStatus
    = ToolboxInitializing
    | ToolboxReady
    | ToolboxError Text
    | ToolboxDisposed
    deriving (Show, Eq, Generic)

Resource Scope

data ResourceScope
    = ScopeGlobal                       -- ^ Global/program scope
    | ScopeAgent AgentId                -- ^ Per-agent scope
    | ScopeConversation ConversationId  -- ^ Per-conversation scope
    deriving (Show, Eq, Generic)

Example: Creating Toolboxes

-- Create a bash toolbox
let bashConfig = ToolboxConfig
    { toolboxName = "bash-tools"
    , toolboxType = ToolboxTypeBash
    , toolboxSettings = object ["directory" .= "./tools"]
    }
bashId <- createToolbox os bashConfig

-- Create a SQLite toolbox
let sqliteConfig = ToolboxConfig
    { toolboxName = "memory-db"
    , toolboxType = ToolboxTypeSqlite
    , toolboxSettings = object 
        [ "path" .= "./memory.db"
        , "access" .= "readwrite"
        ]
    }
sqliteId <- createToolbox os sqliteConfig

-- Bind toolboxes to agent
let agentConfig = AgentConfig
    { ...
    , agentToolboxBindings = 
        [ unToolboxId bashId
        , unToolboxId sqliteId
        ]
    }

Resource Management

Resource Types

data ResourceType
    = SqliteResource SqliteConfig
    | LuaResource LuaConfig
    | HttpResource HttpConfig
    | CustomResource Text Value
    deriving (Show, Eq)

data ResourceHandle = ResourceHandle
    { handleId :: ResourceId
    , handleCleanup :: IO ()
    , handleAccess :: (ResourceAccessor -> IO a) -> IO a
    }

data ResourceInfo = ResourceInfo
    { resourceId :: ResourceId
    , resourceScope :: ResourceScope
    , resourceType :: ResourceType
    , resourceCreatedAt :: UTCTime
    }

Resource Context

data ResourceContext = ResourceContext
    { contextScope :: [ScopeLevel]      -- ^ Active scope path
    , contextRegistry :: ResourceRegistry
    }

data ScopeLevel
    = ProgramScope
    | AgentScope AgentId
    | ToolboxScope ToolboxId
    | ConversationScope ConversationId
    | TurnScope TurnId
    | ToolCallScope ToolCallId
    deriving (Show, Eq)

Resource Operations

-- | Create a new resource.
createResource :: 
    ResourceContext -> 
    ResourceType -> 
    (ResourceId -> IO ResourceHandle) -> 
    IO ResourceId

-- | Cleanup all resources in a scope.
cleanupScope :: ResourceRegistry -> ScopeLevel -> IO ()

-- | Check if a resource is valid in given scopes.
isResourceValid :: ResourceScope -> [ScopeLevel] -> Bool

-- | Access a resource with a function.
withResource :: ResourceRegistry -> ResourceId -> (ResourceAccessor -> IO a) -> IO (Maybe a)

-- | Get total resource count.
getResourceCount :: ResourceRegistry -> IO Int

Example: Resource Lifecycle

import System.Agents.OS.Resources

-- Create registry and context
registry <- atomically newResourceRegistry
let ctx = ResourceContext [ProgramScope] registry

-- Create SQLite resource
rid <- createResource ctx (SqliteResource config) $ \rid -> do
    conn <- openConnection config
    pure ResourceHandle
        { handleId = rid
        , handleCleanup = do
            putStrLn "Closing SQLite connection"
            closeConnection conn
        , handleAccess = \f -> f (SqliteAccessor conn)
        }

-- Use the resource
result <- withResource registry rid $ \accessor -> do
    case accessor of
        SqliteAccessor conn -> queryDatabase conn "SELECT * FROM table"

-- Later, cleanup all agent resources
cleanupScope registry (AgentScope agentId)

Concurrent Access

Access Patterns

data AccessPattern
    = ExclusiveAccess      -- ^ Single accessor (TMVar)
    | ReadWriteAccess      -- ^ Multiple readers, single writer (RWLock)
    | PoolAccess Int       -- ^ Bounded pool (TBQueue)
    | StatelessAccess      -- ^ No synchronization needed
    deriving (Show, Eq)

data AccessControl = AccessControl
    { accessPattern :: AccessPattern
    , accessTimeout :: Maybe NominalDiffTime
    }

Resource Monad

-- | Monad for resource operations.
newtype ResourceM a = ResourceM { unResourceM :: ReaderT ResourceContext (ExceptT ResourceError IO) a }
    deriving (Functor, Applicative, Monad, MonadIO)

-- | Run a ResourceM computation.
runResourceM :: ResourceContext -> ResourceM a -> IO (Either ResourceError a)

-- | Resource errors.
data ResourceError
    = ResourceNotFound ResourceId
    | ResourceBusy ResourceId
    | ResourceClosed ResourceId
    | ResourceAccessTimeout ResourceId NominalDiffTime
    | ResourceInvalidAccess ResourceId Text
    deriving (Show, Eq)

Access Operations

-- | Execute with exclusive access.
withExclusive :: ResourceId -> ResourceM a -> ResourceM a

-- | Execute with read access.
withRead :: ResourceId -> ResourceM a -> ResourceM a

-- | Execute with write access.
withWrite :: ResourceId -> ResourceM a -> ResourceM a

-- | Execute with pooled resource.
withPooled :: ResourceId -> ResourceM a -> ResourceM a

-- | Execute without synchronization.
withStateless :: ResourceId -> ResourceM a -> ResourceM a

Example: Concurrent Access Patterns

import System.Agents.OS.Concurrent

-- SQLite with WAL mode (supports concurrent reads)
sqliteRid <- createSqliteResource ctx config True  -- WAL mode enabled

-- Multiple concurrent reads (safe)
forConcurrently_ [1..10] $ \_ -> do
    result <- runResourceM ctx $ withRead sqliteRid $ do
        queryData
    print result

-- Exclusive write (blocks readers)
runResourceM ctx $ withWrite sqliteRid $ do
    modifyData

-- Lua interpreter (requires exclusive access)
luaRid <- createLuaResource ctx config
result <- runResourceM ctx $ withExclusive luaRid $ do
    runScript "return 1 + 1"

-- HTTP connection pool
httpRid <- createHttpPool ctx 10  -- 10 connections
results <- forConcurrently urls $ \url ->
    runResourceM ctx $ withPooled httpRid $ do
        fetchUrl url

Initialization Helpers

-- | Initialize access control.
initAccessControl :: AccessPattern -> Maybe NominalDiffTime -> AccessControl

-- | Initialize SQLite access (WAL mode flag).
initSqliteAccess :: Bool -> IO SqliteAccess

-- | Initialize Lua access.
initLuaAccess :: IO LuaAccess

-- | Initialize HTTP access.
initHttpAccess :: Int -> ResourceId -> IO HttpAccess

Conversation Tracking

Conversation Components

data ConversationConfig = ConversationConfig
    { conversationTitle :: Maybe Text
    , conversationMetadata :: Map Text Value
    } deriving (Show, Eq, Generic)

data ConversationState = ConversationState
    { conversationAgentId :: AgentId
    , conversationStatus :: ConversationStatus
    , conversationStartedAt :: UTCTime
    , conversationLastActivity :: TVar UTCTime
    } deriving (Generic)

data ConversationStatus
    = ConversationActive
    | ConversationPaused
    | ConversationCompleted
    | ConversationError Text
    deriving (Show, Eq, Generic)

Turn Components

data TurnConfig = TurnConfig
    { turnConversationId :: ConversationId
    , turnParentTurnId :: Maybe TurnId
    } deriving (Show, Eq, Generic)

data TurnState = TurnState
    { turnStatus :: TurnStatus
    , turnStartedAt :: UTCTime
    , turnCompletedAt :: Maybe UTCTime
    } deriving (Show, Eq, Generic)

data TurnStatus
    = TurnStarting
    | TurnProcessing
    | TurnWaitingForTools
    | TurnCompleted
    | TurnError Text
    deriving (Show, Eq, Generic)

Tool Call Components

data ToolCallConfig = ToolCallConfig
    { toolCallTurnId :: TurnId
    , toolCallParentId :: Maybe ToolCallId
    , toolCallName :: Text
    , toolCallInput :: Value
    } deriving (Show, Eq, Generic)

data ToolCallState = ToolCallState
    { toolCallStatus :: ToolCallStatus
    , toolCallResult :: Maybe Value
    , toolCallStartedAt :: UTCTime
    , toolCallCompletedAt :: Maybe UTCTime
    } deriving (Show, Eq, Generic)

data ToolCallStatus
    = ToolCallPending
    | ToolCallExecuting
    | ToolCallCompleted
    | ToolCallFailed Text
    deriving (Show, Eq, Generic)

Lineage Tracking

data Lineage = Lineage { unLineage :: [LineageFrame] }
    deriving (Show, Eq)

data LineageFrame = LineageFrame
    { frameType :: FrameType
    , frameEntityId :: EntityId
    , frameTimestamp :: UTCTime
    } deriving (Show, Eq)

data FrameType
    = ProgramFrame
    | AgentFrame
    | ToolboxFrame
    | ConversationFrame
    | TurnFrame
    | ToolCallFrame
    deriving (Show, Eq, Enum, Bounded)

Lineage Operations

-- | Empty lineage.
emptyLineage :: Lineage

-- | Push a frame onto the lineage.
pushLineage :: FrameType -> EntityId -> UTCTime -> Lineage -> Lineage

-- | Get lineage depth.
lineageDepth :: Lineage -> Int

-- | Get the most recent frame.
lineageHead :: Lineage -> Maybe LineageFrame

-- | Get the oldest frame.
lineageRoot :: Lineage -> Maybe LineageFrame

-- | Build context from lineage.
buildLineageContext :: Lineage -> LineageContext

-- | Find frames by type.
findFramesByType :: FrameType -> Lineage -> [LineageFrame]

-- | Check if in specific context.
isInConversation :: Lineage -> Bool
isInTurn :: Lineage -> Bool
currentFrameType :: Lineage -> Maybe FrameType

Example: Conversation and Lineage

import System.Agents.OS.Conversation

-- Create conversation
convId <- createEntity
now <- getCurrentTime
lastActivity <- newTVarIO now

atomically $ do
    setComponent world convId ConversationConfig
        { conversationTitle = Just "My Chat"
        , conversationMetadata = Map.empty
        }
    setComponent world convId ConversationState
        { conversationAgentId = agentId
        , conversationStatus = ConversationActive
        , conversationStartedAt = now
        , conversationLastActivity = lastActivity
        }

-- Create turn
turnId <- createEntity
atomically $ do
    setComponent world turnId TurnConfig
        { turnConversationId = convId
        , turnParentTurnId = Nothing
        }
    setComponent world turnId TurnState
        { turnStatus = TurnStarting
        , turnStartedAt = now
        , turnCompletedAt = Nothing
        }

-- Build lineage
let lineage = pushLineage ConversationFrame (unConversationId convId) now $
              pushLineage TurnFrame (unTurnId turnId) now $
              emptyLineage

-- Check depth
print $ lineageDepth lineage  -- 2

-- Find conversation frames
let convFrames = findFramesByType ConversationFrame lineage

OS Events

Subcall lifecycle and tool-call activity are reported through a single emission mechanism, OSEmission (System.Agents.OS.Events), and the session runner (System.Agents.Host.Runner) is the mechanism’s one consumer that matters in practice: it turns each OSEmission into an EventBody (System.Agents.Protocol) and publishes it on the owning session’s event stream, which every runner client (the in-process TUI, the HTTP/SSE server) subscribes to. There is no more OSEvent type or ctxEventQueue; System.Agents.Session.Base.Agent.ctxEmit / System.Agents.Tools.Context.ToolExecutionContext.ctxEmit is the one hook (see todos/os-as-standalone-server.md, Phase 2c and Phase 3c).

OSEmission Type

-- | The single in-library event emission type.
data OSEmission
    = EmitSubcallStarted SessionId SessionId Text Int
      -- ^ Parent session id, child session id, the helper's slug, call depth.
    | EmitSubcallCompleted SessionId (Maybe Text)
      -- ^ Child session id, the subcall's result text (when it succeeded).
    | EmitSubcallFailed SessionId Text
      -- ^ Child session id, the failure message.
    | EmitToolCallActivity ToolCallActivity
    | EmitError Text
      -- ^ A hook (e.g. a before/after tool-call command hook) failed
      -- outside of the normal tool-call result path. Not a session
      -- failure: the run continues.
    deriving (Show)

ToolCallActivity (also in System.Agents.OS.Events) carries a tool call’s session/conversation/tool-call ids, the LLM provider’s own call id when known, the tool name, a ToolCallPhase (ToolCallStarted, ToolCallProgressed Value, ToolCallCompleted, ToolCallFailed Text, ToolCallCancelled) and a timestamp.

Subcall Event Lifecycle

Parent Conversation
       │
       ▼ triggers agent call
┌───────────────────────────┐
│ EmitSubcallStarted         │ ctxEmit'd when the subcall begins
│ - parent session id        │
│ - child session id         │
│ - agent slug                │
│ - depth                    │
└──────────────┬─────────────┘
               │
               ▼
        Agent execution
               │
               ▼ (completion)
┌───────────────────────────┐
│ EmitSubcallCompleted        │ ctxEmit'd on success
│ - child session id         │
│ - result                   │
└─────────────────────────────┘
               │
               ▼ (or failure)
┌───────────────────────────┐
│ EmitSubcallFailed           │ ctxEmit'd on error
│ - child session id         │
│ - error                    │
└─────────────────────────────┘

Phase 3c retires the old OSEvent_SubcallProgress (it used to carry a whole Session, snapshotted after every step, purely for the TUI’s benefit): there is no runner-event equivalent. Phase 5 gives a real sub-agent session its own SessionUpdated events instead, once prompt_agent_* spawns through spawnSession rather than running inline.

The Runner’s Event Stream

System.Agents.Host.Runner.toEventBody is the one place that converts OSEmission into Protocol.EventBody:

toEventBody :: OSEmission -> EventBody
toEventBody = \case
    EmitSubcallStarted parent child slug depth -> SubcallStarted parent child slug depth
    EmitSubcallCompleted child result -> SubcallCompleted child result
    EmitSubcallFailed child msg -> SubcallFailed child msg
    EmitToolCallActivity activity -> ToolCallProgressed activity
    EmitError msg -> HookFailed msg

newAgent installs withEmit (emit runner sid . toEventBody) on every root agent it builds, so every emission from that session (and its subcalls, which inherit ctxEmit the same way they inherit ctxWorld) ends up on sid’s own event stream, wire-encoded with kind subcall.started / subcall.completed / subcall.failed / tool.progressed / hook.failed (see documentation/agents-server.md’s event table for the full list, including the runner’s own run.*/session.* kinds that do not originate from OSEmission at all).

Using ctxEmit Outside a Runner

A local (non-runner) consumer that used to drain ctxEventQueue – a CLI path, a test – installs System.Agents.OS.Events.queueEmitter (or newQueueEmitter, which also allocates the queue) as ctxEmit instead, and reads emissions back off the queue:

import System.Agents.OS.Events (newQueueEmitter, OSEmission (..))

(queue, emitter) <- newQueueEmitter
let agent' = agent{ctxEmit = Just emitter}
-- ... run agent' ...
emissions <- atomically $ flushTQueue queue

ToolExecutionContext Extensions

For subcall visibility, the ToolExecutionContext includes OS integration fields:

data ToolExecutionContext = ToolExecutionContext
    { -- ... existing fields ...
    , ctxWorld :: Maybe World
    -- ^ Optional OS World for ECS operations. When present, subcalls
    -- can insert entities and components into the OS.
    , ctxEmit :: Maybe (OSEmission -> IO ())
    -- ^ Optional emission hook. When present, subcalls emit events to
    -- notify a runner (or other local consumer) of their lifecycle.
    , ctxParentConversation :: Maybe ConversationId
    -- ^ Optional parent conversation ID for subcalls. When present,
    -- indicates this context is for a nested agent invocation.
    }

Helper Functions:

-- | Get the subcall depth (0 if not a subcall).
getSubcallDepth :: ToolExecutionContext -> Int

-- | Check if this context represents a subcall.
isSubcallContext :: ToolExecutionContext -> Bool

-- | Create a nested context for subcall execution.
mkSubcallContext ::
    ToolExecutionContext ->
    Maybe World ->
    ConversationId ->
    ToolExecutionContext

TUI Integration

In the TUI, runner Events are turned into AppEvents by bridgeRunnerEvents (System.Agents.TUI.Core), which maps each EventBody kind onto its AppEvent counterpart – subcall.started / subcall.completed / subcall.failed / tool.progressed map straight across, translating SessionId to ConversationId (they are the same UUID). hook.failed has no dedicated view yet (it is not a session failure, so it is not surfaced as one); see documentation/tui.md.

Persistence Layer

Removed. System.Agents.OS.Persistence was deleted (D1 in todos/os-as-standalone-server.md): sessions persist through the runner’s SessionBackend (SQLite or Postgres), not through ECS components.

Common Patterns

Pattern 1: Agent with Multiple Toolboxes

createFullAgent :: OS -> Text -> [ToolboxId] -> IO AgentId
createFullAgent os name toolboxes = do
    let config = AgentConfig
        { agentName = name
        , agentModel = ModelConfig "openai" "..." "gpt-4" "key"
        , agentSystemPrompt = "You are helpful"
        , agentToolboxBindings = map unToolboxId toolboxes
        }
    createAgent os config

-- Usage
bashTb <- createToolbox os bashConfig
sqlTb <- createToolbox os sqliteConfig
httpTb <- createToolbox os httpConfig

agentId <- createFullAgent os "multi-tool-agent" [bashTb, sqlTb, httpTb]

Pattern 2: Forking a Conversation

forkConversation :: World -> ConversationId -> IO ConversationId
forkConversation world convId = do
    -- Get original config
    Just config <- atomically $ getComponent @ConversationConfig world (unConversationId convId)
    
    -- Create new conversation
    newConvId <- createEntity
    now <- getCurrentTime
    lastActivity <- newTVarIO now
    
    atomically $ do
        setComponent world newConvId config
        setComponent world newConvId ConversationState
            { conversationAgentId = ...
            , conversationStatus = ConversationActive
            , conversationStartedAt = now
            , conversationLastActivity = lastActivity
            }
    
    -- Copy turn history
    turns <- getConversationTurns world convId
    forM_ turns $ \turnId -> do
        Just turnConfig <- atomically $ getComponent @TurnConfig world (unTurnId turnId)
        newTurnId <- createEntity
        atomically $ setComponent world newTurnId turnConfig
            { turnConversationId = ConversationId newConvId
            }
    
    pure $ ConversationId newConvId

Pattern 3: Resource Pool with Timeout

withPooledTimeout :: ResourceContext -> ResourceId -> NominalDiffTime -> ResourceM a -> ResourceM a
withPooledTimeout ctx rid timeout action = do
    currentTimeout <- getResourceTimeout rid
    -- Temporarily set timeout
    updateResourceTimeout rid (Just timeout)
    result <- withPooled rid action
    -- Restore original timeout
    updateResourceTimeout rid currentTimeout
    pure result

Pattern 4: Transaction with Multiple Components

updateAgentAndToolbox :: World -> AgentId -> ToolboxId -> AgentConfig -> ToolboxConfig -> STM ()
updateAgentAndToolbox world agentId toolboxId agentConfig toolboxConfig = do
    -- Both updates happen atomically
    setComponent world (unAgentId agentId) agentConfig
    setComponent world (unToolboxId toolboxId) toolboxConfig

-- Usage
atomically $ updateAgentAndToolbox world agentId toolboxId newAgentConfig newToolboxConfig

Type Index

Core Types

EntityId - Base entity identifier
AgentId, ToolboxId, ConversationId, TurnId, ToolCallId, ResourceId - Phantom-typed IDs
ComponentTypeId - Component type identifier
World - ECS world container

Component Types

AgentConfig, AgentState, AgentStatus - Agent components
ToolboxConfig, ToolboxState, ToolboxBinding - Toolbox components
ConversationConfig, ConversationState, ConversationStatus - Conversation components
TurnConfig, TurnState, TurnStatus - Turn components
ToolCallConfig, ToolCallState, ToolCallStatus - Tool call components
Message, MessageRole - Message components
Lineage, LineageFrame, FrameType - Lineage components

Resource Types

ResourceType, ResourceHandle, ResourceInfo - Resource definitions
ResourceContext, ScopeLevel, ResourceScope - Scope management
ResourceRegistry - Resource storage

Concurrent Types

AccessPattern, AccessControl - Access patterns
ResourceM - Resource monad
ResourceError - Error types
SyncPrimitive, ExclusiveLock, ReadWriteLock, PoolLock - Lock types

Event Types

OSEmission - the single emission type: subcall lifecycle, tool-call activity, hook failures
ToolCallActivity, ToolCallPhase - background tool-call activity payloads
SessionProgress - Session progress tracking

Persistence Types

PersistenceHandle, PersistenceBackendType - Backend types
Persistable - Persistence typeclass
EntityQuery - Query specification

Plan: Durable execution from the CLI

2026-10-04T15:53:54Z

Generated from todos/durable-workflows-cli-start-plan.md, the repository is the canonical source and may be ahead of this page.

Plan: Durable execution from the CLI

Goal

Make the durable-workflow execution mode accessible from the CLI in one-shot form, so a user can:

Start a brand-new session with a prompt using agents session start ....
Have the agent run until it yields (partial turn, deferred tool calls) or completes.
Inspect pending calls with agents session pending.
Complete deferred calls with agents session complete.
Resume with agents session resume.
Iterate step by step, either letting the LLM continue or injecting tool results on behalf of the user.

This closes the gap between the existing run command (fully synchronous, runs to completion) and the session pause/resume/... commands (which only operate on already-existing sessions).

Decisions on open questions

Question	Decision
How many steps should `session start` run?	Zero by default, with an optional `--step` flag to run exactly one scheduling step. The goal is to forfeit as much scheduling as possible to external processes/users.
How is the initial turn created?	Option B: construct the initial `UserTurn` directly in `session start` with the prompt and media, then persist the session.
Isolated execution?	Out of scope for this milestone. Only `RunSync`, `RunAsync`, and `Defer` are supported.
Policy language?	Keep it simple: exact tool-name rules plus a default disposition.
Continuation store?	Keep it simple: rely on the file-based `SessionStore` and the in-memory partial turn. A SQLite continuation store can be added later.

Current state and gaps

What already works

The library has full durable-workflow primitives:
- ToolCallPolicy, ToolCallDisposition (RunSync, RunAsync, Defer, RunIsolated).
- runAsync in System.Agents.Session.Loop runs an async agent until it yields a PartialUserTurn.
- resumeSession and wakeSession in System.Agents.Session.Wake.
- SessionBackend, ContinuationStore, and CLI commands session pause/resume/pending/complete/run-isolated.
The existing run command already accepts prompts, media, session files, and thinking-output options.

What is missing

No CLI command to create a new session in async mode.
- run always uses mainOneShotTextWithThinking, which creates a synchronous agent and calls Loop.run to completion.
- session pause/resume/... require an existing SessionId; they cannot start from a user prompt.
No way to configure a durable policy from the agent JSON file.
- System.Agents.Base.Agent (the JSON config type) has no fields for executionMode, toolCallPolicy, toolExecutor, continuationStore, deploymentRunner, or sessionBackend.
- The CLI’s withAgentForSession builds the runtime agent with defaultToolCallPolicy, so even in async mode every call runs synchronously and no yielding occurs.
No shared parser for run-like options in the session command group.
- parseOneShotOptions is only used by run.
- session start should accept the same prompt/media/thinking options.
No single-step scheduler command wired to the CLI.
- runAsync exists in the library but is not used by any CLI handler.
- handlePause calls runStepM once on an existing session, but it does not create the initial user turn from a prompt.

Proposed CLI additions

New command: `agents session start`

# Create a session with an initial UserTurn, do not call the LLM yet.
agents session start \
    --agent-file my-agent.json \
    --prompt "Research the weather in Paris and New York"

# Same, but also run one scheduling step (e.g. ask the LLM and possibly
# execute the first batch of tool calls).
agents session start --step \
    --agent-file my-agent.json \
    --prompt "Research the weather in Paris and New York"

Semantics:

Creates a fresh Session with sessionVersion = Just 2 and sessionExecutionMode = Just Asynchronous.
Constructs an initial UserTurn containing the prompt (and any media attachments).
Loads the agent, applies the durable configuration from the agent file (see below), and sets ctxExecutionMode = Asynchronous.
By default, persists the session immediately without calling the LLM.
With --step, calls runStepM exactly once before persisting.
Prints:
- the new session-id;
- the current state (UserTurn, LlmTurn, PartialUserTurn, or completed);
- continuation tokens for any deferred calls.

If the session completes after the optional step, the final LLM response is printed as well.

New command: `agents session step`

agents session step <session-id>

Runs exactly one scheduling step on an existing session and persists the result. This is the workhorse for manual iteration:

session start -p "..." → session with UserTurn.
session step <id> → LLM produces tool calls (LlmTurn).
session step <id> → sync tools run, async/deferred calls yield (PartialUserTurn).
session complete <token> result.json → inject external result.
session step <id> → woken results processed, possibly another LLM turn.
Repeat until completion.

session step is functionally similar to the existing session pause, but named for iteration rather than interruption. The existing session pause can be kept for backward compatibility.

Reuse `run` options

Both session start and session step should accept the same prompt-script options as run:

agents session start \
    --agent-file my-agent.json \
    --prompt "Summarize" \
    --file report.md \
    --media screenshot.png \
    --thinking stderr

The cleanest way is to extract a shared parser type, e.g. OneShotPromptOptions, from System.Agents.CLI.OneShot and reuse it for run, session start, and (for prompt injection) future commands.

Agent configuration for durable mode

To make durable execution usable without writing Haskell, the agent JSON config needs to expose at least the following fields.

Minimal viable config

{
  "slug": "my-durable-agent",
  "executionMode": "asynchronous",
  "toolCallPolicy": {
    "default": "runSync",
    "rules": [
      {"tool": "bash_command", "disposition": {"tag": "defer", "reason": "approval required"}},
      {"tool": "fetch_remote", "disposition": {"tag": "runAsync"}}
    ]
  }
}

Proposed JSON schema additions to `System.Agents.Base.Agent`

data Agent = Agent
    { ...
    , executionMode :: Maybe ExecutionMode        -- default Synchronous
    , toolCallPolicyConfig :: Maybe ToolCallPolicyConfig
    }

Only executionMode and toolCallPolicyConfig are in scope for this milestone.

Tool-call policy config design

A declarative policy config that the runtime turns into a ToolCallPolicy:

data ToolCallPolicyConfig = ToolCallPolicyConfig
    { tpcDefaultDisposition :: ToolCallDisposition
    , tpcRules :: [ToolCallPolicyRule]
    }

data ToolCallPolicyRule = ToolCallPolicyRule
    { tprToolName :: Text           -- exact match
    , tprDisposition :: ToolCallDisposition
    }

The runtime combinator:

buildToolCallPolicy :: ToolCallPolicyConfig -> ToolCallPolicy
buildToolCallPolicy cfg _ctx call =
    fromMaybe cfg.tpcDefaultDisposition $
        find ((== callName call) . tprToolName) cfg.tpcRules

Where the policy is applied

In System.Agents.CLI.SessionDurable.withAgentForSession, after building the runtime agent from the OS tree, inspect the JSON Agent config:

let agent' = maybe agent (withExecutionMode Asynchronous) (executionMode jsonAgent)
let agent'' = maybe agent' (withToolCallPolicy . buildToolCallPolicy) (toolCallPolicyConfig jsonAgent)

This makes session start, session step, session pause, and session resume all use the same durable configuration.

Implementation steps

Step 1 — Shared prompt options

Refactor System.Agents.CLI.OneShot so that the prompt/media/thinking options are reusable:

data PromptScriptOptions = PromptScriptOptions
    { psoPromptScript :: PromptScript
    , psoMediaFiles :: [MediaReference]
    , psoThinkingOutput :: OneShot.ThinkingOutput
    }

parsePromptScriptOptions :: Parser PromptScriptOptions

Update OneShotOptions to contain PromptScriptOptions plus sessionFile. Update parseOneShotOptions accordingly.

Step 2 — Add durable config fields to JSON Agent

In System.Agents.Base.Agent:

Add executionMode :: Maybe ExecutionMode.
Add toolCallPolicyConfig :: Maybe ToolCallPolicyConfig with JSON instances.
Keep defaults backward-compatible (Nothing = synchronous, default policy).

Step 3 — Build policy from config

In System.Agents.CLI.SessionDurable:

Add buildToolCallPolicy / applyAgentDurableConfig.
Update withAgentForSession to read the JSON agent config and apply durable settings to the runtime agent.

Step 4 — Implement `session start`

In System.Agents.CLI.SessionDurable:

Add SessionStart PromptScriptOptions Bool command (the Bool is whether to run one step).
Implement handleStart:
1. Load agent tree and build runtime agent (with durable config).
2. Interpret the prompt script (aliases, files, shell, media).
3. Create a fresh Session (version 2, async mode).
4. Construct the initial UserTurn with the prompt/media.
5. If --step, call runStepM exactly once.
6. Persist the resulting session.
7. Print session id, current turn state, and any deferred tokens.

Step 5 — Implement `session step`

In System.Agents.CLI.SessionDurable:

Add SessionStep SessionId command.
Implement handleStep:
1. Load the existing session.
2. Load the agent and apply durable config.
3. Call runStepM exactly once.
4. Persist the resulting session.
5. Print the new turn state and any deferred tokens.

This is very close to the existing handlePause; consider sharing the implementation.

Step 6 — Wire the parsers

In app/Main.hs:

Add SessionStart and SessionStep constructors to SessionDurableCommand.
Add start and step subcommand parsers reusing parsePromptScriptOptions / parseSessionIdArgument.
Update help text: Operate durable sessions: start, step, pause, resume, ....

Step 7 — Tests

Unit tests for ToolCallPolicyConfig JSON parsing.
Unit tests for buildToolCallPolicy.
Integration test exercising the full iterative flow:
- session start -p "..." creates a session with a UserTurn;
- session step <id> produces an LlmTurn with tool calls;
- session step <id> runs sync tools and defers one call (PartialUserTurn);
- session pending <id> lists the deferred token;
- session complete <token> result.json injects the result;
- session step <id> processes the result and produces a final response.

Suggested first milestone

The smallest vertical slice that proves the design:

Extract shared PromptScriptOptions from run.
Add executionMode and toolCallPolicyConfig to the JSON Agent.
Implement buildToolCallPolicy and apply it in withAgentForSession.
Add agents session start that creates a session with an initial UserTurn and optionally runs one step.
Add agents session step for single-step iteration.
Write one integration test exercising the full flow above.

todos/durable-workflows.md — original design plan.
todos/durable-workflows.progress.md — implementation progress.
documentation/durable-workflows-howto.md — user-facing how-to.
src/System/Agents/CLI/SessionDurable.hs — existing session command handlers.
src/System/Agents/Session/Loop.hs — runAsync and runStepM.

OS Model Migration Guide

2026-10-04T15:53:54Z

Generated from documentation/MIGRATION-OS.md, the repository is the canonical source and may be ahead of this page.

OS Model Migration Guide

Complete migration guide for transitioning from the legacy Runtime-based architecture to the new ECS-based OS Model.

Overview

The Agents framework has migrated from a Runtime-per-agent model to an Entity-Component-System (ECS) architecture. This change provides:

Better concurrency control via STM-based component access
Resource lifecycle management with scoped cleanup
Unified agent/toolbox/conversation tracking in a single World
Subcall visibility for multi-agent hierarchies
Persistence layer with pluggable backends

Architecture Comparison

Legacy Runtime Model (Pre-OS)

┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
│   Agent A       │  │   Agent B       │  │   Agent C       │
│ ┌─────────────┐ │  │ ┌─────────────┐ │  │ ┌─────────────┐ │
│ │   Runtime   │ │  │ │   Runtime   │ │  │ │   Runtime   │ │
│ │ ┌─────────┐ │ │  │ │ ┌─────────┐ │ │  │ │ ┌─────────┐ │ │
│ │ │ Session │ │ │  │ │ │ Session │ │ │  │ │ │ Session │ │ │
│ │ │ Tools   │ │ │  │ │ │ Tools   │ │ │  │ │ │ Tools   │ │ │
│ │ │ LLM     │ │ │  │ │ │ LLM     │ │ │  │ │ │ LLM     │ │ │
│ │ └─────────┘ │ │  │ │ └─────────┘ │ │  │ │ └─────────┘ │ │
│ └─────────────┘ │  │ └─────────────┘ │  │ └─────────────┘ │
└─────────────────┘  └─────────────────┘  └─────────────────┘
       │                    │                    │
       └────────────────────┼────────────────────┘
                            │
                    ┌───────▼───────┐
                    │  AgentTree    │
                    │  (references) │
                    └───────────────┘

New OS Model (ECS)

┌─────────────────────────────────────────────────────────────┐
│                           OS                                │
│  ┌───────────────────────────────────────────────────────┐  │
│  │                        World                          │  │
│  │  ┌─────────────┐ ┌─────────────┐ ┌─────────────────┐  │  │
│  │  │ AgentStore  │ │ToolboxStore │ │ ConversationStore│  │  │
│  │  │  (TVar)     │ │  (TVar)     │ │    (TVar)       │  │  │
│  │  ├─────────────┤ ├─────────────┤ ├─────────────────┤  │  │
│  │  │ AgentConfig │ │ToolboxConfig│ │ ConversationCfg │  │  │
│  │  │ AgentState  │ │ToolboxState │ │ ConversationState│  │  │
│  │  │ ...         │ │ ...         │ │ ...             │  │  │
│  │  └─────────────┘ └─────────────┘ └─────────────────┘  │  │
│  └───────────────────────────────────────────────────────┘  │
│  ┌───────────────────────────────────────────────────────┐  │
│  │                   ResourceRegistry                    │  │
│  │    (SQLite, Lua, HTTP pools with scoped cleanup)      │  │
│  └───────────────────────────────────────────────────────┘  │
│  ┌───────────────────────────────────────────────────────┐  │
│  │                   EventQueue (TQueue)                 │  │
│  │          (OSEvents for TUI visibility)                │  │
│  └───────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Migration Phases

The migration was implemented in three phases:

PhaseOldOnly ──► PhaseDual ──► PhaseNewOnly
(removed)        (default)     (available)

PhaseOldOnly: Only legacy Runtime (removed as of March 2026)
PhaseDual: Both Runtime and OS available (current default)
PhaseNewOnly: Only OS Model (available for testing)

Key Changes

1. Agent Creation

Legacy:

import System.Agents.Runtime

-- Create runtime directly
runtime <- newRuntime tracer config tools

New (OS Model):

import System.Agents.OS

-- Initialize OS
os <- initializeOS defaultOSConfig

-- Create agent in OS
agentId <- createAgent os AgentConfig
    { agentName = "my-agent"
    , agentModel = ModelConfig "openai" "..." "gpt-4" "key1"
    , agentSystemPrompt = "You are helpful"
    , agentToolboxBindings = []
    }

2. Toolbox Registration

Legacy:

-- Tools passed to Runtime constructor
let tools = bashTools ++ mcpTools ++ ioTools
runtime <- newRuntime tracer config tools

New:

-- Toolboxes created as entities in the World
bashId <- createToolbox os bashConfig
sqlId <- createToolbox os sqliteConfig

-- Bound to agent via agentToolboxBindings
let agentConfig = AgentConfig
    { ...
    , agentToolboxBindings = 
        [ unToolboxId bashId
        , unToolboxId sqlId
        ]
    }

3. Resource Management

Legacy:

-- Manual resource tracking
connections <- newTVarIO []
-- Cleanup handled per-runtime

New:

import System.Agents.OS.Resources

-- Create registry
registry <- atomically newResourceRegistry
let ctx = ResourceContext [ProgramScope] registry

-- Create scoped resource
rid <- createResource ctx (SqliteResource config) $ \rid -> do
    conn <- openConnection config
    pure ResourceHandle
        { handleId = rid
        , handleCleanup = closeConnection conn
        , handleAccess = \f -> f (SqliteAccessor conn)
        }

-- Automatic cleanup
cleanupScope registry (AgentScope agentId)

4. Conversation Tracking

Legacy:

-- Session managed separately
session <- loadSession sessionPath
-- Conversation ID tracked in Session type

New:

-- Conversation is a first-class entity
convId <- createEntity
atomically $ do
    setComponent world convId ConversationConfig{...}
    setComponent world convId ConversationState{...}

-- Link to agent
atomically $ modifyComponent @AgentState world (unAgentId agentId) $
    \s -> s { agentCurrentConversation = Just (ConversationId convId) }

5. Subcall Visibility

Legacy:

-- Subcalls invisible to TUI
result <- turnAgentRuntimeIntoIOTool agent prompt
-- No visibility into subcall conversation

New:

-- Subcalls visible as OSEvents
result <- turnAgentRuntimeIntoIOTool agent prompt
-- OSEvent_SubcallStarted, OSEvent_SubcallProgress, 
-- OSEvent_SubcallCompleted emitted
-- TUI displays subcall as separate conversation

Compatibility Layer

The System.Agents.OS.Compat.Runtime module provides a compatibility bridge:

import System.Agents.OS.Compat.Runtime

-- Run existing Runtime code with OS backend
withCompatOS $ \compat -> do
    -- Use compatRuntime for legacy code
    let runtime = compatRuntime compat
    
    -- Or use OS directly
    let os = compatOS compat
    
    -- Both are kept in sync

Compatibility Patterns

Pattern 1: Runtime Bridge

bridge <- newRuntimeBridge agentId os
runWithBridge bridge $ do
    -- Legacy Runtime code
    tools <- listTools
    callTool "my-tool" args

Pattern 2: Dual Initialization

-- Initialize both
runtime <- newRuntime tracer config tools
os <- runtimeToOS runtime

-- Use whichever is appropriate
useRuntime runtime  -- Legacy code
useOS os            -- New code

Pattern 3: Migration Wrapper

-- Wrap Runtime operations with OS event emission
wrappedCallTool ctx tool args = do
    -- Emit subcall events if in OS context
    case ctxEventQueue ctx of
        Just q -> emitSubcallStart q
        Nothing -> pure ()
    
    -- Execute tool
    result <- callTool tool args
    
    -- Emit completion
    case ctxEventQueue ctx of
        Just q -> emitSubcallComplete q result
        Nothing -> pure ()
    
    pure result

Module Mapping

Legacy Module	New OS Module	Purpose
`System.Agents.Runtime`	`System.Agents.OS`	Main entry point
`System.Agents.Runtime.Runtime`	`System.Agents.OS.Agents`	Agent operations
`System.Agents.Session.Base`	`System.Agents.OS.Conversation`	Conversation tracking
`System.Agents.Session.Types`	`System.Agents.OS.Conversation.Types`	Conversation types
`System.Agents.AgentTree`	`System.Agents.OS.AgentTree`	Multi-agent hierarchy
(new)	`System.Agents.OS.Core`	ECS core types
(new)	`System.Agents.OS.Resources`	Resource management
(new)	`System.Agents.OS.Concurrent`	Concurrent access
(new)	`System.Agents.OS.Events`	OS event system

Type Mapping

Legacy Type	New Type	Notes
`Runtime`	`OS`	Contains World, not just config
`Session`	`Conversation` + `Turn` + `ToolCall`	Split into components
`AgentId`	`AgentId`	Now wraps `EntityId`
`SessionId`	`ConversationId`	Now wraps `EntityId`
(new)	`EntityId`	Base identifier type
(new)	`ToolboxId`	Phantom-typed EntityId
(new)	`TurnId`	Phantom-typed EntityId
(new)	`ToolCallId`	Phantom-typed EntityId
(new)	`World`	ECS world container

Step-by-Step Migration

Step 1: Update Imports

-- Before
import System.Agents.Runtime
import qualified System.Agents.Runtime.Runtime as Runtime

-- After
import System.Agents.OS
import qualified System.Agents.OS.Agents as OSAgents

Step 2: Initialize OS Instead of Runtime

-- Before
runtime <- newRuntime tracer config tools

-- After
os <- initializeOS defaultOSConfig

-- Create agent
agentId <- createAgent os agentConfig

-- Create toolboxes and bind to agent
toolboxId <- createToolbox os toolboxConfig

Step 3: Update Agent Configuration

-- Before
config = AgentConfig
    { slug = "my-agent"
    , toolDirectory = "./tools"
    , ...
    }

-- After
config = AgentConfig
    { agentName = "my-agent"
    , agentModel = ...
    , agentSystemPrompt = ...
    , agentToolboxBindings = [unToolboxId bashId, unToolboxId sqliteId]
    }

Step 4: Use Resource Context

-- Before
conn <- openConnection config
-- Cleanup handled manually

-- After
rid <- createResource ctx (SqliteResource config) $ \rid -> do
    conn <- openConnection config
    pure ResourceHandle
        { handleId = rid
        , handleCleanup = closeConnection conn
        , handleAccess = \f -> f (SqliteAccessor conn)
        }

-- Automatic cleanup on scope exit
cleanupScope registry scope

Step 5: Update Subcall Handling

-- Before
result <- turnAgentRuntimeIntoIOTool agent prompt

-- After
-- Ensure ToolExecutionContext has event queue
let ctx' = ctx { ctxEventQueue = Just eventQueue }
result <- turnAgentRuntimeIntoIOTool agent prompt
-- Events automatically emitted

Testing Migration

Unit Tests

-- Test OS operations directly
spec :: Spec
spec = describe "OS Core" $ do
    it "creates agents" $ do
        os <- initializeOS testConfig
        agentId <- createAgent os testAgentConfig
        
        mConfig <- atomically $ 
            getComponent @AgentConfig (osWorld os) (unAgentId agentId)
        
        mConfig `shouldSatisfy` isJust

Integration Tests

-- Test full agent lifecycle
spec :: Spec
spec = describe "Agent Lifecycle" $ do
    it "runs conversation" $ do
        os <- initializeOS testConfig
        agentId <- createAgent os testAgentConfig
        
        -- Create conversation
        convId <- startConversation os agentId
        
        -- Add user message
        addMessage os convId UserRole "Hello!"
        
        -- Run agent turn
        result <- runAgentTurn os agentId convId
        
        -- Verify response
        messages <- getConversationMessages os convId
        length messages `shouldBe` 2

Troubleshooting

Issue: “Entity not found” errors

Cause: Trying to access entity that doesn’t exist in World.

Solution: Check entity creation and component registration:

-- Verify entity exists
exists <- atomically $ entityExists world eid
when (not exists) $ error "Entity not found"

-- Check component store registered
hasStore <- atomically $ hasComponentStore @MyComponent world
when (not hasStore) $ error "Component store not registered"

Issue: STM deadlock

Cause: Nested STM transactions with external IO.

Solution: Use atomically only for pure STM, run IO separately:

-- Bad: IO inside atomically
atomically $ do
    val <- getComponent world eid
    result <- liftIO $ expensiveOp val  -- DON'T DO THIS
    setComponent world eid result

-- Good: Separate IO
mVal <- atomically $ getComponent world eid
case mVal of
    Just val -> do
        result <- expensiveOp val
        atomically $ setComponent world eid result
    Nothing -> pure ()

Issue: Resource leaks

Cause: Not cleaning up resources properly.

Solution: Use bracket or ResourceT:

import Control.Exception (bracket)

-- Using bracket
bracket 
    (createResource ctx config mkHandle)
    (\rid -> cleanupResource registry rid)
    (\rid -> do
        withResource registry rid $ \accessor ->
            useResource accessor
    )

-- Or use cleanupScope for batch cleanup
cleanupScope registry (AgentScope agentId)

Issue: Subcall not visible in TUI

Cause: Event queue not configured in ToolExecutionContext.

Solution: Ensure context includes event queue:

-- Check if event queue exists
case ctxEventQueue ctx of
    Nothing -> putStrLn "Warning: No event queue, subcall won't be visible"
    Just q -> putStrLn "Event queue configured"

-- When creating context, include event queue
ctx = ToolExecutionContext
    { ...
    , ctxEventQueue = Just eventQueue  -- Required for visibility
    }

Performance Considerations

STM Contention

Problem: Many threads competing for same TVar.

Solution: Use granular component stores:

-- Good: Separate TVars per component type
agentStore :: TVar (Map EntityId AgentConfig)
toolboxStore :: TVar (Map EntityId ToolboxConfig)

-- Bad: Single TVar for everything
allComponents :: TVar (Map EntityId (AgentConfig, ToolboxConfig, ...))

Resource Pool Sizing

Problem: Too few connections in pool causes contention.

Solution: Size pools based on expected concurrency:

-- HTTP pool: size = expected concurrent requests
httpRid <- createHttpPool ctx 20

-- SQLite with WAL: can have many readers
sqliteRid <- createSqliteResource ctx config True  -- WAL mode

Snapshot Frequency

Problem: Too frequent snapshots impact performance.

Solution: Configure snapshot intervals:

config = defaultOSConfig
    { osSnapshotInterval = Just (secondsToNominalDiffTime 60)
    , osMaxSnapshotsPerEntity = 10
    }

Future Directions

Query System

Planned: ECS-style queries for efficient entity filtering:

-- Query all busy agents
busyAgents <- query world $ 
    select @AgentState 
    where_ (\s -> agentStatus s == AgentBusy)

-- Query conversations with recent activity
recentConvs <- query world $
    select @ConversationState
    where_ (\s -> conversationLastActivity s > cutoffTime)

Distributed OS

Planned: Multi-process OS with shared state:

-- Initialize distributed OS
os <- initializeDistributedOS 
    { osNodeId = "node-1"
    , osCluster = ["node-1", "node-2", "node-3"]
    , osConsensus = RaftConsensus
    }

Persistence Evolution

Planned: Event sourcing for complete history:

-- Replay from event log
events <- loadEvents backend
world <- foldM applyEvent emptyWorld events

-- Where applyEvent applies each event to world state

References

OS API Reference - Complete API documentation
Architecture Overview - System architecture
CHANGELOG-OS-MIGRATION.md - Detailed change log

Summary

The OS Model migration provides:

Better concurrency via STM-based ECS
Resource safety via scoped lifecycle management
Visibility via OS events for subcalls
Flexibility via pluggable persistence
Future-proofing via clean separation of concerns

For new code, use the OS Model directly. For legacy code, use the compatibility layer during migration.

Tool System

2026-10-04T15:53:54Z

Generated from documentation/tools.md, the repository is the canonical source and may be ahead of this page.

Tool System

The tool system provides agents with the ability to execute external commands, call APIs, and interact with other agents. Tools are dynamically registered and exposed to the LLM via the OpenAI function calling API.

Overview

┌────────────────────────────────────────────────────────────────┐
│                       Tool System                               │
├────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐            │
│  │ Bash Tools  │  │  MCP Tools  │  │ OpenAPI     │            │
│  │ (scripts)   │  │ (servers)   │  │ (REST APIs) │            │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘            │
│         │                │                │                    │
│         └────────────────┴────────────────┘                    │
│                          │                                      │
│                   ┌──────▼──────┐                              │
│                   │  Toolbox    │                              │
│                   │  (merging)  │                              │
│                   └──────┬──────┘                              │
│                          │                                      │
│                   ┌──────▼──────┐                              │
│                   │ ToolRegistration│                           │
│                   │ (LLM schema) │                              │
│                   └─────────────┘                              │
└────────────────────────────────────────────────────────────────┘

Tool Types

The framework supports multiple tool types:

Tool Type	Description	Use Case
Bash Tools	Executable scripts	External commands, system integration
MCP Tools	MCP servers	Standardized tool protocols
OpenAPI Tools	REST APIs	API integrations
PostgREST Tools	Database endpoints	Database queries
SQLite Tools	SQLite databases	Local SQL queries
System Tools	System information	Runtime context, session introspection, command execution
Developer Tools	Development utilities	File editing with multi-turn sessions, agent validation/creation
IO Tools	Haskell functions	In-process operations
Lua Tools	Lua scripts	Embedded scripting
Skills	Progressive disclosure	Procedural knowledge

The framework supports media attachments and multi-modal responses for LLM interactions.

Media Type Classification

-- System.Agents.Media.Types
data MediaType
    = MediaImage ImageType
    | MediaAudio AudioType
    | MediaVideo VideoType
    | MediaApplication ApplicationType
    | MediaText TextSubtype

data ImageType = ImagePNG | ImageJPEG | ImageGIF | ImageWebP | ImageSVG
data AudioType = AudioMPEG | AudioWAV | AudioOGG | AudioMP3 | AudioAAC | AudioFLAC
data VideoType = VideoMP4 | VideoWebM | VideoOGG | VideoAVI | VideoMOV
data ApplicationType = AppPDF | AppJSON | AppXML | AppOctetStream | AppZip
data TextSubtype = TextPlain | TextHTML | TextCSS | TextCSV | TextMarkdown

Media Attachments

data MediaAttachment = MediaAttachment
    { mediaMimeType :: Text        -- e.g., "image/png"
    , mediaBase64Data :: Text      -- Base64-encoded content
    , mediaFilename :: Maybe Text  -- Optional filename
    }

Content Parts for Mixed Responses

data ContentPart
    = TextPart Text
    | MediaPart MediaAttachment

Declaring Media Output in Scripts

Tools can declare their output media type in the describe output:

#!/bin/bash

if [ "$1" == "describe" ]; then
    echo '{
        "slug": "generate_chart",
        "description": "Generates a chart image",
        "args": [...],
        "output-media-type": "image/png"
    }'
    exit 0
fi

# Generate and output PNG image to stdout
./generate-chart "$@"

Supported media types:

Images: image/png, image/jpeg, image/gif, image/webp, image/svg+xml
Audio: audio/mpeg, audio/wav, audio/ogg, audio/mp3, audio/aac, audio/flac
Video: video/mp4, video/webm, video/ogg, video/avi, video/quicktime
Documents: application/pdf, application/json, application/xml, application/zip
Generic: application/octet-stream

Tool Result Types with Media

data CallResult call
    = -- | Successful execution with optional media type hint
      BlobToolSuccess call ByteString (Maybe MediaType)
    | -- ... other constructors

data UserToolResponse
    = TextResponse Text                    -- Plain UTF-8 text
    | JsonResponse Aeson.Value             -- Structured JSON data
    | MediaResponse MediaAttachment        -- Single binary media
    | MixedResponse [ContentPart]          -- Multi-modal content

Tool Registration

All tools are registered using the ToolRegistration type:

data ToolRegistration = ToolRegistration
    { innerTool :: Tool ()
    , declareTool :: OpenAI.Tool
    , findTool :: OpenAI.ToolCall -> Maybe (Tool OpenAI.ToolCall)
    }

Registration Flow

Tool sources (bash, MCP, OpenAPI, System, Developer) generate ToolRegistration values
Registrations are combined into a single list
The list is passed to the LLM API as available functions
When the LLM calls a function, the executor is invoked

Bash Tools

Bash tools are executable scripts stored in the agent’s tool directory.

Script Requirements

Scripts must support a describe subcommand that outputs JSON:

#!/bin/bash

if [ "$1" == "describe" ]; then
    echo '{
        "slug": "my-tool",
        "description": "What this tool does",
        "args": [
            {
                "name": "input",
                "description": "Input parameter",
                "type": "string",
                "backing_type": "string",
                "arity": "single",
                "mode": "dashdashspace"
            }
        ],
        "empty-result": {"tag": "AddMessage", "contents": "No results"}
    }'
    exit 0
fi

# Main execution
input="$1"
echo "Result for: $input"

ScriptInfo Schema

data ScriptInfo = ScriptInfo
    { scriptArgs :: [ScriptArg]
    , scriptSlug :: Text
    , scriptDescription :: Text
    , scriptEmptyResultBehavior :: Maybe EmptyResultBehavior
    , scriptOutputMediaType :: Maybe Text  -- NEW: Media type for binary output
    }

data ScriptArg = ScriptArg
    { argName :: Text
    , argDescription :: Text
    , argType :: Text
    , argBackingType :: Text
    , argArity :: Text  -- "single", "optional", "multiple"
    , argMode :: Text   -- "dashdashspace", "space", etc.
    }

Environment Variables

When a bash tool runs, it receives context via environment variables:

Variable	Description
`AGENT_SESSION_ID`	Current session UUID
`AGENT_CONVERSATION_ID`	Conversation UUID
`AGENT_TURN_ID`	Current turn UUID
`AGENT_AGENT_ID`	Agent UUID (if available)
`AGENT_SESSION_JSON`	Full session as JSON

A FileSystemDirectory/SingleTool toolbox description can also carry a Bindings list, tying an argument to a fixed value or to an agent parameter — the argument disappears from the tool’s schema entirely, and the model never sees or chooses the value. A parameter’s value can also be passed as an environment variable only ("mode": "env"), the one way to bind a secret parameter safely. See Parameters, Bindings & Narrowing Sub-Agents.

Bash Toolbox

The BashToolbox module manages script discovery and execution:

data Toolbox = Toolbox
    { tools :: BackgroundVal [ToolRegistration]
    , triggerReload :: STM Bool
    }

initializeBackroundToolbox :: 
    Tracer IO Trace -> 
    FilePath -> 
    IO (Either ToolboxError Toolbox)

Features:

Hot reloading: File changes trigger automatic reload
Background thread: Non-blocking tool discovery
Error isolation: Failed scripts don’t break other tools

MCP Tools

Model Context Protocol (MCP) tools connect to external servers that provide dynamic tool listings.

MCP Server Types

data McpServerDescription
    = McpSimpleBinary McpSimpleBinaryConfiguration
    
data McpSimpleBinaryConfiguration = McpSimpleBinaryConfiguration
    { name :: Text
    , executable :: FilePath
    , args :: [Text]
    }

MCP Client Runtime

data Runtime = Runtime
    { procHandle :: ProcessHandle
    , stdinHandle :: Handle
    , stdoutHandle :: Handle
    , toolsList :: TVar [ToolDescription]
    , callResults :: TVar (Map CallId Value)
    }

MCP Tool Flow

1. Start MCP server process
2. Initialize connection
3. Query available tools
4. Register tools with LLM
5. On LLM call:
   a. Send tool_call message to MCP server
   b. Wait for response
   c. Return result to LLM

MCP Protocol Messages

// Tool list request
{"jsonrpc": "2.0", "method": "tools/list", "id": 1}

// Tool list response
{"jsonrpc": "2.0", "result": 
  {"tools": [
    {"name": "read_file", 
     "description": "Read a file", 
     "inputSchema": {...}}
  ]}, 
  "id": 1}

// Tool call
{"jsonrpc": "2.0", 
 "method": "tools/call", 
 "params": {"name": "read_file", "arguments": {"path": "/tmp/foo"}},
 "id": 2}

OpenAPI Tools

OpenAPI tools convert REST API specifications into LLM-callable tools.

OpenAPI Server Configuration

{
  "mcpServers": [...],
  "openApiToolboxes": [
    {
      "tag": "OpenAPIServer",
      "contents": {
        "SpecUrl": "https://api.example.com/openapi.json",
        "BaseUrl": "https://api.example.com",
        "Headers": {"X-API-Version": "v1"},
        "Token": "${API_TOKEN}"
      }
    }
  ]
}

This description can also carry a Bindings list, same as a bash toolbox — see Parameters, Bindings & Narrowing Sub-Agents.

Conversion Process

-- Load and parse OpenAPI spec
convertOpenAPIToTools :: OpenAPISpec -> [OpenAPITool]

-- Convert operation to tool
convertOperation :: Path -> Method -> Operation -> OpenAPITool

-- Build tool parameters from OpenAPI parameters
buildToolParameters :: Operation -> ToolParameters

-- Convert to OpenAI tool format
toOpenAITool :: OpenAPITool -> OpenAI.Tool

Schema Resolution

The OpenAPI module handles $ref references:

resolveSchema :: Schema -> Components -> Schema
dereferenceSpec :: OpenAPISpec -> OpenAPISpec

Supports:

Internal references (#/components/schemas/Foo)
Nested references
Array item references
anyOf/allOf compositions
Circular reference detection

Name Normalization

OpenAPI operation IDs are normalized for LLM compatibility:

-- Original: "pets.getById"
-- Normalized: "pets_getById"

-- Original: "/users/{id}/posts"
-- Normalized: "_users__id__posts"

The NameMapping type tracks bidirectional mapping:

data NameMapping = NameMapping
    { nmOriginal :: Text
    , nmNormalized :: Text
    }

PostgREST Tools

PostgREST tools generate database query tools from PostgREST APIs.

Configuration

{
  "postgrestToolboxes": [
    {
      "tag": "PostgRESTServer",
      "contents": {
        "SpecUrl": "http://localhost:3000/",
        "BaseUrl": "http://localhost:3000"
      }
    }
  ]
}

This description can also carry a Bindings list, same as a bash toolbox — see Parameters, Bindings & Narrowing Sub-Agents.

Generated Tools

For each table endpoint, the following tools are generated:

HTTP Method	Tool Name Pattern	Purpose
GET	`postgrest_{name}_get_{table}`	Query with filters
POST	`postgrest_{name}_post_{table}`	Insert rows
PUT	`postgrest_{name}_put_{table}`	Update rows
PATCH	`postgrest_{name}_patch_{table}`	Partial update
DELETE	`postgrest_{name}_delete_{table}`	Delete rows

Tool Parameters

PostgREST tools use structured parameter groups:

{
  "filters": {
    "column_name": "filter_value"
  },
  "subset": {
    "limit": 10,
    "offset": 0,
    "columns": "id,name,email"
  },
  "ranking": {
    "order": "created_at.desc"
  },
  "body": {
    "name": "New Item",
    "value": 42
  }
}

SQLite Tools

SQLite tools provide SQL query capabilities against SQLite databases.

Configuration

{
  "builtinToolboxes": [
    {
      "tag": "SqliteToolbox",
      "contents": {
        "Name": "analytics",
        "Description": "Analytics database",
        "Versioning": {"tag": "SqliteReadOnly", "path": "./analytics.db"}
      }
    }
  ]
}

Tool Interface

Each SQLite toolbox exposes a single sqlite_{name}_query tool:

{
  "sql": "SELECT * FROM users WHERE active = 1 LIMIT 10"
}

Parameters:

sql (string, required): SQL query to execute

Security:

Read-only queries are encouraged
Write operations are allowed but logged
No DDL by default (configurable)

File Sandbox System

The file sandbox system provides secure, configurable file access control for tools that need to read from or write to the filesystem. It is used by the System Toolbox (for attach-file), Developer Toolbox (for read-file-range, write-file-range, patch-file), and Lua Toolbox (for the fs module).

Overview

┌─────────────────────────────────────────────────────────────────┐
│                    File Sandbox System                           │
├─────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐             │
│  │ SystemToolbox│  │DeveloperToolbox│  │ LuaToolbox │             │
│  │ (attach-file)│  │(read/write/   │  │ (fs.*)     │             │
│  │              │  │ patch-file)   │  │            │             │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘             │
│         │                 │                  │                    │
│         └─────────────────┼──────────────────┘                    │
│                           │                                       │
│                    ┌──────▼──────┐                                │
│                    │ FileSandbox │                                │
│                    │  (validate) │                                │
│                    └──────┬──────┘                                │
│                           │                                       │
│                    ┌──────▼──────┐                                │
│                    │PathPredicate│                                │
│                    │  (access)   │                                │
│                    └─────────────┘                                │
└─────────────────────────────────────────────────────────────────┘

FileSandboxConfig

Each sandboxed toolbox accepts an optional FileSandbox configuration, either inline (below) or as a reference to a named sandbox:

data FileSandboxConfig = FileSandboxConfig
    { fsbPredicate :: PathPredicate     -- Defines allowed file paths
    , fsbMaxFileSize :: Maybe Integer   -- Max file size in bytes (Nothing = no limit)
    , fsbName :: Maybe Text             -- Human-readable name for the sandbox
    }

Default Configuration (Secure by Default):

defaultFileSandboxConfig :: FileSandboxConfig
defaultFileSandboxConfig = FileSandboxConfig
    { fsbPredicate = AlwaysDeny          -- Deny all access by default
    , fsbMaxFileSize = Just (50 * 1024 * 1024)  -- 50MB default limit
    , fsbName = Nothing
    }

Named Sandboxes

A sandbox can be written once, under a name, in the agent’s fileSandboxes map, and referenced from any builtin toolbox of that agent with {"ref": "<name>"} in place of the inline definition:

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "coder",
    "fileSandboxes": {
      "project": {
        "fsbPredicate": {"tag": "DirectoryRecursive", "contents": "./src"},
        "fsbMaxFileSize": 10485760
      },
      "scratch": {
        "fsbPredicate": {"tag": "DirectoryRecursive", "contents": "/tmp/scratch"}
      }
    },
    "builtinToolboxes": [
      {"tag": "DeveloperToolbox", "contents": {
        "Name": "dev", "Description": "Development tools",
        "Capabilities": ["read-file-range", "patch-file"],
        "FileSandbox": {"ref": "project"}}},
      {"tag": "LuaToolbox", "contents": {
        "Name": "lua", "Description": "Lua orchestration",
        "MaxMemoryMB": 256, "MaxExecutionTimeSeconds": 300,
        "AllowedTools": [], "AllowedHosts": [],
        "FileSandbox": {"ref": "project"}}},
      {"tag": "SystemToolbox", "contents": {
        "Name": "system", "Description": "System context",
        "Capabilities": ["attach-file"],
        "FileSandbox": {"ref": "scratch"}}}
    ]
  }
}

The value of FileSandbox is therefore one of two things:

Form	JSON	Meaning
Inline	`{"fsbPredicate": ..., "fsbMaxFileSize": ..., "fsbName": ...}`	The sandbox itself, as before. Existing agent files need no change.
Reference	`{"ref": "project"}`	The sandbox declared as `project` in this agent's `fileSandboxes`.

Rules:

Resolution happens when the agent’s toolboxes are loaded. Each reference is replaced by the named definition, and the toolbox then behaves exactly as if the definition had been written inline.
An undeclared name is an error, not an empty sandbox: the agent fails to load with a message naming the toolbox and the missing sandbox, and validate-agent reports it too.
ref stands alone. An object carrying both ref and inline fields is refused, so that a sandbox is never silently widened or narrowed at the place it is used.
The name is the default fsbName. A referenced sandbox that sets no fsbName is named after its key in fileSandboxes.
Each toolbox gets its own sandbox instance built from the shared definition: the rules are shared, nothing else is.
fileSandboxes is optional; an agent without it loads as before.

What this does and does not do. Naming a sandbox changes where the rules are written, not what is enforced. The enforcement is the one described in this section: a path predicate and a size limit checked by the builtin tools that take a FileSandbox (attach-file and list-directory in the System Toolbox, the file capabilities of the Developer Toolbox, the Lua fs module). It is not an operating-system sandbox: bash tools, MCP servers, execute-command and build-command run as ordinary processes and are not confined by it.

Scope. fileSandboxes belongs to one agent file; a name declared in one agent is not visible from another, including its sub-agents. To share one definition across several agent files, use an agent template library: define the sandbox once there and either pass it inline to each toolbox or put it in the fileSandboxes of each agent.

PathPredicate DSL

The PathPredicate DSL provides a rich, composable language for defining file access permissions:

data PathPredicate
    = FileExactly FilePath              -- Exact file match
    | DirectoryExactly FilePath         -- Exact directory match (contents not included)
    | DirectoryRecursive FilePath       -- Directory and all subdirectories
    | DirectoryShallow FilePath         -- Directory contents only (not subdirectories)
    | FilePattern String                -- Glob pattern on filename (*, ? wildcards)
    | FileExtension [String]            -- File extension whitelist (no leading dot)
    | FileSizeLessThan Integer          -- Maximum file size in bytes
    | ChildOf FilePath                  -- Path within a parent directory
    | And PathPredicate PathPredicate   -- Logical AND
    | Or PathPredicate PathPredicate    -- Logical OR
    | Not PathPredicate                 -- Logical NOT
    | Any [PathPredicate]               -- OR of all predicates
    | All [PathPredicate]               -- AND of all predicates
    | AlwaysAllow                       -- Allow all (use sparingly)
    | AlwaysDeny                        -- Deny all (secure default)

Predicate Examples

Allow specific files and directories:

{
  "fsbPredicate": {
    "tag": "Any",
    "contents": [
      {"tag": "DirectoryRecursive", "contents": "./src"},
      {"tag": "FileExactly", "contents": "./package.yaml"},
      {"tag": "FileExactly", "contents": "./README.md"}
    ]
  }
}

Allow only Haskell source files under 1MB:

{
  "fsbPredicate": {
    "tag": "And",
    "contents": [
      {"tag": "DirectoryRecursive", "contents": "./src"},
      {"tag": "FileExtension", "contents": ["hs", "lhs"]},
      {"tag": "FileSizeLessThan", "contents": 1048576}
    ]
  }
}

Allow any file in project except build artifacts:

{
  "fsbPredicate": {
    "tag": "And",
    "contents": [
      {"tag": "DirectoryRecursive", "contents": "./my-project"},
      {"tag": "Not", "contents": {"tag": "DirectoryRecursive", "contents": "./my-project/dist"}}
    ]
  }
}

Glob pattern matching:

{
  "fsbPredicate": {
    "tag": "Any",
    "contents": [
      {"tag": "FilePattern", "contents": "*.md"},
      {"tag": "FilePattern", "contents": "test-*.json"}
    ]
  }
}

Path Handling

All paths are canonicalized before validation:

Symlinks are resolved - The predicate is applied to the canonical path
Relative paths are resolved - Relative paths in predicates are resolved relative to the current working directory
Path normalization - .. and . components are resolved, duplicate slashes removed

Important: The predicate applies to the canonical path, not the symlink path. This prevents symlink traversal attacks where a symlink inside an allowed directory points outside the sandbox.

Security Features

Feature	Description
Default Deny	All access is denied unless explicitly allowed (`AlwaysDeny` is default)
Immutable	Sandboxes cannot be modified after creation
Canonicalization	All paths are canonicalized before validation (resolves symlinks, `..`, etc.)
Size Limits	Optional file size limits prevent resource exhaustion
Composable	Predicates can be combined with logical operators for complex rules

System Toolbox (Builtin)

The System Toolbox provides agents with contextual information about the running system through a configurable set of capabilities.

The get-tool-call-status, list-running-tool-calls and cancel-tool-call capabilities only do something for agents that run tool calls in the background; see async-tool-calls.md.

Capabilities

Capability	Description
`date`	Current UTC/local time and timezone info
`operating-system`	OS name, version, kernel, architecture
`env-vars`	Filtered environment variables
`running-user`	Username, UID, GID, home, shell
`hostname`	Machine hostname
`working-directory`	Current working directory
`process-info`	Process ID, parent PID, process name
`uptime`	System uptime
`attach-file`	Attach a file to the conversation
`list-directory`	List directory contents with metadata
`execute-command`	Execute shell commands with optional filter approval
`get-tool-call-status`	Status, progress and result of one of the agent's own tool calls
`list-running-tool-calls`	Tool calls still running in the background
`cancel-tool-call`	Stop a running background tool call
`list-sessions`	List accessible sessions (requires session introspection config)
`search-sessions`	Full-text search across sessions (requires session introspection config)
`read-session`	Read session content (requires session introspection config)
`get-session-stats`	Get session statistics (requires session introspection config)

Configuration

{
  "builtinToolboxes": [
    {
      "tag": "SystemToolbox",
      "contents": {
        "Name": "system",
        "Description": "System context and information",
        "Capabilities": ["date", "operating-system", "running-user", "hostname", "attach-file", "list-directory", "execute-command"],
        "EnvVarFilter": null,
        "FileSandbox": {
          "fsbPredicate": {"tag": "DirectoryRecursive", "contents": "./project"},
          "fsbMaxFileSize": 52428800,
          "fsbName": "system-sandbox"
        },
        "CommandFilter": "/path/to/approval-script.sh"
      }
    }
  ]
}

Attach-File Capability

The attach-file capability allows the agent to attach files to the conversation for multi-modal LLM interactions:

-- Tool accepts:
{
  "capability": "attach-file",
  "filepath": "/path/to/image.png"
}

-- Returns:
-- MediaAttachment with base64-encoded content

Supported file types:

Images: PNG, JPEG, GIF, WEBP, SVG
Audio: MP3, WAV, OGG, AAC, FLAC
Video: MP4, WEBM, MOV, AVI
Documents: PDF, JSON, XML
Generic: Any file (as octet-stream)

Limits:

Maximum file size: 50MB

Sandbox Behavior: When attach-file is enabled, a file sandbox must be configured to specify which files can be attached. If no sandbox is configured, the capability will deny all file access.

List-Directory Capability

The list-directory capability lists directory contents with metadata:

// Input:
{
  "capability": "list-directory",
  "path": "./src",
  "recursive": false,
  "include_hidden": false
}

// Output:
{
  "path": "./src",
  "entries": [
    {"name": "Main.hs", "type": "file", "size": 1234, "modified": "2024-01-15T10:30:00Z"},
    {"name": "Utils", "type": "directory", "size": 0, "modified": "2024-01-14T08:00:00Z"}
  ],
  "total": 2
}

Execute-Command Capability

The execute-command capability allows the agent to execute arbitrary shell commands with an optional approval filter:

// Input:
{
  "capability": "execute-command",
  "command": "ls -la"
}

// Output:
{
  "command": "ls -la",
  "exitCode": 0,
  "stdout": "...",
  "stderr": "",
  "duration": 0.123
}

Command Filter: When CommandFilter is configured, every command is first passed to the filter on stdin. The filter must output a JSON acceptance object:

// Allowed:
{"acceptance": "allowed", "note": "Safe command"}

// Refused:
{"acceptance": "refused", "note": "Command contains dangerous operations"}

Any non-JSON output, missing fields, or non-zero exit code rejects the command.

Session Introspection Capabilities

The System Toolbox supports session introspection capabilities that allow agents to query, search, and read other sessions from the session store. This enables cross-session analysis and context sharing.

Session Introspection Scope

Access control is managed through SessionIntrospectionScope:

Scope	Description
`parents-only`	Can only see parent sessions (ancestors via forkedFromSessionId)
`children-only`	Can only see child sessions (descendants)
`subtree`	Parents + current + children (default)
`all`	All sessions (requires explicit opt-in)

Configuration with Session Introspection

{
  "tag": "SystemToolbox",
  "contents": {
    "Name": "system",
    "Description": "System information and session memory",
    "Capabilities": [
      "date",
      "hostname",
      "list-sessions",
      "search-sessions",
      "read-session",
      "get-session-stats"
    ],
    "SessionIntrospectionScope": "subtree",
    "SessionIntrospectionMaxResults": 50,
    "SessionIntrospectionIncludeToolOutputs": false
  }
}

list-sessions

Lists accessible sessions based on the configured scope.

Input:

{
  "capability": "list-sessions"
}

Output:

{
  "sessions": [
    {
      "sessionId": "uuid",
      "conversationId": "uuid",
      "modificationTime": "2024-01-15T10:30:00Z",
      "turnCount": 15,
      "isParent": false,
      "isChild": true,
      "isLocked": false,
      "status": "idle"
    }
  ],
  "totalAccessible": 42
}

search-sessions

Performs full-text search across accessible sessions.

Input:

{
  "capability": "search-sessions",
  "query": "error handling pattern"
}

Output:

{
  "query": "error handling pattern",
  "results": [
    {
      "sessionId": "uuid",
      "conversationId": "uuid",
      "turnCount": 15,
      "preview": "...context around match...",
      "matchType": "content"
    }
  ],
  "totalMatches": 5,
  "scope": "subtree"
}

read-session

Reads session content with optional slicing and filtering.

Input:

{
  "capability": "read-session",
  "session_id": "target-session-uuid",
  "take_n": 10,
  "include_thinking": false,
  "include_tool_responses": true
}

Parameters:

session_id (string, required): Session UUID to read
take_n (number, optional): Take last N turns (alternative to offset/limit)
drop_n (number, optional): Drop first N turns
offset (number, optional): Starting turn index (0-based)
limit (number, optional): Max turns to return
include_thinking (boolean, optional): Include LLM thinking/reasoning (default: false)
include_tool_responses (boolean, optional): Include tool call responses (default: false)

Output:

{
  "sessionId": "target-session-uuid",
  "conversationId": "uuid",
  "totalTurns": 25,
  "returnedTurns": 10,
  "content": "Turn 1: [User] ...\nTurn 2: [LLM] ...",
  "format": "condensed-text",
  "scope": "subtree",
  "access": "granted"
}

get-session-stats

Returns aggregate statistics about accessible sessions.

Input:

{
  "capability": "get-session-stats"
}

Output:

{
  "totalSessions": 42,
  "totalTurnsAcrossAllSessions": 850,
  "scope": "subtree",
  "note": "Use SessionPrint.calculateStatistics for detailed per-session stats"
}

Configuration Fields

Field	Type	Description
`Name`	string	Unique name for this toolbox instance
`Description`	string	Human-readable description
`Capabilities`	[string]	List of enabled capabilities
`EnvVarFilter`	string?	Optional substring filter for env vars
`SessionIntrospectionScope`	string?	Scope of accessible sessions (default: "subtree")
`SessionIntrospectionMaxResults`	number?	Max sessions to return (default: 50)
`SessionIntrospectionIncludeToolOutputs`	boolean?	Include tool outputs in read operations (default: true)
`FileSandbox`	object?	File sandbox for attach-file/list-directory capabilities (default: deny all); inline or `{"ref": name}`
`CommandFilter`	string?	Optional command approval filter for execute-command

Security Considerations

Capability-based access: Only enabled capabilities are exposed
Session scope enforcement: Strict access control via SessionIntrospectionScope
ScopeAll requires explicit opt-in: Must be explicitly configured, never default
Env var filtering: Use EnvVarFilter to limit variable exposure
Read-only: System tools gather information but cannot modify the system
Command filtering: Use CommandFilter to control command execution
Linux-focused: Initial implementation targets Linux systems

LLM Tool Interface

The system toolbox exposes a single tool named system_{name}_system_info with:

Parameter: capability (string) - Which system info to retrieve
Additional parameters: Vary by capability (see individual capability documentation)
Returns: JSON object with the requested information

Example tool call:

{
  "capability": "date"
}

Example response:

{
  "capability": "date",
  "data": {
    "utc": "2024-01-15T10:30:00.123456Z",
    "local": "2024-01-15T11:30:00.123456",
    "timezone": "CET",
    "timezoneOffset": "+0100"
  },
  "executionTime": 0.001
}

Developer Toolbox

The Developer Toolbox provides utilities for writing and validating agents and tools, with advanced file editing capabilities including multi-turn edit sessions.

Capabilities

Capability	Description
`show-spec`	Displays specification documentation
`validate-agent`	Validates an agent JSON configuration file
`create-agent`	Creates a new agent configuration
`create-tool`	Creates a new tool script
`show-spec`	Displays specification documentation
`read-file-range`	Reads specific line ranges from a file (supports session reads and metadata-only)
`write-file-range`	Replaces line ranges with multi-turn session support
`patch-file`	Applies a unified diff patch to a file with rich error context

Configuration

{
  "builtinToolboxes": [
    {
      "tag": "DeveloperToolbox",
      "contents": {
        "Name": "dev",
        "Description": "Development utilities",
        "Capabilities": ["show-spec", "validate-agent", "create-agent", "create-tool", "read-file-range", "write-file-range", "patch-file"],
        "FileSandbox": {
          "fsbPredicate": {"tag": "DirectoryRecursive", "contents": "./src"},
          "fsbMaxFileSize": null,
          "fsbName": "developer-sandbox"
        }
      }
    }
  ]
}

File Sandbox for Developer Tools

The read-file-range, write-file-range, and patch-file capabilities require a file sandbox to be configured. These capabilities operate on file content and need explicit permission to access the filesystem.

Example: Allow editing source files only:

{
  "FileSandbox": {
    "fsbPredicate": {
      "tag": "Any",
      "contents": [
        {"tag": "DirectoryRecursive", "contents": "./src"},
        {"tag": "DirectoryRecursive", "contents": "./test"},
        {"tag": "FileExactly", "contents": "./package.yaml"}
      ]
    },
    "fsbMaxFileSize": 10485760
  }
}

Security Note: The write capabilities (write-file-range, patch-file) validate write access against the sandbox. For new files, the parent directory must be within the sandbox. For existing files, the file itself must be within the sandbox.

Tool Interface

The developer toolbox exposes a single tool named developer_{name}_developer_tools with:

Parameter: capability (string) - Which operation to perform
Additional parameters vary by capability

show-spec

{
  "capability": "show-spec",
  "spec_name": "bash-tools"
}

Specs: bash-tools

Response: Returns the embedded specification documentation as text.

read-file-range

Reads specific line ranges from a file and returns them with line numbers. Supports reading from staged sessions and metadata-only queries.

Parameters:

{
  "capability": "read-file-range",
  "path": "/path/to/file",
  "ranges": "1-10,20-30",
  "session_id": "sess-abc",
  "metadata_only": false
}

Parameter	Type	Required	Description
`path`	string	Yes	Path to the file to read
`ranges`	string	No	Line ranges (e.g., `"1-10"`, `"5"`, `"head"`, `"tail"`, `"1-5,20-30"`). Omit to read entire file.
`session_id`	string	No	Read from an in-progress write-file-range session's staged buffer
`metadata_only`	boolean	No	Return only metadata (path, line counts, snapshot ref) without content

Range Formats:

Single line: "5" - Reads line 5
Line range: "1-10" - Reads lines 1 through 10
Multiple ranges: "1-5,20-30" - Reads lines 1-5 and 20-30
Head: "head" - Reads from beginning (no-op for read)
Tail: "tail" - Reads to end (no-op for read)

Returns (full content):

{
  "path": "/path/to/file",
  "content": "1\tdef hello():\n2\t    print('Hello, World!')\n3\t    return True\n",
  "linesRead": 3,
  "totalLineCount": 50,
  "totalFileSize": 1234,
  "rangesParsed": ["1-3"],
  "snapshotRef": "a1b2c3d4..."
}

Returns (metadata_only: true):

{
  "path": "/path/to/file",
  "content": "",
  "linesRead": 0,
  "totalLineCount": 50,
  "totalFileSize": 1234,
  "rangesParsed": [],
  "snapshotRef": "a1b2c3d4...",
  "metadataOnly": true
}

The content includes line numbers prepended with a tab separator in the format {line_num}\t{line_content}.

Reading from Staged Sessions: When session_id is provided, the read is served from the session’s in-memory buffer rather than disk. This allows inspecting pending edits before committing:

{
  "capability": "read-file-range",
  "path": "./src/File.hs",
  "session_id": "sess-abc"
}

write-file-range

Replaces specific lines in a file with new content. Supports multi-turn edit sessions for complex multi-step edits.

Parameters:

{
  "capability": "write-file-range",
  "path": "/path/to/file",
  "ranges": "1-2,5-6",
  "contentBlocks": ["new content for lines 1-2", "new content for lines 5-6"],
  "session_id": "sess-abc",
  "expected_snapshot_ref": "a1b2c3d4...",
  "commit": true
}

Parameter	Type	Required	Description
`path`	string	Yes	Path to the file to modify
`ranges`	string	Yes	Range specs: `N` replace line N, `N-M` replace lines N-M, `N+` insert after line N, `head` prepend, `tail` append, `whole` overwrite file
`contentBlocks`	array[string]	Yes	Array of content blocks, one per range. Use empty strings to delete lines.
`session_id`	string	No	Continue an existing edit session
`expected_snapshot_ref`	string	No	Optimistic locking: only proceed if file matches this snapshot
`commit`	boolean	No	If true, write to disk and close session. If false, stage changes.

Range Formats:

N - Replace line N (e.g., 5 replaces line 5)
N-M - Replace lines N through M (e.g., 1-10 replaces lines 1-10)
N+ - Insert after line N (e.g., 54+ inserts after line 54)
head - Prepend content before line 1 (use this to create new files)
tail - Append content after the last line
whole - Replace the entire file

Warning: N and N+ are not interchangeable. Use 54+ to insert after line 54; using 54 will replace line 54.

Multi-Turn Edit Sessions:

For complex edits, use sessions to stage changes across multiple turns before committing:

Turn 1 (start session):
  {
    "capability": "write-file-range",
    "path": "./src/File.hs",
    "ranges": "10-20",
    "contentBlocks": ["new code..."],
    "commit": false
  }
  -> returns sessionId: "sess-abc", sessionStatus: "staged"

Turn 2 (continue, still using ORIGINAL line numbers):
  {
    "capability": "write-file-range",
    "path": "./src/File.hs",
    "session_id": "sess-abc",
    "ranges": "100-110",
    "contentBlocks": ["more code..."],
    "commit": false
  }
  -> sessionStatus: "staged"

Turn N (commit to disk):
  {
    "capability": "write-file-range",
    "path": "./src/File.hs",
    "session_id": "sess-abc",
    "ranges": "200-210",
    "contentBlocks": ["final code..."],
    "commit": true
  }
  -> writes to disk, sessionStatus: "committed"

Session Rules:

Sessions expire after 1 hour of inactivity
Edits within a session must not overlap (in original coordinates)
whole range not supported while session is open
Commit fails if file changed on disk since session started
Reusing a session_id after commit returns an “already committed” error

Optimistic Locking: Use expected_snapshot_ref to prevent conflicts:

{
  "capability": "write-file-range",
  "path": "./src/File.hs",
  "ranges": "1-10",
  "contentBlocks": ["new content"],
  "expected_snapshot_ref": "a1b2c3d4...",
  "commit": true
}

The operation fails if the file’s current snapshot doesn’t match, allowing retry with fresh content.

Returns:

{
  "path": "/path/to/file",
  "rangesModified": 2,
  "linesWritten": 6,
  "beforeSnapshotRef": "a1b2c3d4...",
  "afterSnapshotRef": "e5f6g7h8...",
  "sessionId": "sess-abc",
  "sessionNetDelta": 2,
  "sessionCommitted": true,
  "sessionStatus": "committed"
}

patch-file

Applies a unified diff patch to a file atomically with context validation and rich error reporting.

Parameters:

{
  "capability": "patch-file",
  "path": "/path/to/file",
  "patch": "--- a/src/File.hs\n+++ b/src/File.hs\n@@ -10,5 +10,6 @@ import Foo\n+import Data.Text (Text)\n@@ -100,5 +101,5 @@ func1 x =\n-  oldBody\n+  newBody",
  "expected_snapshot_ref": "a1b2c3d4..."
}

Parameter	Type	Required	Description
`path`	string	Yes	Path to the file to patch
`patch`	string	Yes	Unified diff patch content
`expected_snapshot_ref`	string	No	Optimistic locking: only proceed if file matches

Patch Format: Follows standard unified diff format:

File headers (--- and +++ lines) are ignored
Hunk headers start with @@ (e.g., @@ -10,5 +11,6 @@)
Context lines have no prefix
Removed lines start with -
Added lines start with +

Features:

Atomic application: All hunks are validated before any changes are applied
Context validation: Each hunk’s context lines must match exactly
Overlap detection: Hunks that would overlap are rejected
Bottom-to-top application: Hunks are applied in descending line order to avoid line number shifts
Rich errors: Context-mismatch errors include expected/actual lines for debugging

Rich Error Context: When a context mismatch occurs, the error includes both expected and actual lines:

{
  "error": "Context mismatch at line 100",
  "expected": ["import Foo", "import Bar"],
  "actual": ["import Foo", "import Baz"],
  "message": "Context doesn't match"
}

Returns:

{
  "path": "/path/to/file",
  "hunksApplied": 2,
  "hunksRejected": 0,
  "linesChanged": 3,
  "beforeSnapshotRef": "a1b2c3d4...",
  "afterSnapshotRef": "e5f6g7h8..."
}

Error Responses:

{
  "error": "Context mismatch at line 100: Context doesn't match",
  "expected": ["context line 1", "context line 2"],
  "actual": ["different line 1", "different line 2"]
}

Range Specification Types

-- | Range specification for file operations.
data RangeSpec
    = Lines (Int, Int)  -- ^ 1-based, inclusive line range (start, end)
    | Head              -- ^ Before line 1 (prepend)
    | Tail              -- ^ After last line (append)
    deriving (Show, Eq)

-- | Result of a read file range operation.
data ReadFileRangeResult = ReadFileRangeResult
    { readFilePath :: FilePath
    , readFileContent :: Text
    , readFileLinesRead :: Int
    , readFileTotalLines :: Int
    , readFileTotalSize :: Int
    , readFileRangesParsed :: [Text]
    , readFileSnapshotRef :: Maybe SnapshotRef
    , readFileSessionId :: Maybe Text
    , readFileMetadataOnly :: Bool
    }

-- | Result of a write file range operation.
data WriteFileRangeResult = WriteFileRangeResult
    { writeFilePath :: FilePath
    , writeFileRangesModified :: Int
    , writeFileLinesWritten :: Int
    , writeFileBeforeSnapshotRef :: Maybe SnapshotRef
    , writeFileAfterSnapshotRef :: Maybe SnapshotRef
    , writeFileSessionId :: Maybe Text
    , writeFileSessionNetDelta :: Maybe Int
    , writeFileSessionCommitted :: Bool
    , writeFileSessionStatus :: Text  -- "staged" or "committed"
    }

-- | Result of a patch file operation.
data PatchResult = PatchResult
    { patchFilePath :: FilePath
    , patchHunksApplied :: Int
    , patchHunksRejected :: Int
    , patchLinesChanged :: Int
    , patchBeforeSnapshotRef :: Maybe SnapshotRef
    , patchAfterSnapshotRef :: Maybe SnapshotRef
    }

-- | Patch error with rich context.
data PatchError
    = PatchParseError Text
    | PatchContextMismatch
        { patchMismatchLine :: Int
        , patchMismatchMessage :: Text
        , patchMismatchExpected :: [Text]
        , patchMismatchActual :: [Text]
        }
    | PatchHunkOverlap Int Int
    | PatchFileNotFound FilePath
    | PatchInvalidLineNumber Int

Snapshot System

The Developer Toolbox uses a snapshot system for optimistic locking and file restoration:

-- | Snapshot reference (MD5 hash of content).
newtype SnapshotRef = SnapshotRef { unSnapshotRef :: Text }
    deriving (Show, Eq, Ord)

-- | Snapshot with metadata.
data Snapshot = Snapshot
    { snapshotContent :: ByteString
    , snapshotCreatedAt :: UTCTime
    }

-- | Create snapshot from content.
makeSnapshot :: ByteString -> Snapshot

-- | Get reference for a snapshot.
snapshotRef :: Snapshot -> SnapshotRef

Snapshots are automatically taken during read operations and returned in results. Use expected_snapshot_ref for optimistic locking in write operations.

Generated Templates

Bash Tool Template

#!/bin/bash
# my-tool - A bash tool for agents-exe

set -euo pipefail

case "${1:-}" in
  describe)
    cat <<'DESCRIBE_EOF'
{
  "slug": "my-tool",
  "description": "Tool my-tool - describe what this tool does",
  "args": [],
  "empty-result": {
    "tag": "AddMessage",
    "contents": "--no output--"
  }
}
DESCRIBE_EOF
    ;;
  run)
    # TODO: Implement tool logic
    echo "Tool my-tool executed"
    ;;
  *)
    echo "Usage: my-tool <describe|run>" >&2
    exit 1
    ;;
esac

Python Tool Template

#!/usr/bin/env python3
import json
import sys
import os

def describe():
    return {
        "slug": "my-tool",
        "description": "Tool my-tool - describe what this tool does",
        "args": [],
        "empty-result": {
            "tag": "AddMessage",
            "contents": "--no output--"
        }
    }

def run():
    # TODO: Implement tool logic
    print("Tool my-tool executed")

def main():
    if len(sys.argv) < 2:
        print(f"Usage: {sys.argv[0]} <describe|run>", file=sys.stderr)
        return 1
    
    command = sys.argv[1]
    if command == "describe":
        print(json.dumps(describe()))
        return 0
    elif command == "run":
        run()
        return 0
    else:
        print(f"Unknown command: {command}", file=sys.stderr)
        return 1

if __name__ == "__main__":
    sys.exit(main())

Lua Tools

Lua tools provide embedded scripting capabilities through a sandboxed Lua interpreter. This enables agents to orchestrate complex workflows by combining multiple tools through Lua scripts.

Recursive Language Models (LRM) with Lua

The Lua toolbox enables a powerful pattern called Recursive Language Models (LRM). By configuring an agent to reference itself in extraAgents and granting the Lua toolbox permission to call the resulting io_prompt_agent_{slug} tool, you create an agent that can recursively invoke itself for complex reasoning tasks.

Configuration

{
  "tag": "OpenAIAgentDescription",
  "contents": {
    "slug": "local_lrmlua",
    "flavor": "KimiV1",
    "modelUrl": "https://api.moonshot.ai/v1",
    "apiKeyId": "kimi",
    "modelName": "kimi-k2.5",
    "announce": "The main agent, capable of reasoning.",
    "systemPrompt": [
      "You are a helpful software agent trying to solve user requests"
    ],
    "extraAgents": [
      {
        "slug": "local_lrmlua",
        "path": "kimi-10.lrmlua.json"
      }
    ],
    "bashToolboxes": [
      {
        "tag": "SingleTool",
        "contents": {"Path": "./tools/askuser/ask-user.bash"}
      }
    ],
    "builtinToolboxes": [
      {
        "tag": "LuaToolbox",
        "contents": {
          "Name": "lua",
          "Description": "Sandboxed Lua interpreter",
          "MaxMemoryMB": 256,
          "MaxExecutionTimeSeconds": 300,
          "AllowedTools": [
            "bash_ask_user",
            "io_prompt_agent_local_lrmlua",
            "sqlite_shared_working_memory_query"
          ],
          "AllowedHosts": [
            "localhost",
            "127.0.0.1"
          ],
          "FileSandbox": {
            "fsbPredicate": {
              "tag": "Any",
              "contents": [
                {"tag": "DirectoryRecursive", "contents": "./repro-cases"},
                {"tag": "FileExactly", "contents": "./README.md"}
              ]
            },
            "fsbMaxFileSize": 10485760,
            "fsbName": "lua-fs-sandbox"
          }
        }
      },
      {
        "tag": "SqliteToolbox",
        "contents": {
          "Name": "shared_working_memory",
          "Description": "A base to help you coordinate large works.",
          "Versioning": {"tag": "SqliteReadWrite", "path": "./dev-memory.db"}
        }
      }
    ]
  }
}

LuaToolbox Configuration Fields

Field	Type	Description
`Name`	string	Unique name for this toolbox instance (used as tool prefix)
`Description`	string	Human-readable description
`MaxMemoryMB`	integer	Maximum Lua heap memory in megabytes
`MaxExecutionTimeSeconds`	integer	Maximum script execution time in seconds
`AllowedTools`	[string]	Whitelist of tool names Lua scripts can call via the portal
`AllowedHosts`	[string]	Whitelist of network hosts accessible to Lua HTTP module
`FileSandbox`	object?	File sandbox configuration for Lua `fs` module; inline or `{"ref": name}`

File Sandbox for Lua

The Lua fs module uses the unified file sandbox system. When FileSandbox is configured, all filesystem operations (fs.read, fs.write, fs.list, etc.) are validated against the sandbox.

Migration from allowedPaths:

The legacy allowedPaths field no longer exists and is ignored if present. Use FileSandbox instead:

// Old (removed, ignored)
{
  "allowedPaths": ["./data", "./scripts"]
}

// New
{
  "FileSandbox": {
    "fsbPredicate": {
      "tag": "Any",
      "contents": [
        {"tag": "DirectoryRecursive", "contents": "./data"},
        {"tag": "DirectoryRecursive", "contents": "./scripts"}
      ]
    }
  }
}

Security Features

The Lua toolbox provides a sandboxed execution environment:

Memory limits: a script whose Lua heap grows past MaxMemoryMB fails with a memory-limit error (0 disables the limit)
Timeout enforcement: Scripts that exceed MaxExecutionTimeSeconds are terminated
Path sandboxing: Filesystem access restricted to FileSandbox configuration
Host whitelisting: HTTP requests limited to allowedHosts
Tool whitelist: Only tools in allowedTools can be called through the portal
Dangerous functions removed: os.execute, io.popen, loadfile, dofile, etc. are removed
Empty whitelist = no access: Secure defaults - empty lists mean no access

Lua Standard Library Modules

All modules are pre-registered as global variables - no require() needed. Simply use the module name directly (e.g., json.encode(), text.split()).

Module	Functions	Description
`json`	`json.encode`, `json.decode`	JSON manipulation
`http`	`http.get`, `http.post`, `http.request`	HTTP requests (host-restricted)
`time`	`time.now`, `time.sleep`, `time.format`, `time.diff`	Time utilities
`fs`	`fs.read`, `fs.write`, `fs.list`, `fs.exists`	File system (sandboxed)
`text`	`text.split`, `text.trim`, `text.upper`, `text.lower`, `text.find`, `text.gsub`, `text.startswith`, `text.endswith`, `text.len`, `text.sub`	String utilities
`tools`	`tools.call`, `tools.list`	Tool portal integration

fs Module and Sandbox

The fs module respects the file sandbox configuration:

-- These operations are validated against the fileSandbox
local content = fs.read("./data/file.txt")      -- Must be in sandbox
fs.write("./output/result.txt", "data")          -- Must be in sandbox
local files = fs.list("./scripts")               -- Must be in sandbox
local exists = fs.exists("./README.md")          -- Must be in sandbox

Sandbox Validation:

fs.read: Validates read permission via validateFileRead
fs.write: Validates write permission via validateFileWrite
fs.list, fs.exists, fs.isdir, fs.isfile: Validates access permission
fs.mkdir: Validates write permission on parent directory
fs.patch: Validates write permission on the file

Example Lua Script

-- Modules are pre-loaded as globals and available via require()

-- Read a file (validated against fileSandbox)
local content = fs.read("./repro-cases/input.json")
if not content then
    return {error = "Failed to read file"}
end

-- Call the recursive agent for complex reasoning
local reasoning_result = tools.call("io_prompt_agent_local_lrmlua", {
    what = "Analyze this codebase structure and identify potential refactoring opportunities"
})

-- Process the result
if reasoning_result.status == "ok" then
    local analysis = json.decode(reasoning_result.result_txt)
    
    -- Query the shared working memory
    local db_result = tools.call("sqlite_shared_working_memory_query", {
        sql = "INSERT INTO analysis_results (content) VALUES ('" .. analysis.summary .. "')"
    })
    
    -- Write results (validated against fileSandbox)
    fs.write("./repro-cases/output.json", json.encode(analysis))
    
    return {
        success = true,
        analysis = analysis,
        stored = db_result.status == "ok"
    }
else
    return {
        success = false,
        error = reasoning_result.error
    }
end

Tool Naming

The Lua toolbox exposes a single tool named lua_{name}_execute:

{
  "script": "return json.encode({status='ok'})",
  "timeout": 60
}

Note: Modules (json, text, time, fs, http, tools) are available as pre-loaded global variables.

Error Handling

Lua script errors are captured and returned to the LLM:

data ScriptError
    = LuaRuntimeError [Aeson.Value]      -- Syntax or runtime error
    | TimeoutError Int                   -- Script exceeded time limit
    | MemoryError Int                    -- Script exceeded memory limit
    | SandboxError Text                  -- Attempted sandbox violation
    | ToolInvocationError Text           -- Error calling another tool
    | InitializationError Text           -- Failed to initialize Lua state

Skills System

The Skills system provides procedural knowledge and executable capabilities via progressive disclosure, following the agentskills.io specification.

Overview

Skills are packages of related functionality that can be dynamically enabled/disabled during a session. They implement progressive disclosure:

Initially: Only metadata tools are visible (describe, enable, disable, list)
After enable: Script tools become available for execution

Session Start
      │
      ▼
┌─────────────┐
│ skill_list  │  ← Always available
│ skill_desc  │  ← Always available
│ skill_enable│  ← Always available
└──────┬──────┘
       │
       ▼ (user calls skill_enable_pdf-processing)
┌─────────────┐
│ skill_list  │
│ skill_desc  │
│ skill_enable│
│ skill_pdf_* │  ← NEW: Script tools now visible
└─────────────┘

Skill Structure

A skill directory contains:

skill-directory/
├── SKILL.md          # Frontmatter + instructions
├── scripts/          # Executable scripts
│   ├── extract-text.sh
│   └── convert.sh
└── references/       # Documentation (optional)
    └── api-docs.md

SKILL.md Format

---
name: pdf-processing
description: Extract and manipulate PDF files
license: MIT
compatibility: Linux, macOS
metadata:
  author: team-pdf
  version: "1.0"
---

# PDF Processing

This skill provides tools for working with PDF documents.

## Usage

Enable the skill, then use the available script tools...

Skill Types (`System.Agents.Tools.Skills.Types`)

-- | Validated skill name (1-64 chars, lowercase, digits, hyphens)
newtype SkillName = SkillName { unSkillName :: Text }

-- | Complete skill with metadata, instructions, scripts, and references
data Skill = Skill
    { skillMetadata :: SkillMetadata
    , skillInstructions :: Text
    , skillPath :: FilePath
    , skillScripts :: [ScriptInfo]
    , skillReferences :: [ReferenceInfo]
    }

-- | Metadata from SKILL.md frontmatter
data SkillMetadata = SkillMetadata
    { smName :: SkillName
    , smDescription :: Text
    , smLicense :: Maybe Text
    , smCompatibility :: Maybe Text
    , smMetadata :: Map Text Text
    }

-- | Script following the describe/run protocol
data ScriptInfo = ScriptInfo
    { siName :: ScriptName
    , siPath :: FilePath
    , siDescription :: Maybe Text
    , siArgs :: [ScriptArgInfo]
    }

Skill State

-- | Session state tracking which skills and scripts are enabled
newtype SkillsSessionState = SkillsSessionState
    { sssActiveSkills :: Map SkillName SkillScriptsState
    }

-- | Script state within a skill
type SkillScriptsState = Map ScriptName ScriptState
data ScriptState = Enabled | Disabled

-- | Monoid instance for folding over session turns
instance Monoid SkillsSessionState where
    mempty = SkillsSessionState Map.empty
    -- Later state overrides earlier state

Toolbox Integration (`System.Agents.Tools.Skills.Toolbox`)

-- | Compute all available skill tools from session state
computeSkillTools :: SkillsStore -> Session -> [ToolRegistration]
computeSkillTools store session =
    let state = foldSession session
        -- Always available
        metaTools = concatMap makeMetaTools (allSkills store)
        -- Available only when enabled
        scriptTools = 
            concatMap (makeScriptToolsForSkill state store) 
                      (sssActiveSkills state)
     in listTool ++ metaTools ++ scriptTools

-- Tool naming convention
skill2LLMName :: Text -> SkillName -> ToolName
-- skill_describe_pdf-processing
-- skill_enable_pdf-processing
-- skill_pdf-processing_extract-text

Generated Tools

For each skill, these tools are generated:

Tool	Purpose	Always Available
`skill_list`	List all skills	Yes
`skill_describe_{name}`	Get skill metadata	Yes
`skill_enable_{name}`	Enable skill scripts	Yes
`skill_disable_{name}`	Disable skill scripts	Yes
`skill_{name}_{script}`	Execute script	No (requires enable)

Skill Sources

Skills can be loaded from:

data SkillSource
    = SkillDirectory FilePath           -- Local directory
    | SkillGitRepo GitUrl (Maybe Subdirectory)  -- Git repository

Configuration:

{
  "skillSources": [
    { "tag": "SkillDirectory", "contents": "./skills" },
    { "tag": "SkillGitRepo", 
      "contents": { 
        "url": "https://github.com/org/skills-repo",
        "subdir": "pdf-tools"
      }
    }
  ],
  "autoEnableSkills": ["core-utils"]
}

Progressive Disclosure Benefits

Reduced context window: Only relevant tools visible
Discoverability: Users learn about skills organically
Modularity: Skills are self-contained packages
Safety: Scripts only accessible after explicit enable
Auditability: State changes tracked in session

Tool Validation (`System.Agents.Tools.Validation`)

Tool input validation helps LLMs self-correct when they make incorrect tool calls.

Validation Types

-- | Single validation error with context
data ValidationError = ValidationError
    { errorPath :: Text       -- JSON path (e.g., "user.name")
    , errorMessage :: Text    -- Human-readable description
    }

-- | Validation configuration
data ValidationConfig = ValidationConfig
    { allowExtraProperties :: Bool
    , strictMode :: Bool
    }

Validation Function

-- | Validate tool input against its schema
validateToolInput :: 
    [ParamProperty] ->  -- Tool schema
    Aeson.Value ->      -- Input value
    [ValidationError]   -- Empty if valid

-- Example usage:
let errors = validateToolInput toolSchema inputValue
case errors of
    [] -> proceedWithToolCall
    errs -> returnValidationErrors errs

Supported Validations

Check	Description
Required fields	Ensures required properties are present
Type checking	Validates string, number, boolean, enum, object
Enum values	Checks string is in allowed values list
Nested objects	Recursively validates nested structures
Extra properties	Optionally rejects unknown properties

CLI: check-tool-call

# Validate a tool call payload
echo '{"filepath": "/path/to/file"}' | \
    agents-exe check-tool-call --tool ./tools/read-file.sh

# Example output (invalid):
# Tool call validation failed for 'read-file' with 2 errors:
# 1. filepath: Required property missing
# 2. content: Required property missing
#
# Please correct these issues and try again.

Error Formatting

formatValidationErrors :: Text -> [ValidationError] -> Text
-- Produces:
-- Tool call validation failed for 'tool-name' with 2 errors:
--
-- 1. filters.status: Invalid enum value: pending. Allowed: active, inactive
-- 2. user.age: Expected number but got string
--
-- Please correct these issues and try again.

Tool Portal (`System.Agents.ToolPortal`)

The Tool Portal enables inter-toolbox communication, allowing tools to invoke other tools through a controlled callback mechanism.

Use Cases

Lua scripts calling other tools via tools.call()
Orchestration tools that coordinate multiple operations
Composite tools that build on existing tools

Portal Types

-- | Tool portal callback type
type ToolPortal = ToolCall -> IO ToolResult

-- | Portal execution errors
data PortalError
    = PortalToolNotFound Text
    | PortalToolNotAllowed Text [Text]
    | PortalInvalidArguments Text
    | PortalExecutionError Text

Creating a Portal

import System.Agents.ToolPortal

-- Create portal from registered tools
let portal = makeToolPortal tracer registrations

-- Create context with portal
let ctx = mkPortalContext
        sessId convId turnId mAgentId mSession
        callStack maxDepth (Just portal) allowedTools

Lua Integration

Lua scripts can call tools through the portal:

local tools = require("tools")

-- Call another tool
local result = tools.call("read_file", {
    filepath = "/path/to/file"
})

-- Access result
print(result.data)
print(result.duration)

Security

Tool whitelist: Only allowed tools can be called
No nested portals: Prevents infinite recursion
Execution tracking: Each portal call is timed and logged
Minimal context: Portal tools execute without their own portal

Portal Result

data ToolResult = ToolResult
    { resultData :: Aeson.Value
    , resultDuration :: NominalDiffTime
    , resultTraceId :: Text
    }

IO Tools

IO tools are Haskell functions that run within the agent process.

Definition

type IOToolBuilder = AgentSlug -> AgentId -> ToolRegistration

exampleTool :: IOToolBuilder
exampleTool slug agentId = ToolRegistration
    { innerTool = ...
    , declareTool = OpenAI.Tool
        { toolName = "example"
        , toolDescription = "An example tool"
        , toolParamProperties = [...]
        }
    , findTool = \call -> ...
    }

Use Cases

Agent-to-agent calls (via turnAgentRuntimeIntoIOTool)
Direct system integration
Performance-critical operations
Stateful operations

IO Tool with Context

ioTool ::
    (Aeson.FromJSON llmArg) =>
    IOTools.IOScript llmArg ByteString ->
    Tool ()

data IOScript arg result = IOScript
    { description :: IOScriptDescription
    , ioRun :: ToolExecutionContext -> arg -> IO result
    }

Tool Execution Context

Tools receive execution context for recursion tracking:

data ToolExecutionContext = ToolExecutionContext
    { ctxSessionId :: SessionId
    , ctxConversationId :: ConversationId
    , ctxTurnId :: TurnId
    , ctxAgentId :: Maybe AgentId
    , ctxFullSession :: Maybe Session
    , ctxCallStack :: [CallStackEntry]
    , ctxMaxDepth :: Maybe Int
    , ctxToolPortal :: Maybe ToolPortal
    , ctxAllowedTools :: [Text]
    }

data CallStackEntry = CallStackEntry
    { callAgentSlug :: AgentSlug
    , callConversationId :: ConversationId
    , callDepth :: Int
    }

Recursion Control

pushAgentContext :: 
    AgentSlug -> 
    ConversationId -> 
    ToolExecutionContext -> 
    Either RecursionError ToolExecutionContext

Prevents infinite loops by tracking call depth and failing when maxDepth is exceeded.

OS Integration (Subcall Visibility)

For TUI visibility of subcall conversations, the context includes OS integration fields:

data ToolExecutionContext = ToolExecutionContext
    { -- ... existing fields ...
    , ctxWorld :: Maybe World
    -- ^ OS World for ECS operations. Enables subcall conversations
    -- to be tracked as first-class entities in the OS.
    , ctxEventQueue :: Maybe (TQueue OSEvent)
    -- ^ Event queue for OS event emission. Enables the TUI to receive
    -- notifications about subcall lifecycle (start, progress, completion).
    , ctxParentConversation :: Maybe ConversationId
    -- ^ Parent conversation ID for nested agent calls.
    }

Helper Functions:

-- | Create a nested context for subcall execution.
mkSubcallContext ::
    ToolExecutionContext ->
    Maybe World ->
    Maybe (TQueue OSEvent) ->
    ConversationId ->
    ToolExecutionContext

-- | Get the subcall depth (0 for root conversations).
getSubcallDepth :: ToolExecutionContext -> Int

-- | Check if this context is for a subcall.
isSubcallContext :: ToolExecutionContext -> Bool

Tool Result Types

data CallResult call
    = BlobToolSuccess call ByteString (Maybe MediaType)
    | JsonToolSuccess call Aeson.Value
    | ToolSkipped call
    | BashToolError call BashError
    | IOToolError call IOToolError
    | McpToolError call Text
    | OpenAPIToolError call String
    | PostgRESToolError call String
    | SqliteToolError call SqliteError
    | SystemToolError call SystemInfoError
    | DeveloperToolError call ValidationError
    | McpToolResult call McpToolResult
    | OpenAPIToolResult call OpenAI.ToolResult
    | PostgRESToolResult call OpenAI.ToolResult
    | SqliteToolResult call SqliteQueryResult
    | SystemToolResult call SystemQueryResult
    | DeveloperToolResult call ValidationResult
    | DeveloperToolSpecResult call Text
    | DeveloperToolAgentValidationResult call AgentValidationResult
    | DeveloperToolCreateResult call CreateResult
    | DeveloperToolReadFileRangeResult call ReadFileRangeResult
    | DeveloperToolWriteFileRangeResult call WriteFileRangeResult
    | DeveloperToolPatchResult call PatchResult
    | LuaToolResult call Aeson.Value
    | LuaToolError call Text

Tool Schema

Tools expose JSON Schema for LLM function calling:

data ParamProperty = ParamProperty
    { propertyKey :: Text
    , propertyType :: ParamType
    , propertyDescription :: Text
    , propertyRequired :: Bool
    }

data ParamType
    = NullParamType
    | StringParamType
    | BoolParamType
    | NumberParamType
    | EnumParamType [Text]
    | OpaqueParamType Text
    | MultipleParamType Text
    | ObjectParamType [ParamProperty]

Example JSON Schema:

{
  "type": "object",
  "properties": {
    "filename": {
      "type": "string",
      "description": "Name of the file to read"
    },
    "lines": {
      "type": "integer",
      "description": "Number of lines to read"
    }
  },
  "required": ["filename"]
}

Naming Conventions

Tools are named according to their type and toolbox:

Tool Type	Naming Pattern	Example
Bash	`bash_{slug}`	`bash_read_file`
MCP	`mcp_{toolbox}_{name}`	`mcp_filesystem_read_file`
OpenAPI	`openapi_{toolbox}_{operation}`	`openapi_pets_getById`
PostgREST	`postgrest_{toolbox}_{method}_{table}`	`postgrest_mydb_get_users`
SQLite	`sqlite_{toolbox}_query`	`sqlite_analytics_query`
System	`system_{toolbox}_system_info`	`system_system_system_info`
Developer	`developer_{toolbox}_developer_tools`	`developer_dev_developer_tools`
Lua	`lua_{toolbox}_execute`	`lua_utils_execute`
IO	`io_{slug}`	`io_calculator`
IO (Agent)	`io_prompt_agent_{slug}`	`io_prompt_agent_helper`
Skill (meta)	`skill_{action}_{name}`	`skill_describe_pdf-processing`
Skill (script)	`skill_{name}_{script}`	`skill_pdf-processing_extract-text`

Combining Tool Sources

The runtime merges tools from all sources:

newRuntime :: ... -> IO (Either String Runtime)
newRuntime ... = do
    -- Bash tools from tool directory
    bashTools <- BashToolbox.initializeBackroundToolbox ...
    
    -- IO tools from code
    let ioTools = [mk slug uid | mk <- mkIoTools]
    
    -- MCP tools from configured servers
    mcpTools <- mapM initializeMcpToolbox mcpConfigs
    
    -- OpenAPI tools from specs
    openApiTools <- mapM loadOpenApiTools openApiConfigs
    
    -- PostgREST tools
    postgrestTools <- mapM loadPostgRESTools prConfigs
    
    -- SQLite tools from builtin toolboxes
    sqliteTools <- readSqliteToolsRegistrations tracer sqliteToolboxes
    
    -- System tools from builtin toolboxes
    systemTools <- readSystemToolsRegistrations tracer systemToolboxes
    
    -- Developer tools from builtin toolboxes
    devTools <- readDeveloperToolsRegistrations tracer devToolboxes
    
    -- Lua tools from builtin toolboxes
    luaTools <- readLuaToolsRegistrations tracer luaToolboxes
    
    -- Skills tools from skill sources
    skillsStore <- loadSkillsFromSources skillSources
    let skillsTools = computeSkillTools skillsStore session
    
    -- Combine all
    let allTools = ioTools ++ bashTools ++ mcpTools ++ openApiTools ++ 
                   postgrestTools ++ sqliteTools ++ systemTools ++ 
                   devTools ++ luaTools ++ skillsTools

Error Handling

Bash Tool Errors

data ToolboxError
    = ToolboxDirectoryNotFound FilePath
    | ScriptParseError FilePath String
    | ScriptExecutionError FilePath Int String

MCP Errors

data McpError
    = ProcessStartError Text
    | ProtocolError Text
    | ToolCallError Text

OpenAPI Errors

data OpenAPIError
    = SpecParseError String
    | SchemaResolutionError Text
    | ReferenceError RefPath

System Toolbox Errors

data QueryError
    = CapabilityNotEnabledError Text
    | SystemInfoError Text
    | FileNotFoundError FilePath
    | FileTooLargeError FilePath Int
    | UnsupportedFileTypeError FilePath Text
    | SessionStoreNotConfiguredError
    | SessionNotFoundError Text
    | InvalidSessionIdError Text
    | SessionAccessDeniedError Text SessionIntrospectionScope
    | MissingParameterError Text
    | CommandRefusedError Text
    | InvalidFilterOutputError Text

Developer Toolbox Errors

data DeveloperToolError
    = CapabilityNotEnabledError Text
    | ValidationError Text
    | FileExistsError FilePath
    | InvalidTemplateError Text
    | InvalidRangeError Text
    | RangeOutOfBoundsError Text
    | PermissionError Text
    | PatchValidationError PatchError
    | SessionNotFoundError Text
    | SessionExpiredError Text
    | SessionAlreadyCommittedError Text
    | RangeOverlapError Text

Patch Errors

data PatchError
    = PatchParseError Text
    | PatchContextMismatch
        { patchMismatchLine :: Int
        , patchMismatchMessage :: Text
        , patchMismatchExpected :: [Text]
        , patchMismatchActual :: [Text]
        }
    | PatchHunkOverlap Int Int
    | PatchFileNotFound FilePath
    | PatchInvalidLineNumber Int

Validation Errors

data ValidationError = ValidationError
    { errorPath :: Text
    , errorMessage :: Text
    }

Portal Errors

data PortalError
    = PortalToolNotFound Text
    | PortalToolNotAllowed Text [Text]
    | PortalInvalidArguments Text
    | PortalExecutionError Text

Best Practices

Idempotency: Tools should be safe to call multiple times
Clear descriptions: Help the LLM understand when to use each tool
Validation: Validate inputs before execution
Timeouts: Set reasonable timeouts for external calls
Logging: Use the tracer for observability
Error messages: Return clear error messages for LLM consumption
Parameter naming: Use descriptive parameter names
Required vs Optional: Mark truly required parameters as required
Progressive disclosure: Use Skills for complex tool suites
Portal safety: Always whitelist tools for portal access
Recursive agents: When using LRM pattern, set appropriate maxDepth to prevent infinite recursion
Lua security: Always specify allowedTools, allowedPaths, and allowedHosts - empty means no access
Media output: Declare output-media-type for tools that produce binary content
Mixed responses: Use MixedResponse for rich multi-modal tool outputs
File range operations: Use read-file-range and write-file-range for precise file editing rather than reading/writing entire files
Line numbers: When using read-file-range, the output includes line numbers to help LLMs understand file structure
Range formatting: Always use 1-based line numbers for ranges (e.g., “1-10” for lines 1 through 10)
Atomic file edits: For complex multi-range edits, use write-file-range with contentBlocks array
Patch for context validation: Use patch-file when context validation is needed before applying changes
Session introspection: Enable session introspection capabilities in SystemToolbox for cross-session analysis
File sandbox configuration: Always configure FileSandbox for SystemToolbox (attach-file), DeveloperToolbox (read/write/patch), and LuaToolbox (fs module)
Secure by default: The default file sandbox denies all access (AlwaysDeny). Explicitly configure allowed paths.
Path canonicalization: The file sandbox canonicalizes all paths, so predicates apply to resolved paths (not symlink paths)
Multi-turn edit sessions: Use commit: false for staging multiple edits, then commit: true to finalize
Optimistic locking: Use expected_snapshot_ref to prevent conflicts when multiple agents edit the same file
Rich patch errors: Patch context mismatches now include expected/actual lines for easier debugging
Command filtering: Use CommandFilter in SystemToolbox to control arbitrary command execution

Example: Complete Tool Configuration

{
  "slug": "file-agent",
  "apiKeyId": "openai",
  "flavor": "openai",
  "modelUrl": "https://api.openai.com/v1",
  "modelName": "gpt-4o",
  "announce": "A file management assistant with vision",
  "systemPrompt": ["You help users manage and analyze files."],
  "toolDirectory": "tools",
  "bashToolboxes": [
    {"tag": "FileSystemDirectory", "contents": {"Path": "./extra-tools"}}
  ],
  "mcpServers": [
    {
      "tag": "McpSimpleBinary",
      "contents": {
        "name": "filesystem",
        "executable": "/usr/bin/mcp-filesystem",
        "args": ["--root", "/home/user"]
      }
    }
  ],
  "openApiToolboxes": [
    {
      "tag": "OpenAPIServer",
      "contents": {
        "SpecUrl": "https://api.github.com/openapi.json",
        "BaseUrl": "https://api.github.com",
        "Token": "${GITHUB_TOKEN}"
      }
    }
  ],
  "postgrestToolboxes": [
    {
      "tag": "PostgRESTServer",
      "contents": {
        "SpecUrl": "http://localhost:3000/",
        "BaseUrl": "http://localhost:3000"
      }
    }
  ],
  "builtinToolboxes": [
    {
      "tag": "SqliteToolbox",
      "contents": {
        "Name": "analytics",
        "Description": "Analytics database",
        "Versioning": {"tag": "SqliteReadOnly", "path": "./analytics.db"}
      }
    },
    {
      "tag": "SystemToolbox",
      "contents": {
        "Name": "system",
        "Description": "System context and session memory",
        "Capabilities": ["date", "hostname", "working-directory", "attach-file", "list-directory", "execute-command", "list-sessions", "search-sessions"],
        "EnvVarFilter": null,
        "SessionIntrospectionScope": "subtree",
        "SessionIntrospectionMaxResults": 50,
        "SessionIntrospectionIncludeToolOutputs": false,
        "FileSandbox": {
          "fsbPredicate": {"tag": "DirectoryRecursive", "contents": "./project"},
          "fsbMaxFileSize": 52428800,
          "fsbName": "system-sandbox"
        },
        "CommandFilter": "/path/to/approval-script.sh"
      }
    },
    {
      "tag": "DeveloperToolbox",
      "contents": {
        "Name": "dev",
        "Description": "Development utilities",
        "Capabilities": ["show-spec", "validate-agent", "create-agent", "create-tool", "read-file-range", "write-file-range", "patch-file"],
        "FileSandbox": {
          "fsbPredicate": {
            "tag": "Any",
            "contents": [
              {"tag": "DirectoryRecursive", "contents": "./src"},
              {"tag": "DirectoryRecursive", "contents": "./test"},
              {"tag": "FileExactly", "contents": "./package.yaml"}
            ]
          },
          "fsbMaxFileSize": 10485760,
          "fsbName": "developer-sandbox"
        }
      }
    },
    {
      "tag": "LuaToolbox",
      "contents": {
        "Name": "lua",
        "Description": "Lua scripting tools",
        "MaxMemoryMB": 256,
        "MaxExecutionTimeSeconds": 300,
        "AllowedTools": ["bash_read_file", "sqlite_analytics_query", "system_system_attach_file"],
        "AllowedHosts": ["localhost"],
        "FileSandbox": {
          "fsbPredicate": {"tag": "DirectoryRecursive", "contents": "./scripts"},
          "fsbMaxFileSize": 10485760,
          "fsbName": "lua-sandbox"
        }
      }
    }
  ],
  "skillSources": [
    { "tag": "SkillDirectory", "contents": "./skills" }
  ],
  "autoEnableSkills": ["core-utils"],
  "extraAgents": [
    {"slug": "helper", "path": "./helper.json"}
  ]
}

Module	Purpose
`System.Agents.Media.Types`	Media types for multi-modal support
`System.Agents.Tools.Base`	Core tool types
`System.Agents.Tools.Context`	Tool execution context
`System.Agents.Tools.Bash`	Bash script execution
`System.Agents.Tools.BashToolbox`	Bash tool management
`System.Agents.Tools.McpToolbox`	MCP server integration
`System.Agents.Tools.OpenAPIToolbox`	OpenAPI conversion
`System.Agents.Tools.SqliteToolbox`	SQLite tools
`System.Agents.Tools.SystemToolbox`	System information, session introspection, command execution
`System.Agents.Tools.SystemToolbox.Directory`	Directory listing
`System.Agents.Tools.SystemToolbox.Execute`	Command execution
`System.Agents.Tools.DeveloperToolbox`	Development utilities, file editing
`System.Agents.Tools.DeveloperToolbox.Read`	File reading with sessions
`System.Agents.Tools.DeveloperToolbox.Write`	File writing with sessions
`System.Agents.Tools.DeveloperToolbox.Patch`	Patch application
`System.Agents.Tools.LuaToolbox`	Lua scripting
`System.Agents.Tools.Skills.Toolbox`	Skills system
`System.Agents.Tools.Skills.Types`	Skill types
`System.Agents.Tools.Validation`	Input validation
`System.Agents.ToolPortal`	Inter-tool communication
`System.Agents.ToolRegistration`	Tool registration
`System.Agents.ToolSchema`	Schema definitions
`System.Agents.FileSandbox`	File sandbox resource management
`System.Agents.FileSandbox.Predicate`	Path predicate DSL
`System.Agents.OS.Events`	OS event types for subcall visibility

Analysis of Issue Dependencies and DAG-Aware Agent Processing

2026-10-04T15:53:54Z

Generated from documentation/sqq-dag.md, the repository is the canonical source and may be ahead of this page.

Analysis of Issue Dependencies and DAG-Aware Agent Processing

Overview

This document analyzes the current interaction between sqq-agent.sh and create-issue.sh, focusing on how issue dependencies are handled and the resulting impact on parallel development and git branch management.

Current State Analysis

1. Issue Creation (`create-issue.sh`)

Mechanism: The script takes a dependencies argument (e.g., #42,#43).
Labeling Logic:
- No dependencies: Labeled agents/to-be-taken.
- Has dependencies: Labeled agents/wait.
Limitation: The labeling is static at creation time. There is no automated process to monitor dependencies and “unlock” agents/wait issues.

2. Task Fetching (`sqq-agent.sh from_github`)

Mechanism: Queries GitHub for issues with the agents/to-be-taken label.
Limitation: It is “blind” to any issue labeled agents/wait. This requires manual intervention (human or another agent) to re-label issues once their dependencies are met.

3. Worktree Execution (`sqq-agent.sh worktree_exec`)

Branching Strategy: Always executes git worktree add "$name" "origin/main".
The “Context Loss” Problem:
- If Issue B depends on Issue A, Issue B must have the code changes from Issue A to proceed correctly (e.g., using a new Type or API defined in A).
- If Issue B is started before Issue A is merged into main, Issue B will be missing critical context.
- Currently, sqq-agent.sh has no awareness of which branch a dependency might be residing in.

4. The “Unfinished Work” Problem

Users reported that merging multiple parallel PRs can lead to main having “unfinished work” if dependencies aren’t respected.
Without a strict DAG (Directed Acyclic Graph) enforcement, agents might implement redundant or conflicting logic because they are both branching from an older state of main.

Proposed DAG-Aware Model

To resolve these issues, we propose a transition from a “label-polling” model to a “DAG-aware” model.

1. Automated Issue Promotion

We need a “Manager” process (or an update to from_github) that:

Lists all issues with agents/wait.
Parses the dependency IDs from the issue description or metadata.
Checks the status of these dependencies via gh issue view or gh pr view.
Promotion Rule: If all dependencies are “Closed” (or their associated PRs are “Merged”), the label is updated to agents/to-be-taken.

2. Branch Chaining (Optional but Powerful)

Instead of always branching from main, the system could support “Branch Chaining”:

If Issue B depends on Issue A, and Issue A’s work is in branch gh-A, Issue B could branch from origin/gh-A instead of origin/main.
Pros: Allows pipelined development without waiting for merges.
Cons: Requires complex rebasing if Issue A is updated after Issue B starts.

3. Strict Sequential Processing (Recommended)

Given the user requirement that “the processing of the second issue should wait for the first issue to be finalized”, a strict sequential approach is safer:

Rule: An issue is only promoted to to-be-taken when all its dependencies are merged into main.
This ensures origin/main always contains the necessary context for the next task.
This prevents “unfinished work” from polluting main because each task starts from a “finalized” baseline.

Implementation Recommendations

Short-term Fixes:

Update sqq-agent.sh: Add a promote_issues command that uses gh issue list --label agents/wait and checks dependency status.
Standardize Dependency Metadata: Ensure create-issue.sh stores dependencies in a machine-readable format (e.g., a specific line in the body like Depends-on: #123, #124).

Long-term Architecture:

Centralized DAG State: The sqq queue database could be extended to store task dependencies directly, allowing the scheduler to respect the DAG without polling GitHub repeatedly.
Feature Branch Isolation: Maintain the current practice of using gh-<number> branches, but ensure the BASE_BRANCH is dynamically determined if “Branch Chaining” is desired.

Summary of Benefits

Zero Manual Intervention: Issues flow from wait -> to-be-taken -> taken -> completed automatically.
Consistency: Agents always work on top of the latest “finalized” code they depend on.
Clean Main: Prevents the “unfinished work” syndrome by ensuring a logical ordering of merges.

Terminal UI (TUI)

2026-10-04T15:53:54Z

Generated from documentation/tui.md, the repository is the canonical source and may be ahead of this page.

Terminal UI (TUI)

The Terminal UI provides an interactive, real-time interface for agent conversations with support for multiple agents, streaming responses, visual feedback, file attachments, clipboard integration, subcall conversation visibility, and a tabbed interface for organizing different views.

Overview

┌─────────────────────────────────────────────────────────────────┐
│ Agents │ Chats │ History │ Help                                  │
├─────────────────────────────────────────────────────────────────┤
│  Agents                                                         │
│  ─────────────────────────────────────────────────────────────  │
│  file-assistant                                                 │
│  code-reviewer                                                  │
│  documenter                                                     │
│                                                                 │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  # Slug: file-assistant                                         │
│  # Announce: A helpful file assistant                           │
│  # Model: claude-sonnet-4-20250514                              │
│                                                                 │
│  # Tools:                                                       │
│  - [A] read_file                                               │
│  - [A] write_file                                              │
│  - [D:bash] bash_command                                       │
│                                                                 │
├─────────────────────────────────────────────────────────────────┤
│ [Tab] Switch  [Enter] Send  [Ctrl+C] Quit  [Ctrl+[|]] Prev/Next │
└─────────────────────────────────────────────────────────────────┘

Tabbed Interface

The TUI features a tabbed interface with four main tabs:

Tab	Description	Content
Agents	Browse and select agents	Agent list and detailed agent information
Chats	Active conversations	Conversation list and message interface
History	Past sessions	Session list and history view
Help	Keyboard shortcuts	Command reference and key bindings

Key	Action
`Ctrl+[`	Switch to previous tab
`Ctrl+]`	Switch to next tab

Agents Tab

The Agents tab displays:

Left sidebar: List of available agents
Main area: Detailed agent information including:
- Slug and announce text
- Model name
- Tools with activation status
- System prompt

Chats Tab

The Chats tab is for active conversations:

Left sidebar: List of ongoing conversations with status indicators:
- ⟳ - Active (agent is processing)
- ● - Waiting for input (unread)
- ⏸ - Paused
- 📎 - Has file attachments
Main area: Message editor, attachment list, the draft panel (when the conversation has unsent draft text), and conversation history

Draft (unsent text while the session is busy)

Typing a message while a conversation is active, paused, or blocked on deferred calls does not post it right away: it is appended to that conversation’s draft, one editable, unsent buffer per conversation (todos/os-as-standalone-server.md §5, D3). Three messages sent in a row while the model is thinking are almost always one message being elaborated, so each send appends a new paragraph to the draft instead of queuing a discrete message.

Collapsed view (default, shown below the message editor whenever the focused conversation has a non-empty draft): the draft’s first line plus a size indicator (N chars, M paragraphs).
Edit (Ctrl+A): loads the draft into the message editor (its attachments join the composer’s) and clears it – the editor is a full text editor over the whole draft text; further sends fold right back into a draft, or post, the normal way.
Send now (Ctrl+G): posts the draft immediately, as one message, and clears it.
Clear (Ctrl+D): discards the draft.
Ships automatically: once the session’s run stops with a status that accepts input (idle or ready – not paused, not blocked on deferred calls, not failed), the TUI posts the whole draft as a single message and clears it. A paused conversation keeps its draft until it is resumed and stops again.

The kernel never sees a draft: it only ever receives real messages and interrupts. An interrupt (Ctrl+U) always bypasses the draft and posts straight through.

History Tab

The History tab shows saved sessions, across every backend the runner is configured with:

Left sidebar: List of sessions (Client.listSessions, newest updated first), refreshed live as sessions are created, updated, or deleted elsewhere – a burst of updates during a run coalesces into at most one refresh per heartbeat, and the current selection is kept by session id across a refresh.
Main area: the selected session’s full turn history (fetched once via Client.getSession and cached by id), with the same usage summary, signal metrics, and turn navigation/forking as the Chats tab.

Help Tab

The Help tab displays keyboard shortcuts and command reference for quick access to all TUI functionality.

Architecture

The TUI is a client of the in-process runner (System.Agents.Host.Runner.SessionRunner), not a second agent runtime: agents-exe tui opens a System.Agents.Host.Host the same way agents-exe serve does and drives it through an in-process System.Agents.Host.Client.RunnerClient — every conversation action (new message, pause, fork, cancel, …) is a Command sent to the runner, and every screen update comes from Events the runner emits.

Embedded and attached. agents-exe tui (embedded) builds that client over a runner in its own process. agents-exe tui --attach URL|PATH (attached) opens nothing locally and builds System.Agents.Host.Client.Http.httpClient instead: the same RunnerClient, over a running agents-exe serve’s HTTP API and SSE event feed, on TCP or its --socket. Everything above the client is the same code, so an attached TUI loses nothing: agents, chats, drafts, pause, interrupt, hard cancel, fork, History, pending calls. What it shows depends on the server version, though: hook.failed and the subcall events only reach it from a server that emits them. Differences that follow from where the runner lives: the agents, API keys and database are the server’s (--db is refused with --attach, and --agent-file or --agent select nothing); --params-file/--set values are still sent with every create and message (secrets are resupplied by the client, D7); with --auth-tokens on the server, pass --token/--token-file, and the TUI then sees only that owner’s sessions. Quitting an embedded TUI stops the runs it started (StopRun mail) since its runner dies with it; quitting an attached TUI leaves them running on the server. A dropped event stream reconnects on its own with Last-Event-ID, so nothing is missed across a short network hiccup.

Sessions live in the SQLite database at System.Agents.CLI.ConfigLoader.defaultServerDatabasePath (next to the resolved sessions directory) unless overridden with tui --db PATH. Old sessions written by the pre-runner file store (conv.<uuid>.json, under the config’s sessions read locations) stay readable: the host composites them in as a read-only fallback (Host.hcLegacySessionDirs) behind the SQLite backend.

Sub-agent calls (prompt_agent_*) run as real sessions of their own (todos/os-as-standalone-server.md, Phase 5), so live progress for a child is its own session.updated/text.delta/tool.* stream, exactly like the parent conversation — not a separate “subcall progress” event. The TUI still gets subcall.started/subcall.completed/subcall.failed for the start/complete/fail transitions themselves (used to show the child in the conversation list and its hierarchy), but token-by-token and tool-by-tool progress in between now comes from subscribing to the child session directly, the same as any other session. A sub-agent call made with per-call narrowing (bindings/with/as) still runs in-tool, with no session and no live progress of its own, until that case is supported the same way.

Component Structure

┌─────────────────────────────────────────────────────────────────┐
│                         TUI.Core                                │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐      │
│  │   TUI.Types  │───>│  TUI.Render  │───>│  TUI.Event   │      │
│  │   (state)    │    │  (display)   │    │  (input)     │      │
│  └──────────────┘    └──────────────┘    └──────────────┘      │
│         ▲                                            │          │
│         └────────────────────────────────────────────┘          │
│                      (event loop)                               │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

State Management

-- TUI.Types
data Tab
    = AgentsTab
    | ChatsTab
    | HistoryTab
    | HelpTab
    deriving (Show, Eq)

data UIState = UIState
    { _uiFocusRing :: FocusRing WidgetName
    , _currentTab :: Tab           -- Current active tab
    , _helpContent :: [Text]       -- Help text lines
    , _turnNavigation :: Maybe TurnNavigationState
    -- ^ When Just, we are in turn navigation mode
    , _attachedFiles :: Map ConversationId [MediaAttachment]
    -- ^ Media attachments per conversation
    , _attachmentDialogState :: AttachmentDialogState
    -- ^ File attachment dialog state
    , _filePathInput :: Editor Text WidgetName
    -- ^ Editor for file path input
    , _selectedAttachmentIndex :: Maybe Int
    -- ^ Selected attachment index
    , ...
    }

data TUIState = TuiState
    { _tuiCore :: TVar Core
    , _tuiUI :: UIState
    , _eventChan :: BChan AppEvent
    , _sessionConfig :: SessionConfig
    }

Background Tool Calls

Agents configured for asynchronous execution (see async-tool-calls.md) run tool calls in the background. The TUI shows what is running, with the latest progress each tool reports:

Background tool calls running: run_tests

[Partial] > run the tests and the linter
  ✓ lint (call_1): completed
  ⏳ run_tests (call_2): running: 128 tests passed

Call states are pending, deferred, running, completed, completed (delivered later) and failed. “Delivered later” means the model had already been answered with a placeholder for that call, and got the result in a following message.

While calls run in the background and the LLM has nothing to do, the conversation still accepts input: whichever comes first — your message or the results — is sent to the model.

A conversation that waits on deferred calls (completed by an external worker) stops with a status message instead of waiting; see “Pending calls” below to answer them from the TUI itself.

Pending calls

When a run stops on deferred calls (calls.deferred), the conversation’s status becomes “blocked on deferred” and a Pending panel appears below the Draft panel, listing each call’s tool name, a prefix of its continuation token, and its arguments:

┌ Pending (1) ──────────────────────────────────────┐
│ Ctrl+O: select next | Ctrl+Y: answer selected, ... │
│                                                     │
│ > bash_command (token a1b2c3d4): {"cmd": "ls"}     │
│ - search (token 5e6f7a8b): {"q": "todo"}           │
└─────────────────────────────────────────────────────┘

> marks the selected call, the first one with a continuation token until you move it: Ctrl+O (select-pending) selects the next one, wrapping around. Ctrl+Y and Ctrl+W act on the selected call.

Ctrl+Y (answer-pending) puts the message editor into “answer mode” for the selected pending call; type the result and send it (Ctrl+Enter/the usual send trigger) the way you would any other message. That calls completeCall with autoResume = true, the same mechanism the server’s own GET /v1/sessions/:id/pending workers use, so the run resumes on its own — a fresh run.started/run.stopped pair follows. The panel clears once the run starts again.

Ctrl+W (fail-pending) fails the selected call: it completes it (the same completeCall, autoResume = true) with the text Error: <reason>, where the reason is what is in the message editor (which is cleared), or “the user declined this call” when it is empty. UserToolResponse has no dedicated error form, so a failed call reaches the model as an ordinary text result that reads as an error, the way a failing tool’s own error does. Selecting, answering and failing work the same over an embedded runner and with --attach, since they only use completeCall.

Subcall Conversation Visibility

When agents call other agents as tools (via prompt_agent_<slug>), the subcall conversations are now visible in the TUI with visual distinction and hierarchy tracking.

Visual Hierarchy

Subcall conversations are displayed with tree-branch styling to show parent-child relationships:

Conversations
├─ @file-assistant (⟳)                    -- Root conversation
│  ├─ @code-reviewer (⟳)                  -- Subcall depth 1
│  │  └─ @syntax-checker (●)              -- Subcall depth 2
│  └─ @documenter (●)                     -- Subcall depth 1
└─ @helper-agent (●)                      -- Another root conversation

Conversation Indicators

Visual	Meaning
`@slug`	Root conversation (depth 0)
`├─ @slug`	Subcall with siblings
`└─ @slug`	Last subcall in branch
`│`	Continuation line for parent with more children
Dimmed text	Subcall conversation
Normal text	Root conversation
`⟳`	Active (processing)
`●`	Waiting for input
`⏸`	Paused

Subcall Event Flow

Parent Agent (TUI visible)
       │
       ▼ calls helper agent
┌─────────────────────────┐
│ turnAgentRuntimeIntoIOTool
│ (OneShotTool.hs)        │
│                         │
│ 1. Insert into OS World │
│    - ConversationConfig │
│    - ConversationState  │
│    - Lineage (parent)   │
│                         │
│ 2. Emit SubcallStarted  │◄── OSEvent
│    (parentId, convId)   │
│                         │
│ 3. Run sub-agent        │
│                         │
│ 4. Emit SubcallProgress │◄── OSEvent (after each step)
│                         │
│ 5. Emit SubcallCompleted│◄── OSEvent (on success)
│    or SubcallFailed     │◄── OSEvent (on error)
└──────────┬──────────────┘
           │
           ▼ OS Event Queue
┌─────────────────────────┐
│   TUI Event Handler     │
│   (Event.hs)            │
│                         │
│ - Create conversation   │
│ - Show in list          │
│ - Update on progress    │
│ - Mark completed        │
└─────────────────────────┘

Subcall Event Types

-- OS Events (from System.Agents.OS.Events)
data OSEvent
    = ...
    | OSEvent_SubcallStarted
        { subcallParentConversationId :: ConversationId
        , subcallConversationId :: ConversationId
        , subcallAgentSlug :: Text
        , subcallDepth :: Int
        }
    | OSEvent_SubcallProgress
        { subcallProgressConversationId :: ConversationId
        , subcallProgressSession :: Session
        }
    | OSEvent_SubcallCompleted
        { subcallCompletedConversationId :: ConversationId
        , subcallCompletedResult :: Text
        }
    | OSEvent_SubcallFailed
        { subcallFailedConversationId :: ConversationId
        , subcallFailedError :: Text
        }

-- App Events (TUI internal)
data AppEvent
    = ...
    | AppEvent_SubcallStarted ConversationId ConversationId Text Int
    | AppEvent_SubcallProgress ConversationId Session
    | AppEvent_SubcallCompleted ConversationId Text
    | AppEvent_SubcallFailed ConversationId Text

Conversation Types

data Conversation = Conversation
    { conversationId :: ConversationId
    , conversationAgent :: TuiAgent
    , conversationThreadId :: Maybe ThreadId
    , conversationSession :: Maybe Session
    , conversationName :: Text
    , conversationChan :: BChan (Maybe UserQuery)
    , conversationStatus :: ConversationStatus
    , conversationOnProgress :: OnSessionProgress
    , conversationIsSubcall :: Bool      -- ^ NEW: Is this a subcall?
    , conversationParentId :: Maybe ConversationId  -- ^ NEW: Parent conversation
    , conversationSubcallDepth :: Int    -- ^ NEW: Nesting depth (0 = root)
    }

Rendering Subcall Hierarchy

-- Render conversations with tree structure
renderConversationForest :: [ConversationTree] -> [Widget N]
renderConversationForest trees =
    concatMap (renderTreeNode [] False) trees

-- Build tree from flat list
buildConversationForest :: [Conversation] -> [ConversationTree]
buildConversationForest convs =
    let -- Find roots (no parent or orphaned)
        isRoot c = conversationParentId c == Nothing
                || conversationParentId c `notElem` map (Just . conversationId) convs
        roots = filter isRoot convs
        
        -- Build recursively
        buildTree conv = ConversationTree conv $
            map buildTree (findChildren conv)
        findChildren parent =
            filter (\c -> conversationParentId c == Just (conversationId parent)) convs
    in map buildTree roots

-- Make prefix with tree branches
makePrefix :: [Bool] -> Bool -> Text
makePrefix ancestorIsLasts isLast
    | null ancestorIsLasts = if isLast then "└─" else "├─"
    | otherwise =
        let continuation = mconcat $
                map (\isLast' -> if isLast' then "  " else "│ ") ancestorIsLasts
        in continuation <> (if isLast then "└─" else "├─")

Visual Attributes

tui_appAttrMap :: TuiState -> AttrMap
tui_appAttrMap _ =
    attrMap Vty.defAttr
        [ ...
        , -- Subcall visual distinction
          (subcallAttr, fg white `withStyle` dim)
        , (subcallSelectedAttr, fg black `on` brightWhite `withStyle` bold)
        , (treeBranchAttr, fg white `withStyle` dim)
        , (rootConversationAttr, defAttr)
        ]

File Attachments

The TUI supports attaching files to messages for multi-modal LLM interactions.

Attaching Files

Via File Path Input:

Press Ctrl+F to open the file path input dialog
Type or paste the absolute path to the file
Press Enter to attach, Esc to cancel

Via the file browser: the dialog Ctrl+F opens lists a directory, with its current path under the listing.

Key	Action
`↑` / `↓`	Move the cursor
`Enter`	Open the directory under the cursor (`..` goes up), or attach the file under it
`Backspace`	Go to the parent directory
`/`	Filter the listing by name; `Enter` keeps the filter, `Esc` stops searching
`Esc`	Cancel (or stop a search in progress)

A directory that cannot be read (for instance for lack of permission) is reported under the listing; Backspace leaves it.

Supported file path formats:

/path/to/image.png                    # Auto-detect MIME type
image/png;/path/to/image.png          # Explicit MIME type

Attachment Display

Attached files are displayed below the message editor:

┌─────────────────────────────────────────────────────────────┐
│ Message [2 attachments]                                     │
├─────────────────────────────────────────────────────────────┤
│ > Your message here...                                      │
│                                                             │
├─────────────────────────────────────────────────────────────┤
│ Attachments (2) - Del/Backspace: remove | Ctrl+Shift+F: clear all│
│   📎 screenshot.png (image/png, 245KB)                     │
│ ▶ 📎 report.pdf (application/pdf, 1.2MB)                   │
└─────────────────────────────────────────────────────────────┘

Managing Attachments

Key	Action
`Ctrl+F`	Open file path input dialog
`Ctrl+Shift+F`	Clear all attachments
`Del` / `Backspace`	Remove selected attachment
`Up` / `Down`	Select attachment (when focus is on attachment list)

Supported File Types

The TUI can attach any file type. MIME type detection is automatic based on file extension:

Category	Extensions	MIME Types
Images	`.png`, `.jpg`, `.jpeg`, `.gif`, `.webp`, `.svg`	`image/png`, `image/jpeg`, `image/gif`, `image/webp`, `image/svg+xml`
Documents	`.pdf`, `.txt`, `.md`, `.json`, `.xml`	`application/pdf`, `text/plain`, `text/markdown`, `application/json`, `application/xml`
Audio	`.mp3`, `.wav`, `.ogg`, `.aac`, `.flac`	`audio/mp3`, `audio/wav`, `audio/ogg`, `audio/aac`, `audio/flac`
Video	`.mp4`, `.webm`, `.mov`, `.avi`	`video/mp4`, `video/webm`, `video/quicktime`, `video/avi`
Archives	`.zip`	`application/zip`

Size Limit: 50MB per file

Attachment State

-- Attachments are stored per conversation
type AttachedFiles = Map ConversationId [MediaAttachment]

data MediaAttachment = MediaAttachment
    { mediaMimeType :: Text        -- e.g., "image/png"
    , mediaBase64Data :: Text      -- Base64-encoded content
    , mediaFilename :: Maybe Text  -- Original filename
    }

-- Dialog state for file attachment
data AttachmentDialogState
    = AttachmentDialogClosed
    | AttachmentDialogPathInput

Attachment Flow

User presses Ctrl+F
       │
       ▼
┌──────────────────┐
│ Open path dialog │
│ (text input)     │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ User enters path │
│ Presses Enter    │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Load file        │
│ Detect MIME type │
│ Base64 encode    │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Add to           │
│ attachedFiles    │
│ map              │
└────────┬─────────┘
         │
         ▼
┌──────────────────┐
│ Render in        │
│ attachment list  │
└──────────────────┘

Clipboard Integration

The TUI supports pasting content from the system clipboard, including images, file paths, and text.

Clipboard Pasting

Press Ctrl+V to paste from clipboard:

Content Type	Action
Image	Save to temp file and attach
File path	Attach the file
Multiple file paths	Attach all valid files
Text	Insert into message editor

Platform Support

Platform	Backend	Required Tools
Linux (X11)	`xclip` or `xsel`	`xclip` or `xsel`
Linux (Wayland)	`wl-clipboard`	`wl-paste`
macOS	Built-in	`pbpaste` (included)
Windows	PowerShell	`powershell.exe`

Smart Content Detection

The clipboard module automatically detects content type:

data ClipboardContent
    = ClipboardImage ByteString Text    -- Image data with MIME type
    | ClipboardFilePath FilePath        -- Single file path
    | ClipboardText Text                -- Plain text
    | ClipboardFilePaths [FilePath]     -- Multiple file paths
    | ClipboardUnknown                  -- Unsupported content

Detection order:

Check for image data (via magic bytes: PNG \x89PNG, JPEG \xFF\xD8\xFF, GIF GIF87a/GIF89a, WebP RIFF)
Check for file paths (valid paths that exist)
Check for multiple file paths (one per line)
Fall back to plain text

Clipboard Module

-- System.Agents.TUI.Clipboard

-- Detect available backend
detectBackend :: IO ClipboardBackend

-- Read clipboard content
readClipboard :: ClipboardBackend -> IO (Maybe ByteString)

-- Detect content type
detectClipboardContent :: IO (Maybe ClipboardContent)

-- Analyze and convert to action
analyzeContent :: ClipboardContent -> IO ContentAction

data ContentAction
    = AttachAsMedia MediaAttachment
    | PasteAsText Text
    | AttachMultipleFiles [MediaAttachment]
    | IgnoreContent

Image Pasting from Clipboard

When an image is pasted from clipboard:

Detect image format from magic bytes
Save to temporary file with appropriate extension
Create MediaAttachment with detected MIME type
Attach to current conversation
Show status: “Attached from clipboard: image.png”

Temporary file location: $TMPDIR/agents-exe-clipboard/clipboard-*.png

Size limit: 50MB for clipboard images

File Drop Support (Terminal Protocols)

The clipboard module also supports file drop events from modern terminals:

Protocol	Terminal	Support
iTerm2 File Drop	iTerm2 (macOS)	✅ Supported
Kitty Graphics	Kitty	✅ Supported
OSC 52	Various	✅ Read support

Rendering

Tab Bar Rendering

-- TUI.Render
renderTabBar :: Tab -> Widget N
renderTabBar activeTab =
    let tabs = [AgentsTab, ChatsTab, HistoryTab, HelpTab]
        renderTab tab =
            let tabName = case tab of
                    AgentsTab -> " Agents "
                    ChatsTab -> " Chats "
                    HistoryTab -> " History "
                    HelpTab -> " Help "
                tabAttr = if tab == activeTab 
                          then activeTabAttr 
                          else inactiveTabAttr
             in withAttr tabAttr $ txt tabName
     in hBox (intersperse separator $ map renderTab tabs)

Tab-Specific Content

render_contentArea :: TuiState -> Widget N
render_contentArea st =
    case st ^. tuiUI . currentTab of
        AgentsTab -> renderAgentsTab st
        ChatsTab -> renderChatsTab st
        HistoryTab -> renderHistoryTab st
        HelpTab -> renderHelpTab st

Attachment List Rendering

render_attachmentPanel :: TuiState -> [MediaAttachment] -> Widget N
render_attachmentPanel st attachments =
    borderWithFocus
        st
        AttachmentListWidget
        (" Attachments (" <> Text.pack (show $ length attachments) <> ") ")
        $ vBox
            [ txt "Del/Backspace: remove | Ctrl+Shift+F: clear all"
            , txt ""
            , vBox $ zipWith (render_attachment_item selectedIdx) [0 ..] attachments
            ]

render_attachment_item :: Maybe Int -> Int -> MediaAttachment -> Widget N
render_attachment_item selectedIdx idx att =
    let isSelected = selectedIdx == Just idx
        marker = if isSelected then "▶ " else "  "
        filename = maybe "unnamed" id att.mediaFilename
        mimeType = att.mediaMimeType
        sizeStr = formatAttachmentSize att.mediaBase64Data
     in hBox
         [ txt marker
         , txt "📎 "
         , txt filename
         , txt " ("
         , withAttr attachmentSizeAttr $ txt mimeType
         , txt ", "
         , withAttr attachmentSizeAttr $ txt sizeStr
         , txt ")"
         ]

Conversation List with Nesting

-- Render conversation list with subcall hierarchy
render_conversationList :: TuiState -> Widget N
render_conversationList st =
    let convs = Vector.toList (listElements (st ^. tuiUI . conversationList))
        forest = buildConversationForest convs
        hasFocus = focusGetCurrent (st ^. tuiUI . uiFocusRing) == Just ConversationListWidget
        selectedId = case listSelectedElement (st ^. tuiUI . conversationList) of
            Just (_, conv) -> Just (conversationId conv)
            Nothing -> Nothing
     in borderWithFocus
            st
            ConversationListWidget
            "Conversations"
            ( viewport ConversationListWidget Both $
                vBox $ renderConversationForest st selectedId hasFocus forest
            )

renderConversationForest :: TuiState -> Maybe ConversationId -> Bool -> [ConversationTree] -> [Widget N]
renderConversationForest st selectedId hasFocus trees =
    concatMap (\(idx, tree) -> renderTreeNode st selectedId hasFocus [] (idx == length trees - 1) tree) (zip [0 ..] trees)

renderTreeNode :: TuiState -> Maybe ConversationId -> Bool -> [Bool] -> Bool -> ConversationTree -> [Widget N]
renderTreeNode st selectedId hasFocus ancestorIsLasts isLast (ConversationTree conv children) =
    let isSelected = selectedId == Just (conversationId conv)
        nodeWidget = renderNestedConversationItem st hasFocus isSelected ancestorIsLasts isLast conv
        childWidgets = concatMap (\(idx, child) -> renderTreeNode st selectedId hasFocus (ancestorIsLasts ++ [isLast]) (idx == length children - 1) child) (zip [0 ..] children)
     in nodeWidget : childWidgets

Event Handling

Tab Switching

-- TUI.Event
tui_appHandleEvent tracer ev = do
    case ev of
        -- Tab switching
        VtyEvent (Vty.EvKey (Vty.KChar '[') [Vty.MCtrl]) ->
            cycleTabBackward
        VtyEvent (Vty.EvKey (Vty.KChar ']') [Vty.MCtrl]) ->
            cycleTabForward
        -- ...

cycleTabForward :: EventM N TuiState ()
cycleTabForward = do
    current <- use (tuiUI . currentTab)
    let next = nextTab current
    tuiUI . currentTab .= next
    -- Update focus ring for the new tab
    mCurrentFocus <- use (tuiUI . uiFocusRing . to focusGetCurrent)
    tuiUI . uiFocusRing .= buildFocusRingForTabPreserving next mCurrentFocus

Subcall Event Handling

-- Handle subcall events in both normal and navigation mode
handleNormalEvent tracer ev =
    case ev of
        AppEvent (AppEvent_SubcallStarted parentId subcallId slug depth) ->
            handleSubcallStarted tracer parentId subcallId slug depth
        AppEvent (AppEvent_SubcallProgress subcallId sess) ->
            handleSubcallProgress subcallId sess
        AppEvent (AppEvent_SubcallCompleted subcallId result) ->
            handleSubcallCompleted subcallId result
        AppEvent (AppEvent_SubcallFailed subcallId err) ->
            handleSubcallFailed subcallId err
        ...

-- Create subcall conversation entry
handleSubcallStarted :: Tracer IO Trace -> ConversationId -> ConversationId -> Text -> Int -> EventM N TuiState ()
handleSubcallStarted _tracer parentId subcallId slug depth = do
    agents <- use (tuiUI . agentList . to listElements)
    case findAgentBySlug slug agents of
        Just tuiAgent -> do
            inChan <- liftIO $ newBChan 100
            let conv = Conversation
                    { conversationId = subcallId
                    , conversationAgent = tuiAgent
                    , conversationThreadId = Nothing
                    , conversationSession = Nothing
                    , conversationName = "@" <> tuiSlug tuiAgent
                    , conversationChan = inChan
                    , conversationStatus = ConversationStatus_Active
                    , conversationOnProgress = \_ -> pure ()
                    , conversationIsSubcall = True
                    , conversationParentId = Just parentId
                    , conversationSubcallDepth = depth
                    }
            coreRef <- use tuiCore
            liftIO $ atomically $ modifyTVar coreRef $ appendConversation conv
            tuiUI . conversationList %= listInsert 0 conv
        Nothing -> showStatus StatusWarning $ "Agent not found for subcall: " <> slug

-- Update subcall progress
handleSubcallProgress :: ConversationId -> Session -> EventM N TuiState ()
handleSubcallProgress subcallId sess = do
    coreRef <- use tuiCore
    liftIO $ atomically $ modifyTVar coreRef $ \c ->
        c { coreConversations = updateConversationSession subcallId sess (coreConversations c) }

-- Mark subcall completed
handleSubcallCompleted :: ConversationId -> Text -> EventM N TuiState ()
handleSubcallCompleted subcallId _result = do
    updateConversationStatus subcallId ConversationStatus_WaitingForInput
    showStatus StatusInfo "Subcall completed"

Attachment Event Handling

-- Handle Ctrl+F for file attachment
VtyEvent (Vty.EvKey (Vty.KChar 'f') [Vty.MCtrl]) -> do
    resetQuitConfirmation
    openFilePathDialog

-- Handle Ctrl+Shift+F to clear all attachments
VtyEvent (Vty.EvKey (Vty.KChar 'F') [Vty.MCtrl, Vty.MShift]) -> do
    resetQuitConfirmation
    handleClearAllAttachments

-- Handle clipboard paste
VtyEvent (Vty.EvKey (Vty.KChar 'v') [Vty.MCtrl]) -> do
    resetQuitConfirmation
    handleClipboardPaste tracer

Clipboard Paste Handler

handleClipboardPaste :: Tracer IO Trace -> EventM N TuiState ()
handleClipboardPaste _tracer = do
    hasSupport <- liftIO hasClipboardSupport
    if not hasSupport
        then showStatus StatusError "Clipboard not available - install xclip, wl-clipboard, or pbpaste"
        else do
            mContent <- liftIO detectClipboardContent
            case mContent of
                Nothing ->
                    showStatus StatusWarning "Clipboard is empty or inaccessible"
                Just content -> do
                    action <- liftIO $ analyzeContent content
                    case action of
                        IgnoreContent ->
                            showStatus StatusWarning "No attachable content in clipboard"
                        PasteAsText text -> do
                            editorContents <- use (tuiUI . messageEditor . editContentsL)
                            let newContents = TextZipper.insertMany text editorContents
                            tuiUI . messageEditor . editContentsL .= newContents
                            showStatus StatusInfo "Text pasted from clipboard"
                        AttachAsMedia attachment -> do
                            mConv <- getFocusedConversation
                            case mConv of
                                Nothing -> showStatus StatusError "No conversation selected"
                                Just conv -> do
                                    let convId = conversationId conv
                                    tuiUI . attachedFiles %= Map.insertWith (\new old -> old ++ new) convId [attachment]
                                    let filename = maybe "unnamed" id attachment.mediaFilename
                                    showStatus StatusInfo $ "Attached from clipboard: " <> filename
                        AttachMultipleFiles attachments -> do
                            mConv <- getFocusedConversation
                            case mConv of
                                Nothing -> showStatus StatusError "No conversation selected"
                                Just conv -> do
                                    let convId = conversationId conv
                                    tuiUI . attachedFiles %= Map.insertWith (\new old -> old ++ new) convId attachments
                                    showStatus StatusInfo $ "Attached " <> Text.pack (show $ length attachments) <> " files from clipboard"

Turn navigation allows you to browse through conversation history turn-by-turn and fork new conversations from any point.

Press Enter when focused on the Conversation view or Session view to enter turn navigation mode:

┌─────────────────────────────────────────────────────────────┐
│ Conversation - Turn Navigation (3/8) [Enter:exit F:fork]    │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ -----------------------                                 │ │
│ │▶ < What is the best approach for...      ← SELECTED    │ │
│ │  + ...                                                  │ │
│ │                                                         │ │
│ │ -----------------------                                 │ │
│ │  < You could consider using...                          │ │
│ └─────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘

Key	Action
`Up`	Navigate to earlier turn
`Down`	Navigate to later turn
`F`	Fork conversation at selected turn
`Enter`	Exit navigation mode
`Esc`	Exit navigation mode

Forking Conversations

Forking creates a new conversation starting from the selected turn, preserving only the turns before it:

Turn 0: User - "Hello!"
Turn 1: Assistant - "Hi there!"
Turn 2: User - "How do I..." ← Selected for fork
Turn 3: Assistant - "You can..."

Forking at Turn 2 creates new conversation with:
Turn 0: User - "Hello!"
Turn 1: Assistant - "Hi there!"
(New conversation starts here)

The original session remains untouched. The forked session has forkedFromSessionId set to the original session’s ID.

-- | State for turn-by-turn navigation
data TurnNavigationState = TurnNavigationState
    { _navSession :: Session
    -- ^ The session being navigated
    , _navSelectedTurnIndex :: Int
    -- ^ Currently selected turn index (0-based)
    , _navTotalTurns :: Int
    -- ^ Total number of turns for display
    }

-- | Widget name for turn navigation viewport
data WidgetName
    = ...
    | TurnNavigationWidget
    -- ^ For viewport scrolling during turn navigation

Message Queue Management

When a conversation is paused, you can manage queued messages - messages typed while the agent was processing.

Queue Management UI

When paused with queued messages, the Chats tab shows a queue management panel:

┌─────────────────────────────────────────────────────────────┐
│ Message                                                      │
├─────────────────────────────────────────────────────────────┤
│ > User's typed message...                                   │
│                                                             │
├─────────────────────────────────────────────────────────────┤
│ Queued Messages (2) - Ctrl+D: clear  Del: delete selected │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ ▶ First queued message text...                          │ │
│ │   Second queued message that is longer...               │ │
│ └─────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────┤
│ Conversation                                                 │
│ ...existing conversation content...                          │
└─────────────────────────────────────────────────────────────┘

Queue Management Controls

Key	Action
`Ctrl+D`	Clear all queued messages
`Del` / `Backspace`	Delete selected queued message
`Up`	Select previous message
`Down`	Select next message

Activation

Queue management is only available when:

The conversation status is ConversationStatus_Paused
There are queued messages in the buffer

To pause/unpause a conversation, press Ctrl+E.

Queue State

data UIState = UIState
    { ...
    , _uiBufferedMessages :: Map ConversationId [Text]
    -- ^ Copy of buffered messages from Core for UI rendering
    , _queuedMessagesFocus :: Maybe Int
    -- ^ Index of currently selected queued message (Nothing = none selected)
    }

-- The Core also maintains the source of truth
data Core = Core
    { ...
    , coreBufferedMessages :: TVar (Map ConversationId [Text])
    -- ^ Buffered messages per conversation
    , corePausedConversations :: Set ConversationId
    -- ^ Set of paused conversation IDs
    }

Keyboard Shortcuts

Key	Action
`Tab`	Cycle focus forward through widgets
`Shift+Tab`	Cycle focus backward through widgets
`Ctrl+Z`	Toggle zoom mode for current widget

Tabs

Key	Action
`Ctrl+[`	Switch to previous tab
`Ctrl+]`	Switch to next tab

Conversations

Key	Action
`Ctrl+N`	Start new conversation with selected agent
`Ctrl+C`	Continue restored session
`Meta+Enter`	Send message
`Ctrl+E`	Pause/unpause conversation

File Attachments

Key	Action
`Ctrl+F`	Open file path input dialog
`Ctrl+Shift+F`	Clear all attachments
`Del` / `Backspace`	Remove selected attachment
`Up` / `Down`	Select attachment (when focused)

Clipboard

Key	Action
`Ctrl+V`	Paste from clipboard (images, files, text)

Key	Action
`Enter`	Enter turn navigation mode (when on conversation)
`Up/Down`	Navigate between turns (in navigation mode)
`F`	Fork conversation at selected turn
`Enter/Esc`	Exit turn navigation mode

Key	Action
`Ctrl+D`	Clear all queued messages
`Del` / `Backspace`	Delete selected queued message
`Up/Down`	Select queued message

Session Export

Key	Action
`Ctrl+P`	Export session to markdown file
`Ctrl+T`	View session in external viewer (chronological)
`Ctrl+R`	View session in external viewer (reverse)

Session Search (History Tab)

Key	Action
`/`	Start search
`n`	Next result
`N`	Previous result
`Esc`	Clear search

Other

Key	Action
`F5`	Refresh tools for selected agent
`Esc`, `Ctrl+Q`	Quit application

Session Export and Viewing

The TUI supports exporting and viewing session content in markdown format.

Export to Markdown

Press Ctrl+p to export the current session to a markdown file:

handleDumpSessionToMarkdown :: EventM N TuiState ()
handleDumpSessionToMarkdown = do
    mSession <- use (tuiCore . coreSession)
    mConvId <- getFocusedConversationId
    case (mSession, mConvId) of
        (Just session, Just (ConversationId cid)) -> do
            let markdown = formatSessionMarkdown Chronological session
                fileName = "conv." <> show cid <.> "md"
            liftIO $ TextIO.writeFile fileName markdown
            showStatus StatusInfo $ "Exported to " <> Text.pack fileName
        ...

View with External Viewer

The TUI can display session content using an external markdown viewer configured via the AGENT_MD_VIEWER environment variable.

Chronological Order (Oldest First): Press Ctrl+t to view the session in chronological order (oldest messages first):

VtyEvent (Vty.EvKey (Vty.KChar 't') [Vty.MCtrl]) ->
    handleViewSessionWithExternalViewer Chronological

Antichronological Order (Newest First): Press Ctrl+r to view the session in reverse chronological order (newest messages first):

VtyEvent (Vty.EvKey (Vty.KChar 'r') [Vty.MCtrl]) ->
    handleViewSessionWithExternalViewer Antichronological

Example:

# Set viewer (e.g., glow, bat, less)
export AGENT_MD_VIEWER="glow -p"

# Or use a pager
export AGENT_MD_VIEWER="less -R"

# Then start TUI
agents-exe tui --agent-file agent.json

Order Preference

data OrderPreference = Chronological | Antichronological

formatSessionMarkdown :: OrderPreference -> Session -> Text.Text
formatSessionMarkdown orderPref session =
    let opts = SessionPrintOptions
            { ...
            , orderPreference = orderPref
            , ...
            }
     in formatSessionAsMarkdown opts session

Use cases:

Chronological (Ctrl+t): Best for reviewing the full conversation from start to finish
Antichronological (Ctrl+r): Best when you care about recent changes and want to see the most recent messages first

Session Search

The History tab includes session search functionality for finding past conversations.

Search Interface

┌─────────────────────────────────────────────────────────────┐
│ Sessions                                [Search: database] │
│ ─────────────────────────────────────────────────────────  │
│  ▶ 2024-01-15 10:30 - database migration                  │
│    2024-01-14 15:20 - api design                          │
│  ▶ 2024-01-13 09:00 - database schema review              │
│                                                             │
├─────────────────────────────────────────────────────────────┤
│ Session View                                                 │
│ ...selected session content...                              │
└─────────────────────────────────────────────────────────────┘

Search Features

Real-time filtering: Sessions are filtered as you type
Full-text search: Searches across session content
Highlighting: Matching terms are highlighted
Keyboard navigation: n/N to jump between results

Running the TUI

Main Entry Point

-- TUI.Core
runTUI :: Tracer IO Trace -> SessionStore -> LoadedApiKeys -> [Props] -> IO ()
runTUI tracer store apiKeys propsList = do
    let config = fileSessionConfig store apiKeys
    runTUIWithConfig tracer config propsList

runTUIWithConfig :: Tracer IO Trace -> SessionConfig -> [Props] -> IO ()
runTUIWithConfig tracer config props = do
    -- Load agent trees and create TuiAgents
    trees <- traverse loadAgentTree props
    let itrees = [tree | Initialized tree <- trees]

    -- Create TUI agents from OS-native trees
    let tuiAgents = map createTuiAgent itrees

    -- Load existing session files
    loadedSessions <- loadSessionFiles config.sessionStore

    -- Collect tools from all agents
    agentTools <- collectAgentTools tuiAgents

    -- Create event channel
    evChan <- newBChan 100

    -- Create OS event queue for subcall visibility
    osEventQueue <- newTQueueIO

    -- Start the event bridge
    startOSEventBridge osEventQueue evChan

    -- Create and initialize the OS World
    world <- atomically initWorld

    -- Create core state with World and EventQueue
    core0 <- initCore (Just world) (Just osEventQueue)
    coreTVar <- newTVarIO core0

    -- Create UI state
    let ui0 = (initUIState defaultHelpContent tuiAgents [s | (_, Just s) <- loadedSessions])
                { _uiAgentTools = agentTools }

    -- Create TUI state
    let st = TuiState coreTVar ui0 evChan config

    -- Build and run the app
    let app = App
            { appDraw = tui_appDraw
            , appChooseCursor = tui_appChooseCursor
            , appHandleEvent = tui_appHandleEvent tracer
            , appStartEvent = tui_appStartEvent
            , appAttrMap = tui_appAttrMap
            }

    void $ customMainWithDefaultVty (Just evChan) app st

Screenshots and end-to-end tests

The screenshots on this page are not taken by hand. The agents-tui-e2e test-suite runs the real agents-exe tui in a pseudo-terminal with tuispec, sends it keys, and compares the screen with a baseline at the end of each step. The model is a fake OpenAI-compatible endpoint the suite serves itself, with fixed answers, and HOME is a scratch directory: no API key, no network, none of your own configuration.

Scenario	Keys	Snapshots
`launch`	none	`agents-tab`
`chat`	`Ctrl+N`, `Ctrl+]`, `Tab` twice, a message, `Meta+Enter`	`message-typed`, `reply`
`pending`	`Down`, the same with an agent whose tool calls are deferred, then `Ctrl+Y`, an answer, `Meta+Enter`	`pending-panel`, `pending-answered`

The suite is opt-in, behind the tui-e2e cabal flag, so a plain cabal test neither builds tuispec nor needs a terminal:

# compare the screens with the baselines
cabal test agents-tui-e2e -ftui-e2e

# accept the screens as the new baselines, after a deliberate UI change
TUISPEC_UPDATE_SNAPSHOTS=1 cabal test agents-tui-e2e -ftui-e2e

# the same, and render the screenshots again
TUISPEC_UPDATE_SNAPSHOTS=1 AGENTS_TUI_E2E_PNG=1 cabal test agents-tui-e2e -ftui-e2e

Files, under test/tui-snapshots/:

Path	Content
`snapshots/<scenario>/<name>.ansi.txt`	The baseline: what the terminal received, with session ids zeroed
`snapshots/<scenario>/<name>.txt`	The same screen as plain text, to read a change in a diff
`snapshots/<scenario>/<name>.meta.json`	The terminal size (100x30)
`png/<name>.png`	The screenshot, written only with `AGENTS_TUI_E2E_PNG` set
`tests/`	What the last run captured, and a failure bundle per failed scenario; not tracked

Two screens are equal when their cells are: characters and colours, at 100 columns by 30 rows. Session ids are random and the sidebar shows them, so they are zeroed before comparing.

The PNGs are drawn by test/tui-e2e/render_png.py from the cells of the baseline (characters and colours), the way a terminal would: on the grid of the font’s advance, with the box-drawing characters as lines running from edge to edge of their cell, so that borders are continuous. It needs python3 with Pillow and a monospace TTF: DejaVu Sans Mono or Liberation Mono where distributions install them, or the file TUISPEC_FONT_PATH names. A glyph that font lacks (the status icons of the conversation list: ⧗, ⏸) is taken from Noto Sans Symbols 2, Noto Sans Math or DejaVu Sans when installed (fonts-noto-core on Debian and Ubuntu), and is the font’s missing-glyph box otherwise. The comparison itself needs none of this. Linux only: tuispec drives a PTY.

The website shows the same files: website/scripts/sync-repo-docs.sh copies them into the site, so rendering them again and publishing is all it takes to refresh the screenshots there.

AGENTS_EXE selects another binary than the one cabal just built. The scenarios are in test/tui-e2e/Main.hs; tuispec also has a JSON-RPC server (tuispec server) for driving the TUI from another program.

Styling

Attributes

tui_appAttrMap :: TuiState -> AttrMap
tui_appAttrMap _ =
    attrMap Vty.defAttr
        [ (headerAttr, fg white `on` blue)
        , (activeTabAttr, fg black `on` brightWhite `withStyle` bold)
        , (inactiveTabAttr, fg white `on` blue)
        , (userAttr, fg cyan)
        , (agentAttr, fg green)
        , (toolAttr, fg yellow)
        , (toolSuccessAttr, fg green)
        , (toolErrorAttr, fg red)
        , (systemAttr, fg magenta)
        , (inputAttr, fg white)
        , (queuedMessageAttr, fg yellow)
        , (queuedMessageSelectedAttr, bg blue `withStyle` bold)
        , (selectedTurnAttr, bg blue `withStyle` bold)
        , (attachmentAttr, fg cyan)
        , (attachmentSelectedAttr, bg blue `withStyle` bold)
        , (attachmentSizeAttr, fg white `withStyle` dim)
        , -- Subcall conversation attributes
          (subcallAttr, fg white `withStyle` dim)
        , (subcallSelectedAttr, fg black `on` brightWhite `withStyle` bold)
        , (treeBranchAttr, fg white `withStyle` dim)
        , (rootConversationAttr, defAttr)
        ]

Status Bar

render_statusBar :: Maybe StatusMessage -> Widget N
render_statusBar Nothing = emptyWidget
render_statusBar (Just msg) =
    withAttr (statusAttr msg.statusSeverity) $
        txt $ " " <> statusText msg

data StatusSeverity
    = StatusInfo
    | StatusWarning
    | StatusError
    deriving (Show, Eq)

Best Practices

Performance

Limit scrollback: Keep only last N messages in memory
Lazy rendering: Don’t render off-screen content
Rate limiting: Throttle UI updates during streaming

User Experience

Visual feedback: Show typing indicators and tool calls
Error handling: Display errors without crashing
Help text: Always show keyboard shortcuts
Tab organization: Group related functionality into logical tabs
Pause before queue management: Queue management only works when paused to prevent accidental modifications

Multi-Agent UI

Clear indicators: Show which agent is active
Separate contexts: Each agent maintains its own conversation
Easy switching: Tab between agents quickly
Subcall visibility: Show nested agent calls with visual hierarchy

Conversation Forking

Non-destructive: Original session always preserved
Clear lineage: Forked sessions track their origin
Agent selection: Current agent selection used for fork
Navigation mode: Enter navigation to review before forking

File Attachments

Size limits: Warn users about large files
MIME detection: Automatic type detection from extensions
Visual feedback: Show attachment count in UI
Temporary cleanup: Clipboard images are temporary files

Clipboard Integration

Graceful degradation: Handle missing clipboard tools gracefully
Security: Validate file paths before attachment
Size limits: Prevent memory issues with large clipboard content
Platform detection: Auto-detect best clipboard backend

Subcall Visibility

Tree rendering: Show parent-child relationships clearly
Visual distinction: Use different attributes for subcalls vs root conversations
Event bridging: Convert OS events to AppEvents for Brick integration
Orphan handling: Handle async race conditions where child arrives before parent
Nesting depth: Track and display recursion depth

Customization

Custom Event Handlers

customHandleEvent :: TUIState -> BrickEvent () CustomEvent -> EventM () (Next TUIState)
customHandleEvent state ev = case ev of
    -- Add custom shortcuts
    VtyEvent (EvKey (KChar 's') [MCtrl]) -> do
        liftIO $ saveSession state
        continue $ state { status = Ready }

    VtyEvent (EvKey (KChar 'l') [MCtrl]) -> do
        newState <- liftIO $ loadSession state
        continue newState

    _ -> handleEvent state ev  -- Fall through to default

Custom Widgets

customProgressBar :: Float -> Widget ()
customProgressBar progress =
    let width = 20
        filled = round (progress * fromIntegral width)
        bar = replicate filled '█' ++ replicate (width - filled) '░'
    in withAttr progressAttr $ str $ "[" ++ bar ++ "]"

Module	Purpose
`System.Agents.TUI.Core`	Main TUI application
`System.Agents.TUI.Types`	TUI state and types
`System.Agents.TUI.Render`	Rendering functions
`System.Agents.TUI.Event`	Event handling
`System.Agents.TUI.Clipboard`	Clipboard and drag-and-drop support
`System.Agents.Media.Types`	Media attachment types
`System.Agents.SessionPrint`	Session formatting
`System.Agents.OS.Events`	OS event types for subcall visibility
`System.Agents.AgentTree.OneShotTool`	Subcall execution

agents-exe

Iterative Development Workflow

Iterative Development Workflow

Invocation Patterns

Pattern 1: Fresh Start (No Previous State)

Pattern 2: Continue Previous Work

Pattern 3: Handle New Code Changes

Pattern 4: Deep Dive on Subsystem

Chunking Strategy

By Subsystem

By Document

State Tracking Between Iterations

Recursive Documentation

When Documenting Recursion Features

Decision Tree

Handoff Notes

Common Iteration Sequences

Sequence: Full Documentation Pass

Sequence: New Feature Documentation

Sequence: Maintenance Update

Tool Usage by Phase

Discovery Phase

Analysis Phase

Documentation Phase

Verification Phase

Anti-Patterns to Avoid

Success Metrics

Quick Reference Card

Spec: bash tools v2, long-running tool processes

Spec: bash tools v2, long-running tool processes

Goal

1. The describe document, v2

1.1 process: what kind of process this is

1.2 runtime: how to run it

1.3 commands and help: several actions in one binary (from #539)

1.4 sandbox: what the process may touch, without images

1.5 Health: a real probe, and ordered backends per capability

2. The runtime side

2.1 A oneshot v2 tool

2.2 A stream tool

2.3 A service tool: the process handle

2.4 Process tools: one typed send per family, one shared processes

3. What the model sees, end to end

Decisions (proposed)

Resolved in review (2026-09-25, PR #574 comments)

Open questions

File Loader

File Loader

Overview

Core Types

FileLoader (System.Agents.FileLoader)

FileLoader.JSON (System.Agents.FileLoader.JSON)

FileLoader.Section (System.Agents.FileLoader.Section)

Directory Scanning

listJsonDirectory

With Tracing

JSON Parsing

Agent Description Parsing

JSON Schema

Section Extraction

Use Case: Multi-part Prompt Files

Extracting Sections

Named Sections

Configuration Loading

agents-exe.cfg.json

Configuration Schema

Error Handling

JSON Parse Errors

File Not Found

Validation

File Watching

Hot Reload Support

File Notification

Best Practices

Configuration Files

Section Extraction

Directory Structure

Examples

Loading Multiple Agents

Extracting Code from Markdown

1. The `describe` document, v2

1.1 `process`: what kind of process this is

1.2 `runtime`: how to run it

1.3 `commands` and `help`: several actions in one binary (from #539)

1.4 `sandbox`: what the process may touch, without images

2.1 A `oneshot` v2 tool

2.2 A `stream` tool

2.3 A `service` tool: the process handle

2.4 Process tools: one typed `send` per family, one shared `processes`

FileLoader (`System.Agents.FileLoader`)

FileLoader.JSON (`System.Agents.FileLoader.JSON`)

FileLoader.Section (`System.Agents.FileLoader.Section`)

Parameters: `$ctx`

The `agents` library

`agents-exe` (process scope)

`agents-server` (session and message scope)

`whenUnbound: "expose"`

Passing parameters to sub-agents (`with`)

Naming a narrowing (`derive_agent`)

Session Types (`System.Agents.Session.Types`)

Media Types (`System.Agents.Media.Types`)