next-tjwater-agent/docs/agent-ui-protocol.md

Agent UI Protocol

Controlled content and display commands

answer-ui@1 describes retained answer content rendered by the frontend component catalog. Stable block_id, process/final phase and standalone [[ui:block_id]] anchors govern placement. Publishing or reading answer content never replays a map action.

Map, chart, overlay and SCADA display use a single flat command object: {action, ...typed fields}. Chat calls present_view({command, reason?}) with an optional concise purpose; tasks call publish_task_presentation({request_id, command}); Dev JSONL wraps the same command in {record_type:"presentation", command}. No executable JS, JSX, HTML, CSS, URL or arbitrary request is accepted. The former chat name/params action shape and frontend-action@1 are rejected; no migration adapter is provided.

The ten commands are locate_features, zoom_to_map, highlight_features, clear_highlights, open_chart, close_chart, set_overlay, clear_overlay, render_scada_analysis and clear_scada_analysis. Historical data must first be queried through an authorized MCP tool and then supplied to open_chart. Command definitions, including bounded GeoJSON, charts and target lists, are generated from the supply frontend's src/features/agent/presentation/command-schema.ts; the exported src/frontendAction/command.ts is also used with OpenCode's native Zod instance. Regenerate using node scripts/export-view-command-contract.mjs in the supply frontend; add --check to verify copies. The same exporter also synchronizes tests/fixtures/unified-presentation.jsonl and chart-publication-cases.json to Agent and Server. JSON Schema covers structural limits; runtime refinement and the shared conformance tests cover timestamp semantics.

Chat transport and receipts

The frontend advertises frontend-action@2, and discovers supported actions through GET /api/v1/frontend-action-registry, schema_version frontend-action-registry@2.

Generation uses POST /api/v1/responses. A tjwater.frontend_action SSE event carries response_id, sequence_number and data containing:

{
  "version": "frontend-action@2",
  "actionId": "host-action-id",
  "toolCallId": "native-call-id",
  "sessionId": "session-id",
  "command": {"action":"highlight_features","layer":"pipes","feature_ids":["533793"],"effect":"pulse"},
  "risk": "view",
  "replayPolicy": "never",
  "issuedAt": 1710000000000,
  "expiresAt": 1710000015000
}

Session subscriptions carry the same data in a frontend_action event. The browser validates the active session and command, executes in stream order, and submits frontend-action-result@2 through POST /api/v1/frontend-actions/:action_id/result, with the scoped session header. Results contain actionId, status, output or error, and completedAt. Status is succeeded, failed, rejected, cancelled or expired. Locate/highlight output reports actual matched and missing IDs; publishing a tool call is not proof of display success. Duplicate delivery reuses the terminal receipt, not the map side effect. Session switches cancel running and queued work; expiry prevents later execution.

Task and preview lifetimes

Task display is asynchronous: Server verifies the task/run/generation, active lease, source session, evidence ownership and idempotency key before recording operational.presentation.requested. A recorded result does not mean a browser displayed it; tasks progress without any connected browser. New subscriptions take a fresh event baseline and do not replay historical display commands.

Task chart windows are managed by the browser in one parent window containing process chart groups and report Chart components. Only new live open_chart commands for the followed running investigation can automatically open that parent once per task-view session; updates preserve dismissal and never automatically open individual child windows. Opening, switching or refreshing a task, loading completed tasks, reconnecting history, and navigating saved snapshots restore data without opening windows. Each child still supports manual independent enlargement. Chat and Dev previews use manual opening. Window intent is transient browser state, not a field in the command or saved event; displayed means chart data was loaded, not that a popup opened.

Task open_chart requires persisted evidence references. Task set_overlay can omit data; Server resolves immutable evidence into bounded GeoJSON. Supplied data must match that resolution and never overrides the stored snapshot. Chat and Dev supply resolved data for structural validation before rendering.

Same-time-of-day current/history comparisons use chart_type=time_comparison; ordinary charts retain line or bar. The comparison axis contains shared UTC+8 HH:mm or HH:mm:ss slots, with current line and historical scatter series and aligned real sample_times. The Server automatically validates new task presentation and report chart publications, rejecting date-appended axes, missing timestamps, misplaced points and nonhistorical reference dates with TIME_COMPARISON_INVALID. Chat open_chart is checked by both the native tool and host coordinator using the same generated chart refinement as task publication; Dev commands and ui_block.Chart use it too. Ordinary line/bar charts skip comparison semantics, but a line/scatter comparison cannot be labelled line. Historical stored chart read contracts remain unchanged.

Chat, task following and Dev previews share the frontend command dispatcher but own independent map sources, SCADA layers and chart state. Clearing or switching one owner does not clear another. Dev JSONL is an explicit local preview/import/export container, not an alternative online business protocol. Plain data records and ui_block records do not execute embedded actions. The earlier agent-ui@1 registry has no active tool mappings and is not an action entry point.

Running and waiting input delivery

Text output has separate append, flush and close operations. Smoothing releases English words and bounded Chinese chunks at the configured interval (20 ms by default). A short tail that cannot form a regular chunk waits at most 100 ms after input starts or the last visible release; this deadline never dumps a whole queued paragraph. Text segment completion, tool transitions and permission/question waits drain preceding text at the same pace before the next event. Publishing present_answer also drains preceding text but leaves output open for later narration. Only terminal cleanup closes output. Cancellation immediately drains already-received text before native interruption; it does not manufacture a final answer. The first-token log records emission from this buffer, rather than input admission into it.

Ordinary chat and task collaboration share one final-answer handoff. A completed final_answer is held until all observed tools are terminal and the inbox close guard permits completion. The host then emits tjwater.final_answer exactly once before interrupting further native rewrites and waiting for native idle. Plain-text and component answers follow the same rule. Tool arguments are not published as partial JSON; answer content and component references must pass their existing validation before publication. Background task_execution keeps its native idle and Server completion checks; a chat answer never completes a business task.

Running text additions use POST /api/v1/responses with stream=false, conversation, tjwater.delivery="steer" | "queue" and a stable tjwater.input_id. HTTP 202/tjwater.response_input is an inbox receipt, not execution completion. The existing SSE connection carries tjwater.user_input pending/delivered states; delivered means the native inbox took the input. GET /api/v1/session/:session_id/inputs lists pending input; PATCH of its /:input_id delivery can change its mode before processing and returns 409 afterward. Restored session streams retain these semantics.

A pending host question or failed continuation does not disable input admission. With no active host turn, steer/queue messages enter the native inbox with resume=false and remain available for the next owned continuation. This does not answer a host question or launch inference. A lost delivery acknowledgement can be reconciled only after verifying both native consumption and completed related output. HTTP 409 with code RESPONSE_FINISHED means the input was rejected before admission because the turn has ended; the client may refresh state and send it as a new response. Other failures do not authorize automatic resubmission.

Chat state projections

src/routes/chat-run-accumulator.ts consumes native and host events before publishing them. Permission replies (including permission_response with status="error" and error), question reconciliation and abort must use publishRunEvent, rather than mutate legacy messages. Canonical parts are the saved history; reconnect compatibility fields are projections of the same events. data-run_status retains completed, waiting_input, error or aborted state for reloads. Host-confirmed interaction updates may update stored parts after a response has finished without reopening its wire stream. UI, redacted transcript and native execution history retain their separate purposes and retention rules.

Grouped object chart examples

Keep one stable chart_id per object. A comparison title is <object_id> <shared title> and every series name starts with <object_id> ; IDs start with a letter or digit and contain only letters, digits, underscores, periods or hyphens. Comparisons group only when their shared title, chart_type, axis names, x_data, current series names without the object prefix and current sample_times match. Different metrics, units or windows stay separate. The browser places single-selection object buttons on the right and series legends under the plot; narrow windows place objects below. Closing a grouped card removes its objects, while close_chart still addresses a single original chart_id. Ordinary overview/bar commands and standalone answer Chart components stay independent. No objects field or new command is introduced.

The complex Dev examples publish eight pressure and four pipe-flow object datasets, each with 96 slots, one current line and seven sparse historical scatter series, plus an independent plan bar chart. The canonical JSONL and shared positive/negative chart cases are copied from the frontend fixtures by the contract exporter; do not edit generated fixture copies locally. Publication checks reject inconsistent dates, misplaced slots, missing observed timestamps and invalid null pairings before execution. Browser tests use synthetic readings and do not establish live Agent or business acceptance.