docs(canvases): document canvas-renderer plugin slot for v0.4 (true OSS extensibility)

The previous "Tier 3: add a renderer needs a core PR" framing wasn't
honest open-source extensibility — it was just a permissive license.
This commit reframes Tier 3 as a renderer plugin model:

- Anyone can ship `@aoagents/ao-plugin-canvas-{name}` packages.
- Plugin declares own canvasType id, payload schema (Zod), and
  React renderer.
- AO's build-time plugin registry walks installed packages, generates
  a TypeScript file extending the CanvasArtifact discriminated union,
  and the web build bundles the renderers.
- Trust boundary is `npm install` — same as every other AO plugin.
- No core PR. No runtime code injection. Build-time only.

Adds a new Tier 4: promote a popular renderer plugin into core's
built-in set when the ecosystem has converged on it. The bar is
"this is now standard infrastructure", not "this is a new idea".

Updates the contract from "constrained UI in core" to three rules:
1. Anyone supplies any data, in any supported type.
2. Anyone ships a new type via plugin (npm trust).
3. Nobody dynamically loads JS at runtime.

The non-goal list now distinguishes "dynamically-loaded React"
(rejected) from "build-time-bundled renderer plugins" (the v0.4
unlock). Mirrors the change across docs/canvases.md (source of
truth), docs/canvases-launch.{md,html} (showcase), and
docs/canvases-feature.html (full writeup).

No code change in v0.1. The plugin slot lands in v0.4 with its own
design pass — this commit just documents the direction.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ashish Huddar 2026-05-09 22:42:26 +05:30
parent 89cf367e5f
commit c7a79a88e8
4 changed files with 140 additions and 51 deletions

View File

@ -189,7 +189,7 @@
<span class="pill">v0.1 — file-based + synthesized git diff</span>
<span class="pill">4 renderer types</span>
<span class="pill">13 codex review passes</span>
<span class="pill">no new plugin slot</span>
<span class="pill">v0.4: renderer plugins</span>
</div>
<section class="tldr">
@ -435,24 +435,47 @@
<div class="tier">
<div class="tier-head">
<span class="tier-num">TIER 3</span>
<h3>Add a new <code>CanvasType</code></h3>
<span class="tier-effort">~half a day · core PR required</span>
<h3>Ship a renderer plugin</h3>
<span class="tier-effort">~half a day · v0.4 · no core PR</span>
</div>
<p>For data that genuinely doesn't fit the 4 existing renderers (flame graphs, Gantt charts, network topologies):</p>
<ol>
<li>Extend the <code>CanvasArtifact</code> discriminated union in <a href="../packages/core/src/types.ts"><code>types.ts</code></a>.</li>
<li>Extend the matching Zod schema in <a href="../packages/core/src/canvas-schema.ts"><code>canvas-schema.ts</code></a>.</li>
<li>Write a <code>Canvas{NewType}.tsx</code> renderer in <a href="../packages/web/src/components/"><code>packages/web/src/components/</code></a>.</li>
<li>Add a <code>case</code> to the switch in <a href="../packages/web/src/components/CanvasRail.tsx"><code>CanvasRail.tsx</code></a>.</li>
<li>Tests + docs paragraph.</li>
</ol>
<p>Gated behind a core PR because the renderer ships in the supervisor dashboard. Worth it once <strong>at least two real callers</strong> need the same shape.</p>
<p>For data that genuinely doesn't fit the 4 existing renderers (flame graphs, Gantt charts, network topologies), publish your own renderer as <code>@aoagents/ao-plugin-canvas-{name}</code> — the same package convention as every other AO plugin. The plugin declares its own <code>canvasType</code> id, payload schema (Zod), and React component.</p>
<p>At AO build time the plugin registry walks installed <code>@aoagents/ao-plugin-canvas-*</code> packages, generates a TypeScript file that imports each renderer plus extends the <code>CanvasArtifact</code> discriminated union, and the web build bundles them. The dashboard's <code>CanvasRail</code> switch dispatches to the plugin's renderer when it sees the new type.</p>
<pre><code>@aoagents/ao-plugin-canvas-flamegraph/
├── package.json <span class="cmt"># name: @aoagents/ao-plugin-canvas-flamegraph</span>
├── src/
│ ├── index.ts <span class="cmt"># manifest + payload schema (Zod)</span>
│ └── renderer.tsx <span class="cmt"># React component, default export</span></code></pre>
<pre><code><span class="cmt">// src/index.ts</span>
import { z } from "zod";
export const manifest = {
name: <span class="str">"flamegraph"</span>,
slot: <span class="str">"canvas-renderer"</span> as const,
canvasType: <span class="str">"flamegraph"</span>, <span class="cmt">// new CanvasType id</span>
version: <span class="str">"0.1.0"</span>,
};
export const payloadSchema = z.object({
samples: z.array(z.object({
name: z.string(),
value: z.number(),
})).max(<span class="str">10_000</span>),
unit: z.enum([<span class="str">"ms"</span>, <span class="str">"samples"</span>]),
});</code></pre>
<p><strong>No core PR needed.</strong> Trust boundary: <code>npm install</code>. Same trust check as adding any other AO plugin. No runtime code injection — plugins are bundled at build time, not loaded dynamically. v0.1 ships the 4 built-ins; v0.4 (queued) ships the plugin loader.</p>
</div>
<div class="tier">
<div class="tier-head">
<span class="tier-num">TIER 4</span>
<h3>Promote a renderer plugin into core</h3>
<span class="tier-effort">rare · core PR · ecosystem consensus</span>
</div>
<p>Once a renderer plugin has <strong>multiple production callers</strong> and the type is genuinely general (not specific to your stack), propose promoting it into core's built-in set via PR. The bar here is <em>"this is now standard infrastructure"</em>, not <em>"this is a new idea worth trying"</em>. Tier 3 is for trying ideas; Tier 4 is for blessing what already worked.</p>
</div>
</div>
<div class="callout">
<strong>The contract</strong> — expressive data, constrained UI. Anyone supplies any data in any supported type (no permission needed). New types come from PRs (UI in core needs review). No third party ships JS into the supervisor dashboard (never planned). Keeps install one-step, the renderer set consistent across every AO instance, and the supervisor sandbox-safe.
<strong>The contract</strong>three rules across all four tiers. Anyone supplies any data, in any supported type (no permission needed). Anyone ships a new type via plugin (same <code>npm install</code> trust as any other AO plugin). Nobody dynamically loads JS at runtime (no remote code, no agent-emitted React). This keeps the trust boundary at install time where it belongs, the renderer set consistent within a given AO build, and the supervisor sandbox-safe.
</div>
<h2 id="paranoia">6. Built with paranoia</h2>
@ -549,19 +572,20 @@ EOF</code></pre>
<thead><tr><th>Version</th><th>Adds</th><th>Status</th></tr></thead>
<tbody>
<tr><td><code>v0.1</code></td><td>File reader, GET endpoint, 4 renderers, synthesized git-diff, right-rail with auto-expand, 5s REST poll</td><td><strong>Shipped</strong></td></tr>
<tr><td><code>v0.2</code></td><td><code>CanvasProducer.listCanvases</code> invoked on agent / scm / tracker plugins</td><td>Queued</td></tr>
<tr><td><code>v0.2</code></td><td><code>CanvasProducer.listCanvases</code> invoked on agent / scm / tracker plugins (Tier 2 above)</td><td>Queued</td></tr>
<tr><td><code>v0.3</code></td><td>Mux WebSocket topic for live updates, replacing 5s poll</td><td>Queued</td></tr>
<tr><td><code>v0.4</code></td><td><code>canvas-renderer</code> plugin slot (Tier 3 above). Build-time bundling of <code>@aoagents/ao-plugin-canvas-*</code>, dynamic discriminated-union extension, type-id collision detection</td><td>Queued — the OSS extensibility unlock</td></tr>
<tr><td>Mobile</td><td>Bottom-sheet or full-screen takeover for canvases on small viewports</td><td>Deferred</td></tr>
<tr><td>Custom UI</td><td>Sandboxed iframe escape hatch, build-time allowlisted renderer packages</td><td>Only if justified</td></tr>
<tr><td>Sandboxed iframe</td><td>Runtime-isolated escape hatch for genuinely-untrusted content (e.g. third-party HTML emitted by an agent we can't trust)</td><td>Only if a real use case shows up</td></tr>
</tbody>
</table>
<h2 id="non-goals">10. Deliberate non-goals</h2>
<ul>
<li><strong>Custom React renderers from third-party plugins.</strong> Security boundary; never planned. New renderer types come from core PRs.</li>
<li><strong>Dynamically-loaded React from third-party sources at runtime.</strong> URL-injected, agent-emitted, or remote-loaded renderers — never planned. v0.4's plugin slot bundles renderers at <em>build time</em> only, with the same <code>npm install</code> trust check as every other AO plugin. That's the OSS extensibility line: trust at install, not at runtime.</li>
<li><strong>Write APIs from the dashboard back into canvases.</strong> Read-only surface in v0.1. Bidirectional adds a whole new failure mode (auth, ordering, conflict).</li>
<li><strong>Action buttons that mutate session state.</strong> A canvas displays; a button-click <em>does</em>. The latter is a different feature with different security implications.</li>
<li><strong>A 9th plugin slot for canvases.</strong> Considered and rejected — canvas is product output, not infrastructure. Existing plugins opt-in via <code>CanvasProducer</code>.</li>
<li><strong>A 9th plugin slot for canvas <em>producers</em>.</strong> Considered and rejected — producing a canvas is session output, not infrastructure. Existing plugins opt-in via <code>CanvasProducer</code>. (v0.4's <code>canvas-renderer</code> slot is different: that's UI infrastructure, where a slot makes sense.)</li>
</ul>
<footer>

View File

@ -301,24 +301,28 @@ EOF
<p>This is what a future <code>scm-github</code> plugin would do to surface a "PR status" stats canvas, or what a <code>tracker-linear</code> plugin would do for "linked issues" tables. v0.1 declares the interface but doesn't invoke it yet — once v0.2 lands, every plugin slot (agent / SCM / tracker) gets called automatically when the API loads canvases.</p>
<p>Effort: ~30 lines per producer. No dashboard changes, no schema changes, no PR review of UI code.</p>
<h3>Tier 3 — A new renderer type</h3>
<p>If your data genuinely doesn't fit <code>markdown</code> / <code>diff</code> / <code>table</code> / <code>stats</code> (e.g. a flame graph, a Gantt chart, a network topology), you add a new <code>CanvasType</code> to core. That's:</p>
<ol>
<li>Extend the <code>CanvasArtifact</code> discriminated union in <a href="../packages/core/src/types.ts"><code>types.ts</code></a> and the matching Zod schema in <a href="../packages/core/src/canvas-schema.ts"><code>canvas-schema.ts</code></a>.</li>
<li>Write a <code>Canvas{NewType}.tsx</code> renderer in <code>packages/web/src/components/</code>.</li>
<li>Add a <code>case</code> to the switch in <a href="../packages/web/src/components/CanvasRail.tsx"><code>CanvasRail.tsx</code></a>.</li>
<li>Tests plus a paragraph in <a href="canvases.md"><code>docs/canvases.md</code></a>.</li>
</ol>
<p>Effort: ~half a day, gated behind a core PR. Worth it when at least 2 real callers need the same shape.</p>
<h3>Tier 3 — Renderer plugin (v0.4, ~half day)</h3>
<p>Ship a new canvas type as <code>@aoagents/ao-plugin-canvas-{name}</code> — the same package convention as every other AO plugin. The plugin declares its own <code>canvasType</code> id, payload schema (Zod), and React renderer. At AO build time the plugin registry walks installed <code>@aoagents/ao-plugin-canvas-*</code> packages and bundles their renderers into the dashboard.</p>
<p><strong>No core PR needed.</strong> Trust boundary is <code>npm install</code> — same as adding <code>@aoagents/ao-plugin-tracker-linear</code> or <code>@aoagents/ao-plugin-notifier-slack</code>.</p>
<p>Sketch:</p>
<pre><code>@aoagents/ao-plugin-canvas-flamegraph/
├── package.json
├── src/
│ ├── index.ts # manifest + payload schema (Zod)
│ └── renderer.tsx # React component, default export</code></pre>
<p>v0.1 ships the 4 built-in types. v0.4 (queued) ships the plugin slot. Until then, if you genuinely need a new type, either reshape your data into one of the 4 existing renderers, or contribute the type to core.</p>
<h3>The trade-off — and what we deliberately won't do</h3>
<p>Third parties can ship canvases without forking AO, but they cannot ship arbitrary React components that the dashboard runs. The contract is <strong>expressive data, constrained UI</strong>:</p>
<h3>Tier 4 — Promote into core (rare)</h3>
<p>Once a renderer plugin has multiple production callers and the type is genuinely general, propose promoting it into core's built-in set via PR. The bar here is "this is standard infrastructure now", not "this is a new idea worth trying".</p>
<h3>The contract</h3>
<p>Three rules that hold across all four tiers:</p>
<ul>
<li>Anyone supplies any data, in any of the supported types — <strong>no permission needed</strong>.</li>
<li>Anyone proposes a new type via PR — <strong>review needed because it's UI in core</strong>.</li>
<li>Nobody ships JS into the supervisor dashboard — <strong>never planned</strong>.</li>
<li><strong>Anyone supplies any data, in any supported type</strong>no permission needed.</li>
<li><strong>Anyone ships a new type via plugin</strong> — same <code>npm install</code> trust as any other AO plugin.</li>
<li><strong>Nobody dynamically loads JS at runtime</strong> — no remote code, no agent-emitted React. Plugins are bundled at build time only.</li>
</ul>
<p>This keeps the install one-step (no per-plugin bundling), the renderer set consistent across every AO instance, and the supervisor sandbox-safe (no third-party code in the app shell).</p>
<p>This keeps install boundaries clear (one trust check per plugin, at install time), the renderer set consistent within a given AO build, and the supervisor sandbox-safe (no third-party code injected at runtime).</p>
<h2 id="deferred">What's deliberately not in v0.1</h2>
<ul>

View File

@ -98,26 +98,37 @@ This is what a future `scm-github` plugin would do to surface a "PR status" stat
Effort: ~30 lines per producer. No dashboard changes, no schema changes, no PR review of UI code.
### Tier 3 — A new renderer type
### Tier 3 — Renderer plugin (v0.4, ~half day)
If your data genuinely doesn't fit `markdown` / `diff` / `table` / `stats` (e.g. a flame graph, a Gantt chart, a network topology), you add a new `CanvasType` to core. That's:
Ship a new canvas type as `@aoagents/ao-plugin-canvas-{name}` — the same package convention as every other AO plugin. The plugin declares its own `canvasType` id, payload schema (Zod), and React renderer. At AO build time the plugin registry walks installed `@aoagents/ao-plugin-canvas-*` packages and bundles their renderers into the dashboard.
1. Extend the `CanvasArtifact` discriminated union in [`types.ts`](../packages/core/src/types.ts) and the matching Zod schema in [`canvas-schema.ts`](../packages/core/src/canvas-schema.ts).
2. Write a `Canvas{NewType}.tsx` renderer in `packages/web/src/components/`.
3. Add a `case` to the switch in [`CanvasRail.tsx`](../packages/web/src/components/CanvasRail.tsx).
4. Tests + a paragraph in [`docs/canvases.md`](canvases.md).
**No core PR needed.** Trust boundary is `npm install` — same as adding `@aoagents/ao-plugin-tracker-linear` or `@aoagents/ao-plugin-notifier-slack`.
Effort: ~half a day, gated behind a core PR. Worth it when at least 2 real callers need the same shape.
Sketch:
### The trade-off — and what we deliberately won't do
```
@aoagents/ao-plugin-canvas-flamegraph/
├── package.json
├── src/
│ ├── index.ts # manifest + payload schema (Zod)
│ └── renderer.tsx # React component, default export
```
Third parties can ship canvases without forking AO, but they cannot ship arbitrary React components that the dashboard runs. The contract is **expressive data, constrained UI**:
v0.1 ships the 4 built-in types. v0.4 (queued) ships the plugin slot. Until then, if you genuinely need a new type, you can either reshape your data into one of the 4 existing renderers, or contribute the type to core.
- Anyone supplies any data, in any of the supported types — **no permission needed**.
- Anyone proposes a new type via PR — **review needed because it's UI in core**.
- Nobody ships JS into the supervisor dashboard — **never planned**.
### Tier 4 — Promote into core (rare)
This keeps the install one-step (no per-plugin bundling), the renderer set consistent across every AO instance, and the supervisor sandbox-safe (no third-party code in the app shell).
Once a renderer plugin has multiple production callers and the type is genuinely general (not specific to your stack), propose promoting it into core's built-in set via PR. The bar here is "this is standard infrastructure now", not "this is a new idea worth trying".
### The contract
Three rules that hold across all four tiers:
- **Anyone supplies any data, in any supported type** — no permission needed.
- **Anyone ships a new type via plugin** — same `npm install` trust as any other AO plugin.
- **Nobody dynamically loads JS at runtime** — no remote code, no agent-emitted React. Plugins are bundled at build time only.
This keeps install boundaries clear (one trust check per plugin, at install time), the renderer set consistent within a given AO build, and the supervisor sandbox-safe (no third-party code injected at runtime).
## What's deliberately *not* in v0.1

View File

@ -6,8 +6,8 @@ This is AO's answer to Cursor's [canvases](https://cursor.com/docs/agent/tools/c
## Design principles
1. **Expressive data, constrained UI.** Producers supply data in one of a fixed set of renderer types. They do not ship React components. This keeps the dashboard install one-step, avoids running third-party JS in the supervisor, and keeps the rendered surface consistent across every AO instance.
2. **No new plugin slot.** Canvases are session output, not infrastructure. Existing plugins (agent, SCM, tracker) opt in by implementing `CanvasProducer`. Core also synthesizes canvases from PR / CI / git data so the feature works without per-agent integration.
1. **Two layers of extensibility.** v0.1 ships a fixed set of 4 renderer types — agents drop JSON, the dashboard renders it, no permission needed. v0.4 adds a **canvas-renderer plugin slot** so anyone can ship a new type as `@aoagents/ao-plugin-canvas-{name}` without a core PR. Trust boundary is `npm install` (the same trust boundary as every other AO plugin), not runtime code injection.
2. **No third-party JS at runtime.** Renderer plugins are bundled at build time, not loaded dynamically. The dashboard never executes code that wasn't in the install. This rules out remote code from agent JSON, but doesn't rule out community-shipped renderer types.
3. **Workspace is the source of truth.** Agents write canvases to `{workspacePath}/.ao/canvases/{id}.json` — same pattern as `activity.jsonl`. Core reads, validates, size-caps, and exposes through the dashboard API. The dashboard never reads agent files directly.
4. **Pull, then push.** v0.1 polls a REST endpoint on the session detail page. Live updates ride on the existing mux WebSocket later — no new SSE channel.
@ -22,7 +22,7 @@ Defined in [`packages/core/src/types.ts`](../packages/core/src/types.ts) as `Can
| `table` | Test results, dependency lists, anything tabular | `{ columns, rows }` |
| `stats` | Cost, token counts, durations, pass/fail counts | `{ metrics: CanvasStatMetric[] }` |
If your data doesn't fit, **shape it to fit one of these** before reaching for a new type. A "test results" canvas is a `table`. A "deployment status" canvas is `stats` plus `markdown`. Adding a new renderer type requires a core PR — propose it as an issue first with at least two real use cases.
If your data fits one of these, just emit JSON — Tier 1 in the [extension model](#extension-model). If it genuinely doesn't (flame graph, Gantt chart, network topology), ship a **renderer plugin** in v0.4 — Tier 3 below. Don't reach for the core PR path until a renderer plugin has multiple ecosystem callers and we want to promote it into the built-in set.
## Producing canvases
@ -110,12 +110,62 @@ Canvases that fail validation are dropped silently and logged. Rules:
Synthesized canvases (PR, CI, cost) are computed on read and not persisted. If a canvas needs to survive workspace cleanup, persist it via the session metadata directory in core — not from the producer.
## Extension model
Three tiers of effort, smallest first.
### Tier 1 — Emit JSON (today, ~3 minutes)
Drop a file at `{workspacePath}/.ao/canvases/{id}.json` matching one of the 4 built-in schemas. The dashboard polls every 5s and renders it. No code change in AO. This is what 90% of agents will do.
### Tier 2 — `CanvasProducer` plugin (v0.2, ~30 lines)
Synthesize canvases programmatically from session data. Implement `CanvasProducer.listCanvases` on an existing `agent` / `scm` / `tracker` plugin. The interface lives in [`types.ts`](../packages/core/src/types.ts). v0.1 declares it; v0.2 invokes it from the canvases API endpoint.
### Tier 3 — Renderer plugin (v0.4, ~half day)
Ship a new canvas type as `@aoagents/ao-plugin-canvas-{name}`. Same trust boundary as every other AO plugin: you `npm install` it, AO bundles the renderer into the dashboard at build time. **No core PR needed.** No runtime code execution from arbitrary sources.
Sketch of the plugin shape (subject to refinement during v0.4 design):
```
@aoagents/ao-plugin-canvas-flamegraph/
├── package.json # name: @aoagents/ao-plugin-canvas-flamegraph
├── src/
│ ├── index.ts # manifest + payload schema (Zod)
│ └── renderer.tsx # React component, default export
```
```typescript
// packages/plugins/canvas-flamegraph/src/index.ts
import { z } from "zod";
export const manifest = {
name: "flamegraph",
slot: "canvas-renderer" as const,
canvasType: "flamegraph", // the new CanvasType discriminator
version: "0.1.0",
};
export const payloadSchema = z.object({
samples: z.array(z.object({ name: z.string(), value: z.number() })).max(10_000),
unit: z.enum(["ms", "samples"]),
});
```
At AO build time the plugin registry walks installed `@aoagents/ao-plugin-canvas-*` packages, generates a TypeScript file that imports each renderer plus extends the `CanvasArtifact` discriminated union, and the web build bundles them. The dashboard's `CanvasRail` switch-on-type dispatches to the plugin's renderer for unknown built-in types.
Trust model: same as adding any plugin — you reviewed it before installing, and its code lives in your `node_modules`. No remote loading, no runtime injection. If you don't trust a plugin, don't install it; this is the npm trust chain, not a sandbox.
### Tier 4 — Promote into core (rare, when ecosystem has converged)
Once a renderer plugin has multiple production callers and the type is genuinely general, propose promoting it into core's built-in set via PR. This is the path that requires reviewer scrutiny — the bar is "this is now standard infrastructure", not "this is a new idea worth trying".
## Roadmap
- **v0.1 (shipped)** — file reader, `GET /api/sessions/[id]/canvases`, four built-in renderers, `core.git-diff` synthesized canvas, right-rail in `SessionDetail` **desktop only** (auto-expands when canvases exist), 5s REST poll with visibility-aware pause.
- **v0.2**`CanvasProducer` invoked on agent / SCM / tracker plugins.
- **v0.2**`CanvasProducer` invoked on agent / SCM / tracker plugins (Tier 2 above).
- **v0.3** — mux topic for live updates, replacing poll.
- **v0.4**`canvas-renderer` plugin slot (Tier 3 above). Build-time bundling of `@aoagents/ao-plugin-canvas-*` packages, dynamic discriminated-union extension, type-id collision detection at registry load.
- **Mobile** — deferred. The rail is gated `!isMobile` in [`SessionDetail.tsx`](../packages/web/src/components/SessionDetail.tsx); below the mobile breakpoint the page falls back to its existing single-column layout. A proper mobile UI (bottom sheet or full-screen takeover) is a separate design pass.
- **Later, only if justified** — sandboxed iframe escape hatch for custom UI, build-time allowlisted renderer packages.
- **Sandboxed iframe escape hatch** — only if a real use case (e.g. third-party HTML emitted by an agent that we genuinely cannot trust) shows up. Adds runtime isolation cost but trades it for unlimited content flexibility.
Out of scope indefinitely: dynamic React imports from third-party plugins, write APIs from the dashboard back into canvases, action buttons that mutate session state.
Out of scope indefinitely: **dynamically loaded** React from third-party sources at runtime (URL-injected, agent-emitted, etc.) — only build-time bundled plugins. Write APIs from the dashboard back into canvases. Action buttons that mutate session state.