Add bot graphics

Bots can now show a concept as a visual: create_graphic (with
update_graphic, read_graphic and list_graphics, on every run, no computer
needed) stores one HTML document in bot_graphics, per user like the
conversation, and the chat renders its [graphic:ID] ref as an inline card
sized to fit, with a large view and a page of its own (no rail).

Graphic HTML is treated as hostile: it is only served at /graphics/<id>
with a CSP sandbox allowing scripts under an origin of its own and no
network, loading only the vendored Chart.js and Mermaid from
/graphics-lib/. The document carries the app's theme vars and scrollbars,
reports its height to the thread, and a frame that navigates away is
closed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-27 08:11:15 +01:00
co-authored by Claude Opus 5.5
parent 51fd4d5538
commit bea656ade8
65 changed files with 1959 additions and 21 deletions
+13 -2
View File
@@ -6,6 +6,7 @@ import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db";
import {
ActionCheckNote,
SecretsNote,
GraphicsNote,
botFilesNote,
ComputerFilesNote,
ScheduledRunNote,
@@ -31,6 +32,7 @@ import askToTakeOverTool from "@/src/harness/tools/computer/ask-to-take-over-too
import importFileTool from "@/src/harness/tools/computer/import-file-tool";
import searchComputersTool from "@/src/harness/tools/computer/search-computers-tool";
import grabBotFileTools from "@/src/harness/tools/storage/grab-bot-file-tools";
import grabBotGraphicTools from "@/src/harness/tools/graphics/grab-bot-graphic-tools";
import reportRunFailedTool from "@/src/harness/tools/bot/report-run-failed-tool";
import computerRef from "@/src/utils/computer-ref";
import grabRunScopes from "@/src/functions/backend/db/bot-files/grab-run-scopes";
@@ -79,8 +81,8 @@ export default async function runBotTask({
const dirs = computerRunDirs({ bot_id: bot.id, bot_kind: bot.kind, user_id, workspace_id });
/**
* The secrets the bot captured or showed, whose chips the report
* always carries.
* The secrets the bot captured or showed and the graphics it made or
* changed, whose chips the report always carries.
*/
const shown: string[] = [];
@@ -225,6 +227,14 @@ export default async function runBotTask({
scopes,
onAttach: ({ media_id }) => attachments.push(media_id),
})),
...grabBotGraphicTools({
bot_id: bot.id,
user_id,
session_id,
workspace_id,
job_id: job.id || null,
onGraphic: ({ ref }) => shown.push(ref),
}),
];
extensions = await openBotExtensions({
@@ -260,6 +270,7 @@ export default async function runBotTask({
TasksNote,
ActionCheckNote,
SecretsNote,
GraphicsNote,
workspace_id ? WorkspaceLearningNote : "",
powers.is_coordinator
? `If this brief asked you to record something in a workspace (its context, or a new one), do it yourself with update_workspace or create_workspace before you report: the report alone changes nothing.`