Open computers from the chat, let bots ask for a take-over, show task progress

- Serve each computer to its owner under /computers/<id>/ (HTTP and
  websockets proxied to the AIO sandbox) and show it in a shared
  ComputerView with browser, terminal and code modes, opened from the
  computers modal or a take-over button in the thread.
- ask_user_to_take_over lets a bot hand a computer to the user for a
  login or captcha; the message gets a take-over button and needs-you,
  and the viewer hands it back with a chat message.
- While a task runs the thread shows "<bot> is working..." with the
  task's latest step; steps read as plain words, not tool names.
- A coordinator with no workers no longer claims to delegate, and bots
  describe started tasks in the first person.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-19 05:35:44 +01:00
co-authored by Claude Opus 5.5
parent b1023ba8ff
commit 56daf0c462
44 changed files with 1289 additions and 75 deletions
+14 -2
View File
@@ -2,6 +2,7 @@ import BunSQLite from "@moduletrace/bun-sqlite";
import type { BUN_SQLITE_COBBLE_BOTS, BUN_SQLITE_COBBLE_COMPUTERS } from "@/db/types/db";
import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers";
import type { HarnessTool } from "../../types";
import askToTakeOverTool from "../computer/ask-to-take-over-tool";
import searchComputersTool from "../computer/search-computers-tool";
import assignComputerTool from "./assign-computer-tool";
import delegateTaskTool from "./delegate-task-tool";
@@ -19,12 +20,17 @@ type Params = {
* The user message being answered.
*/
message_id?: number | string;
/**
* Called when the bot asks the user to take over a computer.
*/
onTakeOver?: (params: { ref: string }) => void;
};
/**
* The tools a bot replies in chat with: its own (memory, instructions),
* start_task for longer work, search_computers to find what's on its
* computers, and for the coordinator delegate_task and assign_computer
* computers, ask_user_to_take_over when only the user can do something on
* one, and for the coordinator delegate_task and assign_computer
* (which computers each crew member can use). Anything
* else on a computer goes through a task, never mid-reply.
*/
@@ -33,6 +39,7 @@ export default async function grabBotChatTools({
user_id,
computers,
message_id,
onTakeOver,
}: Params): Promise<HarnessTool[]> {
const bot_id = bot.id || 0;
const tools = [
@@ -43,7 +50,12 @@ export default async function grabBotChatTools({
computers,
message_id,
}),
...(computers.length ? [searchComputersTool({ bot_id, computers })] : []),
...(computers.length
? [
searchComputersTool({ bot_id, computers }),
askToTakeOverTool({ computers, onAsk: (params) => onTakeOver?.(params) }),
]
: []),
];
if (bot.kind != "coordinator") return tools;
+1 -1
View File
@@ -60,7 +60,7 @@ export default function startTaskTool({
computer_id: computer_id || undefined,
});
return `Task #${job_id} queued. Tell the user you're on it; the result will be posted here when it's done.`;
return `Task #${job_id} queued; you'll work on it yourself in the background. Tell the user in one sentence, in the first person, what you're doing (e.g. "I'm checking your inbox now. I'll report back when I'm done."); the result will be posted here when it's done.`;
},
};
}