cobble/AGENTS.md

# Agent Instructions

This is **Cobble** (`cobble.moduletrace.org`): an open-source crew of small AI worker bots. Unit noun is **bot**, never "agent" or "expert".

**The handoff is outdated.** `assets/handoff/` (gitignored locally: `PRODUCT.md`, `IMPLEMENTATION.md`, `tokens/`, `assets/`) is the *foundation*, not the design: it is what the project started from, and the app has moved well past it. Its layouts, screens (`assets/screens/*.png`), right-hand "working on" panel, and PRODUCT/IMPLEMENTATION briefs describe an early single-chat prototype and must not be used as reference for the current UI or copied into docs, screenshots or marketing. What is still authoritative in it:

- **Brand assets:** the stone icons (`public/brand/icons/*.svg`, mirrored into `src/data/stone-geometry.ts`), the coordinator (`public/brand/coordinator/`), the app icon (`public/brand/app-icon/`), and the wordmark (`public/brand/wordmark/`, mirrored into `src/data/wordmark-paths.ts`). These are the identity and do not change; the mark is a stone with exactly one flat cut, a bot's colour is its identity (never status), the coordinator is hue-less.
- **Tokens:** `tokens/tokens.css` was the origin of the `:root` / `.dark` block in `src/styles/main.css`. From here on `src/styles/main.css` is the source of truth; treat the handoff CSS as historical.

For the current UI, read the code: `src/layouts/bots/` (rail, workspace rail, user popover), `src/pages/admin/bots/[bot_id]/` (conversation header, thread, composer, modals) and `src/pages/admin/workspaces/` (dashboard). When the handoff and the current code or the user disagree, the code and the user win, always.

## Stack

- **Runtime:** Bun - not Node. Use `bun` commands, never `npm`/`npx`.
- **Framework:** `@moduletrace/bunext` - not Next.js. Routing is filesystem-based under `src/pages/`, served by `Bun.serve` in `src/server.ts`.
- **Database:** `@moduletrace/bun-sqlite` - SQLite database. Schema in `db/schema.ts`, generated types in `db/types/db.ts`. Config in `bun-sqlite.config.ts`.
- **UI:** Tailwind CSS v4 + the `@moduletrace/twui` component library (installed package) + shared components in `src/components/general/`. There is **no** shadcn/ui, no `tailwind.config`, and no `components.json` - twui is the only component system. CSS chain: `src/pages/__root.tsx` imports `src/styles/main.css`, which imports `@moduletrace/twui/base.css` (`@import "tailwindcss"`). Theme tokens (colors/radius/fonts) are overridden via the `@theme inline` block in `src/styles/main.css`.
- **Icons:** `lucide-react`.
- **Validation:** `zod` v4 (`[email protected]`).
- **Branding:** Site name/slug live only in `src/data/site-data.ts`. Logo source of truth is `public/logos/logo.svg` (the ember stone; rendered inline via `BotStone` in `src/components/general/logo-icon.tsx`); browser favicon/assets are generated from it. Handoff brand assets (stones, coordinator, wordmarks, app icon) live in `public/brand/`.

## Entrypoint

`src/server.ts` - turns on SQLite foreign keys (`enableForeignKeys`, so the schema's cascades fire; off by default per connection), runs `bunextInit()`, starts `Bun.serve`, then starts cron:

- `/media/*` short-circuits to the media server handler
- bunext's responses go through `compressResponse` (`src/functions/server/compress-response.ts`): page scripts and styles bunext names after their content (`isHashedAssetPath`) are cached `immutable` and compressed once (brotli or gzip, kept in memory within `CompressCacheMaxFiles` and `CompressCacheMaxBytes`, with parallel requests sharing one pass); pages and API JSON are gzipped as they stream; media, streams and small bodies go out as they are. Every variant carries `Vary: Accept-Encoding`. `public/sw.js` serves every `.bunext/public` file from the build's own cache (see PWA). Keep page props small: every page is a full server render, so anything only a modal needs loads when the modal opens. The chat header's models modal fetches the install-wide catalog once (`api/admin/harness/catalog`) and the conversation's small setting separately (`api/admin/bots/harness-overview`), both kept in `localStorage` (the catalog for `AppData.HarnessCatalogCacheMs`, the setting for `AppData.HarnessOverviewCacheMs`) and cleared on logout or after a model save (`src/utils/harness-cache.ts`). `bun run check:bundle` (run in CI after a production build) fails if the conversation page's initial JS passes its budget or the full Lucide set returns. Never render a lucide icon by name (twui's `LucideIcon`, a string `beforeIcon`): that ships every icon; import the icon
- `/ws` upgrades only after `userAuth` succeeds (session frozen into the socket at upgrade time); the `websocket` handler object lives in `src/websocket/`
- everything else goes to `bunext.bunextRequestHandler`
- `ensureDockerNetwork()` makes sure the computers' docker network (`AppData.DockerNetworkName`) exists before the server starts; without docker the process exits (computers are containers, so nothing works without it). Creating a computer's container checks it again first
- `cron()` and the bots queue handler (`botsQueue()`, `src/bots-queue/`) start after the server is up, then `liveRelay()` (`src/live-relay/`, every process, ungated); in production cron and the queue are gated to the `MAIN_SERVER` container

## Commands

| Purpose                          | Command                                                   |
| -------------------------------- | --------------------------------------------------------- |
| Dev server                       | `bun run dev` (runs `bun src/server.ts`)                  |
| Production                       | `bun run start` (`MAIN_SERVER=1 NODE_ENV=production bun src/server.ts`, no build: it serves whatever `.bunext/` holds, a dev build included, and builds only when there is none; build for production first with `NODE_ENV=production bunx bunext build`) |
| Type-check only                  | `bun run typecheck`                                       |
| Bundle budget (after a build)    | `bun run check:bundle`                                    |
| Run tests                        | `bun run test`                                            |
| Regenerate favicon/brand/PWA assets | `bun run gen:favicon` (runs `src/scripts/generate-brand-assets.ts`) |
| DB schema (tables)               | `bun run db:schema`                                       |
| DB admin UI                      | `bun run db:admin`                                        |
| DB backup                        | `bun run db:backup`                                       |
| Seed super admin user            | `bun run seed` (runs `src/scripts/seed-super-admin.tsx`)  |
| Seed sample bot crew             | `bun run seed:bots [email]` (runs `src/scripts/seed-bots.ts`, idempotent: adds sample messages to the existing coordinator and the sample workers, skips users who already created workers; assigns the sample workers to them) |
| Query DB (CLI)                   | `bun llm-code-assistant/query-db.ts "<sql>"`              |
| Install                          | `bun install`                                             |
| Publish a release to GitHub      | `bun run publish:github` (`scripts/publish-github.sh`)    |
| Publish a release to Gitea       | `bun run publish:gitea` (`scripts/publish-gitea.sh`)      |

No linter or formatter configured. `bun run typecheck` and `bun run test` are the verification steps. Tests are colocated `.test.ts`/`.test.tsx` files - run one with `bun test <file>`.

## Querying the Database

Inspect live data with `bun llm-code-assistant/query-db.ts "<sql>"` - a read-only SQL CLI that loads the schema from `db/schema.ts` automatically and prints a normalized JSON envelope (`success`, `payload`, `count`, `insert_return`, `error`, `msg`). Exit code `1` on failure. Use it instead of fishing through ORM code when you need to know what's actually in the database.

## Path Alias

`@/*` resolves from repo root. Import like `@moduletrace/twui/layout/Button`, `@/src/components/general/logo`, `@/src/data/site-data`, `@/db/types/db`.

## Architecture

### Auth

- **Auth guard:** `src/pages/__root.server.ts` - server function that runs on every request.
    - No public pages: anything outside the live admin sections (`AdminSections` in `src/dict/admin-sections-dict.ts`: `/admin/bots`, `/admin/workspaces`, `/admin/routines`, `/admin/logs`, `/admin/help`, `/admin/settings`) and `/auth` redirects to `AdminHomePath` (`/admin/bots`). `/admin` itself only redirects there
    - No users at all: everything redirects to `/auth/signup` (first-run super admin setup, `grabHasUsers`); once a user exists `/auth/signup` redirects away
    - Redirects unauthenticated `/admin/*` → `/auth/login`
    - Unknown `/admin/settings/*` pages (not in `SettingsPages`, `src/utils/is-unknown-settings-path.ts`) redirect to `/admin/settings/general`
    - Already-authenticated users hitting `/auth/*` (except paths containing `/logout`) are redirected to `/admin/bots`; login and signup land there too
- **Session:** encrypted auth cookie (AES-256-GCM, PBKDF2-derived key from `ENCRYPTION_KEY`/`ENCRYPTION_SALT`) plus a CSRF cookie, both `Secure` only when the request came over HTTPS (`isSecureRequest`, `src/utils/is-secure-request.ts`: the URL's scheme or `X-Forwarded-Proto`), so plain HTTP over a VPN or LAN address can still sign in. `userAuth` (`src/functions/backend/auth/user-auth.ts`) decrypts, checks expiry (`AuthExpiryDays`, 60) against the embedded `date`, verifies the CSRF cookie against the token embedded in the session (constant-time, `src/utils/safe-compare.ts`), and re-reads `user_types` from the DB every request (revoked users lose access immediately).
- **Login:** local install, so email or username plus password only (`api/auth/password-login.ts`, rate limited), no emailed code step and no emailed password reset. Google login (`api/auth/google-login.ts`) is for existing accounts only and shows when `GOOGLE_CLIENT_ID` is set; it never creates accounts. Sessions are issued by `src/functions/backend/auth/login-user.ts` (supports `reauth` for silent refresh without audit logging).
- **Signup:** first-run only. `api/auth/signup.tsx` refuses once any user exists, creates the super admin (`create-super-admin-user.ts`, so user #1) with a required password, and logs them in. No Google signup. `bun run seed` (needs `SEED_SUPER_ADMIN_PASSWORD`) is the headless alternative.
- **Passwords:** bcrypt hashes live in the separate `user_passwords` table (never in `users`; blocked from the CRUD API via `src/dict/private-tables.ts`). Set/change at `/admin/settings/change-password`.
- **Auth layout:** `/auth/*` (except `/auth/logout`) renders in `src/layouts/auth/` with no site header/footer.
- **User types** (`UserTypes` in `src/dict/user-types-dict.ts`, the only place they're defined): `super_admin` (the first account, runs the install), `admin` (manages members; only the super admin grants it), `member` (a regular user: new users start with it), `read_only`, `revoked` (can't sign in). A user needs at least one to sign in. Admins grant and remove `member`, `read_only` and `revoked` on anyone but admins and the super admin.
- **RBAC:** `src/utils/check-user-access.ts` (includes/excludes primitive, `super_admin` bypass) plus the declarative engine in `src/utils/rbac/` - `userTypeAssignment` interprets `UserTypeAssignmentMatrix`/`UserTypeDeletionMatrix` in `src/dict/user-types-dict.ts`. `read_only` users are blocked from non-GET CRUD in the router.
- **Audit trail:** `src/functions/backend/logger.ts` - `logEvent`/`logError` write to the `logs` table. Signup, login, Google login, logout, cron runs, and WebSocket connects are all logged.

### WebSocket

- Upgrades on `/ws` with `userAuth` validation (401 if not logged in); `WebSocketSession` (`src/websocket/types.ts`) carries `{ user, user_types }`.
- Rooms declared in `src/dict/websocket-rooms.ts` -> typed `WebsocketRoomType`. Store in `src/websocket/(data)/rooms.ts`: `WeakMap` for per-client metadata, `Map<room, Set<ws>>` for lookups, empty rooms deleted. Join/leave/broadcast in `src/websocket/(functions)/`.
- Message dispatch in `handle-message.ts`: `client:ping` → `server:pong`, unknown events → `error`. The twui `useWebSocket` hook sends a raw `"twui::ping"` keepalive string - non-JSON messages are ignored.
- Client handlers: `src/websocket/client-requests/<event>/index.ts` (folder-per-event), replying through `wsSendData`.

### Cron

`src/cron/index.ts` is the single registration point called from `server.ts`. Tasks live in one folder each (`src/cron/<name>/index.ts`) as default-exported async functions, scheduled with in-process `Bun.cron` via `registerCronJob({ title, schedule, task, run_on_start, quiet })` (`src/cron/register-cron-job.ts`), which logs before each run (unless `quiet`) and catches/logs errors. Schedules are cron expressions or nicknames (`@hourly`) interpreted in UTC; runs never overlap. Gated in production by `MAIN_SERVER`. The bundled `heartbeat` task doubles as a DB-liveness probe; `sync-harness-models` refreshes the model catalog hourly; `run-cron-jobs` (every minute, quiet) queues the bots' due schedules (see Schedules).

### Admin CRUD API

`api/admin/crud/[[...paths]].ts` dispatches `GET/POST/PUT/DELETE` by table name: `userAuth` → `read_only` GET-only gate → per-table switch → generic fallback (super admin only). Tables without custom handlers fall through to generic CRUD with zero code. Customize a table by adding `src/pages/api/admin/crud/(tables)/<table>/` with `index.ts` + per-method files, then add a `case` in the router.

### Bots (the chat shell)

- **Access (the crew is shared):** bots, workspaces and computers belong to the install, not to a user (their `user_id` is only who created them, not a foreign key). Nobody reaches a bot until a super admin or admin grants it: `bot_users` (a worker assigned to a user, for its own chat) and `workspace_users` (a user added to a workspace, to chat with every worker in it there). `crewRole` (`src/utils/rbac/crew-role.ts`): `super_admin` reaches everything and is the only one who ever sees, chats with or edits the coordinator; `admin` reaches every worker and workspace; anyone else (`member`) only their grants. `canManageCrew` (super admins and admins) creates, edits and deletes workers and workspaces, sets their models and pictures, and hands out access (the Bots and Workspaces groups on a user's settings page, `user-access-groups.tsx`, each added or taken away at once by `api/admin/users/add-access` and `remove-access` through `setUserGrant`); the structural DB helpers check it themselves (`assertCrewManager`, `src/functions/backend/db/access/`). The SQL conditions live in `src/utils/crew-access/` (`chatBotsSql`, `workspacesSql`, `workspaceBotsSql`, `computersSql`, tested against SQLite) and back `grabChatBot` (its own chat), `grabUserBot` (anywhere: its own chat or a workspace), `grabUserWorkspace(s)`, `grabWorkspaceBot`, `grabBots` and `grabUserComputer(s)`. Conversations are per user (`bot_sessions.user_id`): a job runs for the user whose session it's in (`runBotJob` reads it), with that user's memory and only what they reach. A bot's face and status line are per user too (`bot_user_faces`, set by `setBotFace` for the job's user, `grabJobUserId`; no row is idle), so needs-you or working only shows to the user the job is for; every bot read for a page overlays the viewer's (`botFaceSql`, `src/utils/bot-face-sql.ts`). Live updates go to the session's user (`pushToSessionUser`), a face change to its user, and a bot change (its computer) to everyone who sees it (`pushBotUpdated`, `grabBotAudience`, each with their own face and active task); its status line never names a task (`BotTaskStatusLine`). Bot and workspace pictures are served to whoever sees them (`grabSharedPictureMedia`). A member with nothing granted lands on an empty `/admin/bots`; a workspace's dashboard is open to everyone in it, its settings link and computers' settings links only to `canManageCrew`. `src/scripts/share-crew.ts` (safe to rerun) moved an install from one crew per user to this.
- **Tables:** `bots` (`kind` worker|coordinator, `color` (a free `#rrggbb` the user picks) and `stone` (which flat-cut shape it's drawn on), `role`, `system_prompt` (the bot's core instructions, markdown); `hue` is the palette column they replaced, read only as a fallback), `bot_user_faces` (`face`, `status_line` per bot and user), `bot_sessions`, `bot_messages`, `bot_tasks`, `bot_task_steps`. Enums in `src/dict/bot-dict.ts`.
- **Threads and sessions:** each user has one conversation with a bot in its own chat and one per workspace it's in (the coordinator in every workspace). Each is split into `bot_sessions` (`user_id`, `bot_id`, `workspace_id` null for its own chat, `status` active|closed, `title` and `summary` written by the bot from the first exchange, `summary_message_count` (how many messages the summary covers), `previous_session_id`, `unread`, `last_activity_at`); `bot_messages`, `bot_tasks` and `bots_queue` carry `session_id` and `workspace_id`. Helpers in `src/functions/backend/db/bot-sessions/`: `ensureBotSession` (the active one, else a new one; `rollover` starts the next once it has `AppData.BotSessionMaxMessages`), `startBotSession`, `reopenBotSession`, `grabBotSessions` (search over title, summary and message text), `grabUserConversation` (access and membership), `grabConversationProps` (the conversation pages' props). Once a session's first exchange is in, `name-bot-session.ts` has the bot name it (the title the rail and list show) and write its summary, after the reply is pushed; a new session's first reply gets the previous one's summary (`grab-session-carry-over.ts`), rewritten first when it covers fewer messages than that session ended with. The header's sessions icon (`src/components/general/bot-sessions-modal.tsx`) lists, searches, starts and reopens sessions, and the new-session icon beside it (`(partials)/new-session-button.tsx`) starts one (`api/admin/bots/list-sessions`, `new-session`, `reopen-session`). The rail lists the open bot's latest `AppData.RailRecentSessions` sessions under its row (`rail-bot-sessions.tsx`), and once there are more, "All sessions" opens the same modal. Unread, the rail's last message and activity are per thread, from its sessions. Job results go into the job's own session, even after it closes.
- **Coordinator:** the install has exactly one `kind: coordinator` bot, the super admin's alone, that coordinates the crew and delegates to every worker. It is created at signup (`bun run seed`) and, as a fallback, by the super admin's `/admin/bots`, all via `ensureCoordinatorBot` (`src/functions/backend/db/bots/ensure-coordinator-bot.ts`, idempotent). Name in `CoordinatorBotName` (`src/dict/bot-dict.ts`). Its default prompt is `CoordinatorSystemPrompt` (`src/data/coordinator-system-prompt.ts`): attached at creation, and backfilled onto a coordinator whose prompt is empty, never over an edited one.
- **Routes:** `/admin/bots` opens the bot this browser was last in (`readLastVisitedBot` from the single `cobble_last_visited_bot_id` cookie, which the conversation pages set through `withLastVisitedBotCookies` in their `res_transform`, and which `/auth/logout` clears; a conversation inside a workspace is never remembered, so the landing page always reopens the bot's own chat), but only while `grabReachableConversation` says the user still reaches it. Otherwise it ensures the coordinator and redirects the super admin to its chat (`/admin/bots/1` on a fresh install), anyone else to their latest bot, else their first workspace, else an empty page; `/admin/bots/[bot_id]` renders the conversation (header, thread and composer, the last two held to 760px; there is no right panel); `/admin/bots/[bot_id]/models` is a worker bot's own model settings, for super admins and admins (the coordinator's redirects to its chat); `/admin/workspaces/[workspace_id]` is a workspace's dashboard (`(sections)/workspace-dashboard.tsx`: read-only picture, name, description and context, latest activity, bots with their per-workspace instructions and memory, computers; its header links into `/admin/settings/workspaces/[workspace_id]`, whose `workspace-overview.tsx` has the editable groups) and `/admin/workspaces/[workspace_id]/bots/[bot_id]` is the same `Conversation` section scoped to the workspace (`pageProps.workspace`; only the coordinator and member bots). All render in the full-bleed `src/layouts/bots/` layout (rail of conversations + user popover), which `__root.tsx` uses for every `/admin/*` page.
- **Server:** `src/functions/backend/db/bots/` (`grab-bots`, `grab-bot-messages`, `grab-active-bot-task`, `grab-user-bot`, `create-bot`, `insert-user-message`). `api/admin/bots/send-message.ts` posts a user message to a bot the session user chats with (zod: `src/schemas/bot-schema.ts`), queues the bot's reply (`enqueueBotReply`) and wakes the queue (`src/bots-queue/wake-bots-queue.ts`; never import `src/bots-queue/index.ts` from a page or API route, it pulls the bunext runtime into the page bundles). `upload-attachment.ts` stores one file for the next message (private `media` row, paradigm `bot-attachment`, type from `attachmentFileKind`, only a file whose extension the extractor unpacks gets `text_content` - `canExtractText`, cut to `AppData.BotAttachmentTextStoreMaxChars` - limits in `AppData.BotAttachment*`) and `send-message` takes their `media_ids` (checked to be the user's own by `grabAttachableMedia`), linked in `bot_message_media`; messages carry `attachments` (`BotMessageWithAttachments`). `mark-read.ts` clears a session's unread when a message arrives in the open conversation; The conversation pages render everything on the server (`grabConversationProps`: rail, bot, header's model and computers, composer's tools and voice, files) except the history, so the header and composer show at once. The history loads on the client: `Conversation` posts to `api/admin/bots/conversation-history` (`use-conversation-history`, `grabConversationHistory`: the active session, marked read, its latest messages and the bot's active task there; or a `redirect` when the user can't chat there any more) with a `LoadingLine` (`src/components/general/loading-line.tsx`, the `animate-loading-line` theme animation in `src/styles/main.css`) under the header meanwhile, and again on a back-forward cache restore (`pageshow` with `persisted`), so the back button never shows an old thread. A load is merged with what the thread already has (`merge-loaded-history`: messages sent or pushed while it was on its way stay) and resets the older-messages loader and the task progress. The history holds a session's latest `AppData.ConversationFirstMessages` messages (`bot_messages_more` says there are older), and the thread loads older ones `AppData.ConversationOlderMessages` at a time as it scrolls to its top (`older-messages.ts`, `use-older-messages`, `older-messages-loader.tsx`). Bot and workspace lists in page props go through `railBots` / `railWorkspaces` (`src/utils/`): no core instructions, memory or workspace context, and the last message cut to a preview (`AppData.RailPreviewMaxChars`); the header avatar's edit form loads the instructions when it opens (`bot-instructions.ts`, `use-bot-instructions`). `session-messages.ts` returns a session's latest messages, which the open conversation polls every `AppData.BotReplyCatchUpMs` while a reply is due (`use-reply-catch-up`), so a reply whose live push was missed (socket down) still lands without a reload (replayed as a `server:bot-message`, so the rail and unread take it too); `create-bot.ts` adds a worker (and puts it in a workspace when given one); `save-harness-settings.ts` saves a worker bot's models.
- **Composer:** a Tiptap editor (`(hooks)/use-composer-editor.ts`, `(partials)/composer-editor.tsx`), not a textarea, loaded with the page and focused automatically, that reads and writes markdown (`@tiptap/markdown`), so drafts and sent messages are markdown. Drafts are written to `localStorage` on a 250ms debounce (`useComposerDraft`), flushed on unmount. Bold, italic, strike, code, lists, quotes and links (no headings), from shortcuts, markdown typing or the selection bubble (`composer-format-bubble.tsx`, items in `(data)/composer-formats.ts`). `@` mentions a bot and `#` a workspace (`(data)/composer-mention-kinds.ts`, menu `composer-mention-list.tsx` above the input, items from `CrewBotsContext` and `CrewWorkspacesContext` via `searchMentionItems`), shown as chips (`composer-mention-chip.tsx`) and written as `[bot:ID]` / `[workspace:ID]` (`(utils)/mention-markdown.ts`, `src/utils/workspace-ref.ts`). Enter sends, Shift+Enter breaks the line; in a list or code block Enter is the editor's and Mod+Enter sends. Sending shows the bubble at once and empties the input (with its quote and files), so a slow connection never waits on the request: the optimistic message rides in `pending_messages` in `conversation.tsx` until the response or a push replaces it with the saved one (`(utils)/drop-sent-message.ts`), and a failed send takes it back and restores the text and files. Selecting text in a message (`data-quotable`, `use-message-selection`) shows a Quote button (`quote-selection-button.tsx`) that inserts it as a blockquote at the cursor (`ComposerHandle.insertQuote`). Before a reply, `grabMessageRefNames` names the refs in the user's messages that the user reaches, and `botMessagesToHarness` adds a `[Mentions: ...]` line after each one (`describeMessageRefs`). `@tiptap/react` needs `use-sync-external-store`, which is CommonJS and calls `require("react")`; bunext keeps React external in page bundles, so the app installs an ESM stand-in from `packages/use-sync-external-store-esm` (a `file:` dependency plus an override; the Dockerfile copies `packages/` before `bun install`).
- **Message rendering:** bot messages are markdown, rendered with `react-markdown` + `remark-gfm` (raw HTML never rendered) through the twui-based map in `[bot_id]/(data)/message-markdown-components.tsx` (`markdown-link`, `markdown-list`, `markdown-code-block` partials). A bot names a crew member as `[bot:ID]` (`src/utils/bot-ref.ts`, linked to `#bot-ID` by `link-computer-refs.ts`): `(partials)/bot-ref-chip.tsx` shows its avatar and name, linked to its conversation (in the open workspace when it's in the crew there), from the live crew (`CrewBotsContext`), so a bot created in that reply links too; the coordinator is told to write them (`CoordinatorReplyNote`), and `find_bot` and `create_bot` return them. A workspace is named as `[workspace:ID]` (`#workspace-ID`, `(partials)/workspace-ref-chip.tsx`, from `CrewWorkspacesContext`). User messages are markdown too, rendered in the ink bubble by `user-bubble.tsx` with `UserMarkdownComponents` and `remark-breaks` (a single line break stays one, so older plain messages read as typed).
- **Replies and reactions:** every bot message has a row under it (`(partials)/message-actions.tsx`, handlers from `BotMessageActionsContext`) to reply or react; the user's own messages only show the bot's reactions, read-only. Replying quotes the message above the composer (`composer-reply-preview.tsx`) and the sent one carries `bot_messages.reply_to_message_id` (not a foreign key: a deleted quote only drops it); `send-message` takes any message of the same conversation (the user's, same bot and workspace, any session, so a rollover or a moved-on thread still sends), and the conversation drops a picked quote when its session changes. Messages carry `reply_to` (`grabMessagesReplies`, also on relayed messages), shown by `message-reply-quote.tsx`, and the bot reads it as `[Replying to ...]` before the user's turn (`botMessagesToHarness`). Reactions are a fixed set (`BotMessageReactions` in `src/dict/bot-dict.ts`) in `bot_message_reactions` (one row per message, emoji and user, or bot); `api/admin/bots/react-message.ts` toggles the user's own on a bot message in their sessions (`grabUserBotMessage`, `toggleMessageReaction`), and every bot gets `react_to_message` (`src/harness/tools/bot/react-to-message-tool.ts`, chat replies and tasks: a message in the session it answers, the user's latest by default). Both push the message's fresh reactions (`grabMessagesReactions`, `mine` for the session user) to the session's user as `server:bot-reaction` (`BotMessageReactionData`), which replaces the message's own; `grabBotMessages` adds them when given `user_id`. Reactions aren't relayed between processes (production runs one).
- **Attachments and thinking:** the composer uploads files as they're picked, pasted or dropped (`use-composer-attachments`), shows them as chips, and sends once none is still uploading; the thread shows images as thumbnails and other files as chips. Replies read the files of the last `BotReplyAttachmentMessages` messages with any (`load-attachment-parts.ts`: images and audio as parts, documents as their text up to `BotAttachmentMaxTextChars`, videos as a note); older ones are only named. Tasks don't get the files yet. While the user has the last word in a session (every user message queues a reply, so this needs no face and shows even while the bot works in another session) or the bot waits for a model (`BotWaitingStatusLine`) or a task for its computer (`BotComputerBusyStatusLine`), `BotThinkingRow` sits under the last message; a message still on its way to the server reads "<name> is getting your message…" (`BotGettingMessageStatusLine`), while a task runs it reads "<name> is working…" with the task's latest step, and a reply working through tools shows its latest step under "is thinking…" (`use-reply-progress`). A coordinator with no workers gets `CoordinatorSoloNote` instead of `CoordinatorReplyNote`, so it never claims to have handed work to a worker. A stop button rides on that row (`stop-bot-button.tsx`, and the header menu's Stop row, both through `StopBotContext`) exactly while `botBusyState` (`src/pages/admin/bots/[bot_id]/(utils)/bot-busy-state.ts`, which also decides the row's line) says the bot has work open for that user anywhere; a message still on its way to the server isn't stoppable, since its reply is queued after it lands. Live notices come from the process that ran the job at once, and from every other process through the live relay.
- **Voice:** the composer's mic (`(partials)/composer-voice-button.tsx`, `use-voice-recorder`, shown once the browser can record, disabled with a note while transcription has no model to run on for the bot there, `bot_voice` page prop, from `resolveHarnessCandidates`) dictates: the recording goes to `api/admin/bots/transcribe.ts` (`transcribeAudio` with the bot's models, after `grabUserConversation`) and the text lands in the input at the cursor, to edit and send; Escape cancels, `AppData.ComposerVoiceMaxMs` stops it. Each bot message has a play button (disabled with a note while speech has no model) (`speak-message-button.tsx`, one player per conversation via `SpeakMessageContext` / `use-speak-message`) that plays `GET api/admin/bots/speak-message.ts?message_id=` straight in an `<audio>` (`speakText` streamed through as the provider makes it, so it starts in a couple of seconds; a failure answers in JSON, which the hook fetches again to show; only the user's own sessions, `grabSpeakableMessage`; text from `speechText`, cut at `AppData.SpeechMaxChars`). Under Speech, every model settings form has "Hear a sample" (`settings/models/(partials)/harness-voice-preview.tsx`), which plays `AppData.VoicePreviewText` in the form's model and voice before they're saved (`GET api/admin/harness/preview-voice`, super admins and admins, that model only via `grabModelCandidate`, streamed). Neither recordings nor audio are stored. Harness errors (`HarnessError`, safe to show) come back as the buttons' titles.
- **Pictures:** a bot or workspace can have a picture (`image_media_id`, a private `media` row, paradigm `bot-image`/`workspace-image`), set or removed through `api/admin/set-profile-image.ts` (`setProfileImage`) or, from chat, by the bots themselves through the `set_picture` tool (`src/harness/tools/bot/set-picture-tool.ts`, every run: a bot sets its own face, the coordinator any worker) and `update_workspace`'s `picture_file_id` (`setBotPicture`, `copyMediaAsPicture`; super admins and admins only, `assertCrewManager`). Both take the image by its `[file:ID]` ref (`grabPictureSource`, so a bot only sets what it could already read; chat uploads land in `uploads/...` and are labelled with their media id in `load-attachment-parts.ts`/`describeAttachment.ts`) or by an upload's media id. The bytes are **copied** into a new picture row, never moved: a chat message that attached the source keeps loading, and only a row whose paradigm is `bot-image`/`workspace-image` is deleted when the picture is replaced (`removeReplacedPicture`, `is-picture-media.ts`). A picture keeps its own format (`imageOutputFormat`, `src/utils/image-output-format.ts`: PNG stays a PNG with its transparency, GIF becomes PNG, everything else JPEG), for the full image and the thumbnail alike. Its change is pushed live (`pushBotUpdated`'s `image_media_id`, `pushWorkspaceUpdated` and `server:workspace-updated`, `relayChangedWorkspaces`), so `BotAvatar` (rounded square, else its stone, its colour still applies) and `WorkspaceAvatar` (else the workspace icon) swap it without a reload. A bot is also edited from the header avatar (`(partials)/bot-profile-modal.tsx`, the same `BotForm` as adding one, `api/admin/bots/update-bot.ts`; the coordinator keeps its name and stays hue-less).
- **Colour:** a bot's colour is free. It lives in `bots.color` (`#rrggbb`, set from the palette tiles, the native swatch or a typed hex in `(partials)/bot-colour-picker.tsx`, `api/admin/bots/`), and the stone it's drawn on is `bots.stone` (`BotStoneShapes`), because shape no longer follows colour. `src/utils/bot-color.ts` is the only place that resolves them (`resolveBotColor`/`resolveBotStone` fall back to the old `hue` for a bot saved before this, `botColorNeedsPaperFace` flips the eyes to paper on a very dark body), and `src/utils/bot-stone-props.ts` gives every row its `color`/`stone` props. A new bot starts on the palette's next free colour (`src/utils/pick-free-color.ts`). `src/scripts/move-bot-colour-to-owners.ts` backfills an install from `hue` (safe to rerun).
- **Avatars:** `BotStone` (`src/components/general/bot-stone.tsx`) renders the stone inline from `src/data/stone-geometry.ts`, the bot's colour as the body's SVG `fill` attribute, face (`idle|working|done|asleep|needs-you`) drawn as eyes. The coordinator is hue-less (ink body, paper eyes; inverts in dark). Never put a stone in a circle/badge or add outline/shadow. Wordmark is vector (`Wordmark`, `src/data/wordmark-paths.ts`), never live text.
- **Copy helpers:** `src/utils/crew-status.ts` (`3 working · 1 needs you`, "Three bots are working. One needs you."), `src/utils/relative-time.ts` (relative timestamps only).
- **Prompt and memory (self-learning):** a bot has `system_prompt` (core instructions) and a separate core `memory` (markdown, general knowledge about its job, the same for every user it works for, never about one user). What it remembers about a user lives in `bot_user_memories` (one row per bot and user, seen only by that user's runs; `grab-bot-user-memory`, `update-bot-user-memory`), like workspace memory stays in its workspace. The bot rewrites them as it adapts (`update-bot-system-prompt`, `update-bot-memory` in `src/functions/backend/db/bots/`; whole-text replace, capped at `AppData.PromptMaxLength`). `grab-bot-prompt` builds the full prompt for the run's user via `composeBotPrompt` (`src/utils/compose-bot-prompt.ts`): core instructions, then the install-wide `# Install-wide instructions` (`bot_system_prompt_append` in `app_settings`, set by a super admin on Settings > General, `grabSystemPromptAppend`; empty when unset), then (in a workspace only) `# Workspace: <name>` with its description and `system_prompt`, the bot's `# Your instructions in <name>` (`workspace_bots.system_prompt`), `# Memory`, `# Memory about the user`, then `# Memory in <name>` (`workspace_bots.memory`). `update_memory` saves to `user` by default (`workspace` in a workspace) and to `core` only for what holds for everyone. In a workspace the bot learns into the workspace's instructions and memory by default (`scope: workspace` on `update_instructions`/`update_memory`, `WorkspaceLearningNote`) and into its core ones only when told to, or after suggesting it. `src/scripts/move-bot-memory-to-owners.ts` (run once per install) moved core memory written before this into the owner's.
- **Queue:** every bot job goes through `bots_queue`, with `kind` (`BotJobKinds`): `reply` (answers the user in chat, no task row) or `task` (long-running work, typically on a computer). Long-running work goes through `bots_queue`, a bun-sqlite `isQueue` table worked with the native `BunSQLite.queue` API (`status` pending|processing|completed|failed, `attempts`, `available_at`, `locked_at`/`locked_by`, `last_error`; options in `AppData.BotQueue*`). Not every message needs a job: the bot replies right away that it's on it and `enqueueBotJob` queues the brief (`instructions`, `params`, optional `message_id`/`computer_id`, the computer checked against the bot's access; a `bot_tasks` row is created unless `task_id` is given). DB helpers in `src/functions/backend/db/bots-queue/`: `claimBotJobs`, `updateBotJob` (renews the lock, saves `progress`/`checkpoint`; false once lost or cancelled), `completeBotJob` (result message, task done, face done/needs-you for the job's user), `failBotJob` (backoff retry, or `fatal`; out of attempts posts a needs-you message via `notifyBotJobFailed`), `cancelBotJob` (failed with `BotJobCancelledError`, `src/dict/bot-queue-dict.ts`), `stop-bot-jobs` (a user pressing Stop: every pending or processing job of that bot in their conversations, `open-bot-jobs-sql`, failed with `BotJobStoppedError`, its task closed, a `Stopped` step recorded and one closing message per session, `stopped-job-report`; the running handler's controller is aborted from this process through `job-abort-registry`, so a job waiting on a model stops at once instead of on its next heartbeat, and a late `setFace` after the abort is dropped so a stopped bot is never pushed back to `working`; `api/admin/bots/stop.ts` then pushes the messages and the idle face, `quiet_push` so the user who pressed it gets no push notification), `releaseWorkerBotJobs`. Every model turn of a job is a step in `bots_queue_updates` (`kind` tool|note, one-line `text`, JSON `metadata`; tool names, never arguments; `describe-job-step.ts`), written through the job context's `report` (the text loop's `onStep`), which also sets `progress` and pushes `server:bot-job-update` (`BotJobUpdateData`). Steps are deleted with their job when it's pruned. One attempt has a deadline (`AppData.BotJobAttemptTimeoutMs`): the heartbeat keeps the lock alive for as long as the run takes, so a run past the deadline is aborted, reported as a failed try and retried, and a model or tool that never answers can't hold its slot, conversation or computer forever. The active task's latest `AppData.BotJobUpdatesShown` steps ride on `BotTaskWithSteps.updates` (`grabBotJobUpdates`); the conversation (`bot_task` page prop, `use-task-progress`) shows the latest one under `BotThinkingRow`'s "<name> is working…" while a task runs. Step text is the first line the model wrote, else plain labels for its tools (`JobStepToolLabels` in `src/dict/bot-queue-dict.ts`; other tools in words, an MCP tool as "Using Exa search (web search exa)").
- **Queue handler:** `src/bots-queue/index.ts`, started from `server.ts` after cron and gated the same way. It releases its own cut-off jobs on startup (stable worker `AppData.BotQueueWorker`; polling starts even if that release fails, and a production process without `MAIN_SERVER` says so in the log instead of going quiet), then polls every `BotQueuePollMs`: reports jobs bun-sqlite failed on a lock timeout (`report-lost-bot-jobs`), claims per kind (`claimBotJobs` mirrors bun-sqlite's atomic claim plus `kind`, since `queue.claim` has no filter) up to `BotQueueReplyConcurrency` replies and `BotQueueConcurrency` tasks on separate slots, and runs each with `processBotJob` (heartbeat at a quarter of the lock timeout, abort on cancel/loss, one task per computer and one reply per session via `createKeyLock`, so a bot's other sessions answer side by side; a task waiting for its busy computer shows face `asleep` with `BotComputerBusyStatusLine`; face `working` once it starts, outcome pushed live; a job finishing while others for that user still run puts back the running one's face, `grabRunningBotJobFace`). The actual work is `run-bot-job.ts` (`BotJobRunner`, context in `types.ts`), which dispatches to `run-bot-reply.ts` and `run-bot-task.ts`. Throw `BotJobFatalError` for failures retrying can't fix. When every model is resting (`HarnessError.resting`, nothing was tried) the job is put back with `deferBotJob` until the first comes back, its attempt refunded and the bot's face `asleep`; other retryable harness errors retry no sooner than their `retry_at`. The `prune-bots-queue` cron deletes finished jobs after `BotQueueRetentionDays`.
- **Live updates:** server push only. `sendToUser` (`src/websocket/(functions)/send-to-user.ts`) sends to one user's sockets; `pushBotNotice` sends `server:bot-message` (the message row, user messages too, for other tabs) and `server:bot-updated` (`BotUpdatedData`: face, status line, active task). The bots layout opens one socket (twui `useWebSocket`); components listen with `useServerEvent` (`src/hooks/`), and `useLiveBots` keeps the rail and header current. The conversation dedupes messages by id. Every process also runs `liveRelay` (`src/live-relay/`): every `AppData.LiveRelayPollMs` it reads new `bot_messages`, new `bots_queue_updates`, changed faces and changed bots from the database and pushes them to its own sockets, so work done by another process still shows live; clients drop what they already have. `server:bot-updated` carries the bot's `computer_id` when known, which keeps the header's computer in step across tabs, and its `workspace_ids` after a membership change, so a workspace's crew drops a bot that left it. A bot that joins the crew or a workspace is pushed as `server:bot-created` (`BotCreatedData`: its rail row, the workspaces it's in, and whether it belongs in that user's own chat), from the tool that created it and from the live relay for other processes, so the rail lists it without a reload. A workspace a bot created is pushed the same way as `server:workspace-created` (`WorkspaceCreatedData`: its rail row), from the tool and from the live relay. A deleted bot is pushed as `server:bot-removed` (`BotRemovedData`) to the audience read before the delete, since its assignments go with it, so its rail row goes at once. A deleted workspace is pushed the same way as `server:workspace-removed` (`WorkspaceRemovedData`), read before the delete too. A rail row's subtitle is the bot's status line while it has one, else (coordinator) the crew counts while a worker is busy or needs you, else its last message as plain text (`src/utils/message-preview.ts`).
- **Workspaces:** `workspaces` (`name`, `description`, `system_prompt` shared by every bot in it, `computer_id` its default computer (not a foreign key), `archived`), with bots assigned through `workspace_bots` (plus that bot's `system_prompt` and `memory` there; the coordinator is an implicit member) and computers through `workspace_computers`. The first computer linked becomes the default; the user (workspace settings, chat header), any bot there (`set_workspace_computer`) or the coordinator can switch it. A workspace with no computer has none: tasks needing one fail. Workspace models (`workspace_harness_settings`) win over a bot's own, then the defaults. Helpers in `src/functions/backend/db/workspaces/` (`create-workspace`, `update-workspace`, `delete-workspace`, `add/remove-bot-*`, `update-workspace-bot`, `add/remove-computer-*`, `set-workspace-computer`, `grab-user-workspaces`, `grab-workspace-bots`, `grab-workspace-computers`, `grab-workspace-activity`), all access-checked against the session user, the structural ones for super admins and admins only; API in `api/admin/workspaces/`. The coordinator sets a member's instructions there with `set_bot_workspace_instructions`, creates a workspace with `create_workspace`, and changes any workspace's name, description or context with `update_workspace` (chat replies, and its own tasks so findings it was asked to record land in the workspace; a chat-created one is pushed as `server:workspace-created`).
- **Computers:** a computer is an AIO sandbox in its own docker container on the `AppData.DockerNetworkName` network. `containers` records every docker container the app knows about except the main app container (`name` doubles as the hostname, plus `docker_id`, `image`, `network`, `port`, `status`, `metadata`). `computers` (the install's, `kind` main|sandbox, `container_id`) and `bot_computers` (direct access). The app creates and manages every computer's container over the docker socket (`src/functions/backend/docker/`; `DOCKER_SOCKET` or `AppData.DockerSocketPath`): `createComputer` records it and `provisionComputer` (in the background, status `creating`, then the container's or `failed` with `metadata.error`, pushed as `server:computer-updated`) pulls `AppData.ComputerImage`, binds the home dir and starts `<ComputerContainerPrefix>-<id>` limited to the computer's `cpus` and `memory_mb` (set at creation, default `AppData.ComputerDefaultCpus`/`ComputerDefaultMemoryMb`: 1 CPU, 2 GB), attached to the network in the same create call, with a docker health check (`AppData.ComputerHealthcheck`: the sandbox API on `ComputerPort` and its browser); the computer stays `creating` until it passes (`waitForHealthyContainer`), then records its IP address on the network (`containers.ip_address`, refreshed by `sync-computers`). `updateComputer` (`api/admin/computers/update-computer`) saves name, description and resources; new resources stop, remove and recreate the container (`provisionComputer`), the home dir kept. A computer's home (`/home/gem`) is `DATA_DIR/computers/<id>` (`AppData.ComputerDataDir`, owned by uid 1000); deleting the computer (`deleteComputer`, never the main one) removes the container and keeps that directory. Run in docker, the app resolves host paths from its own container's mounts and joins the network; outside docker (dev) it can't resolve container hostnames, so it reaches them at their IP address (`pickReachableAddress`: hostname first, then the IP, the pick cached for `AppData.ComputerAddressCacheMs`). Only super admins create, delete or retry computers (`api/admin/computers/`). The `sync-computers` cron records container states every five minutes. The install has one main computer, created at signup (`bun run seed`) and as a fallback by the super admin's `/admin/bots` via `ensureMainComputer` (`src/functions/backend/db/computers/ensure-main-computer.ts`, idempotent), which grants the coordinator access; with `MAIN_COMPUTER_URL` set (dev) it isn't provisioned. That one grant can't be removed (`isLockedComputerAccess`, enforced in `remove-bot-from-computer` and the `bot_computers` CRUD handler); every other grant can. A bot's effective computers (`grab-bot-computers`): the workspace's computers in a workspace, its default first; in its own chat, its direct grants and those of the workspaces it's attached to (the coordinator's are every computer), the one it's on first: `bots.computer_id` (not a foreign key; null or a lost grant means its main one), so `computers[0]` is where its tasks run by default. The user switches it from the chat header (`(partials)/bot-computers-modal.tsx`, a ghost icon beside the model icon; the header names the current computer and the text model and provider under the bot's name) or by clicking a computer the bot named in a message: bots write `[computer:ID]` (`src/utils/computer-ref.ts`, turned into `#computer-ID` links by `link-computer-refs.ts`, rendered by `(partials)/computer-ref-chip.tsx`), both sharing `BotComputersContext` / `use-bot-computers`, saved by `api/admin/bots/set-computer.ts` (`setBotComputer`). Enums in `src/dict/computer-dict.ts`.
- **Opening a computer:** the app serves each computer to whoever sees it (`computersSql`) under `/computers/<id>/` (`ComputerProxyPrefix`): `src/server.ts` hands those paths to `handleComputerProxy` (`src/functions/server/computers/`: `userAuth`, access via `grabUserComputer`, then the sandbox's address from `grabComputerMcpUrl`), which fetches plain requests (app session cookies dropped, `X-Forwarded-Host/Proto/Prefix` set so the sandbox builds its links and CDP address under the prefix, framing headers and redirects fixed) and bridges websockets (`computerProxySocket`, dispatched from `src/websocket/` when `ws.data.proxy` is set). `ComputerView` (`src/components/general/computer-view.tsx`) shows a computer in one of `ComputerViewModes` (browser `/browser-ui`, terminal `/terminal`, code `/code-server/`, desktop: the sandbox's noVNC at `/vnc/vnc.html`, its socket pointed back through the proxy by the `{prefix}` placeholder that `computerViewUrl` fills in; desktop is the only view with Chrome's toolbar (extensions) and a clipboard panel, since the browser view streams pages over CDP and hides `chrome-extension://` targets; more of the sandbox's apps can be added there), keeping visited modes loaded. In the chat, `(partials)/computer-viewer-modal.tsx` shows it, opened from a computer card's open button in the computers modal or a take-over button (`openComputer` on `BotComputersContext`).
- **Take-over:** a bot asks the user to take over a computer with `ask_user_to_take_over` (`src/harness/tools/computer/ask-to-take-over-tool.ts`, chat replies and tasks, defaulting to the task's computer). It writes `[take-over:ID:mode]` (`src/utils/take-over-ref.ts`; `withTakeOverRefs` adds it if the bot left it out and flags the message needs-you), linked by `link-computer-refs.ts` and rendered by `(partials)/take-over-chip.tsx`. The viewer then offers "Hand back to <bot>", which sends a chat message so the bot carries on; a task stops and reports instead of waiting.
- **Health:** `containers.status` is docker's state with `running` narrowed by the health check (`starting`, `unhealthy`; `inspectDockerContainer`). Only a `running` computer is reached: `grabComputerMcpUrl` returns `ready: false` otherwise (re-reading a stale state from docker first, `refreshComputerContainer`, which `sync-computers` uses too), and callers say why (`computerNotReadyMessage`).
- **Schedules (routines in the UI):** `cron_jobs` (per user and bot, `workspace_id` null for its own chat, `title`, `instructions` (the brief), `schedule` (a `Bun.cron` expression, UTC), `computer_id` (not a foreign key; null runs on the bot's current one), `enabled`, `max_runs` (1 for a one-off), `run_count`, `next_run_at`, `last_run_at`, `last_job_id`, `last_error`). Helpers in `src/functions/backend/db/cron-jobs/` (`create-cron-job`, `update-cron-job`, `delete-cron-job`, `grab-cron-jobs`, `grab-user-cron-job`, `fire-cron-job`), all ownership-checked, and with `bot_id` limited to that bot's own. `nextCronRun` (`src/utils/next-cron-run.ts`, `Bun.cron.parse`) validates a schedule and refuses one that runs more often than `AppData.CronJobMinIntervalMs`; a bot has at most `AppData.CronJobMaxPerBot`. Every bot gets `create_schedule`, `list_schedules`, `update_schedule`, `delete_schedule` (`grabBotTools`, every run; descriptions via `describe-cron-job.ts`). The `run-cron-jobs` cron fires due ones (`fireCronJob`: claims the run atomically by moving `next_run_at`, skips it while the last run's job is still open, queues the brief as a task with `enqueueBotJob` in the bot's active session there, turns off after `max_runs` or when there's no next run, keeps why it couldn't in `last_error`, logs each run under `cron`). A run missed while the server was down runs once, then carries on from now. A run acts on its own (`ScheduledRunNote`: the brief is already approved, so it sends rather than drafts); a run that can't do its brief calls `report_run_failed` (scheduled runs only), which throws `ScheduledRunFailedError` so the queue retries it, `AppData.ScheduledRunRetryMs` apart and doubling (`scheduledRunRetryAt`); once out of attempts the user gets one "failed after N tries" message and the schedule's `last_error` keeps why (`recordCronJobFailure`). `/admin/routines` lists every one of the user's (pause/resume, edit, delete; `api/admin/routines/update-routine`, `delete-routine`); bots create them from chat, and each conversation's own `(partials)/bot-routines-modal.tsx` ("Schedules" in the header menu, beside Tasks) lists the ones that run there (a bot's own chat or one workspace, `routineScopeSql` in `src/utils/routine-scope/`) through `api/admin/bots/list-routines`, which owns them through `grabUserConversation` and `grabCronJobs`'s `workspace_id` filter and loads only when the modal opens. It shares `RoutineForm`, `RoutineRow` and `RoutineEditModal` with the routines page (both in `src/components/general/`).
- **Tasks (the crew's to-do list):** `tasks` is the tracker, one collection per bot (`bot_id`) and one per workspace (`workspace_id`; exactly one is set, enforced in `taskScopeValues`, since SQLite treats NULL in a unique constraint as distinct - the `envs` precedent). It is not `bot_tasks` or `bots_queue`: those record a queued job, this records the work. Fields: `title`, `description` (the brief, markdown), `status` (`TaskStatuses` in `src/dict/task-dict.ts`: `todo|doing|blocked|done|cancelled`), `blocked_reason` (only while blocked), `origin` (`user|bot`), `created_by_bot_id`/`created_by_user_id` (not foreign keys), `completed_at`. `task_notes` is the progress log (author `user|bot`, `bot_id`, `text`); `task_media` links a file to a task as a `deliverable` or a `reference` (`task_id`, `media_id` FK, `bot_file_id` not a FK, `kind`, `name`, `added_by_bot_id`), the bytes being a private `media` row (paradigm `task-media`) served at `/media/<id>` through `grabTaskMediaFile` (registered in `handle-media-server.ts` beside `grabBotFileMedia`). Visibility is SQL in `src/utils/task-access/` (`taskScopesSql` for a user: `chatBotsSql` bots plus `workspacesSql` workspaces; `botTaskScopesSql` for a run: its own plus every workspace it works in, everything for the coordinator; tested against SQLite). A workspace's tasks are visible to every bot working there and to the coordinator. Helpers in `src/functions/backend/db/tasks/`: `create-task` (`assertCrewManager` for a manual add, `TaskMaxPerScope` cap), `grab-tasks` (filters, joins the bot's face, the workspace and the latest note), `grab-task`, `update-task` (stamps `completed_at`, clears `blocked_reason` when it leaves blocked), `delete-task`, `add-task-note`, `attach-task-file`/`detach-task-file` (a bot file resolved through `grabBotFile` so a run can't link outside its folders; a chat upload must be the user's own media; `TaskMediaMaxPerTask`), `grab-task-media`, `grab-task-media-file`, `find-duplicate-task`, `purge-task-media` (`grabTaskMediaIds`, read before `deleteBot`/`deleteWorkspace` so a task's own chat uploads aren't stranded; a bot file keeps its folder). Every run gets `list_tasks`, `create_task`, `update_task` (`src/harness/tools/bot/tasks/`, from `grabBotTools`): a worker may only add to its own list or the workspace it works in, the coordinator to any worker's, any workspace's, or its own, and `update_task` takes `status`, `blocked_reason`, a `note` and `deliverable_file_ids`/`reference_file_ids` (`[file:ID]` refs) in one call. `TasksNote` (every reply and task) says how to keep the list honest; `TaskReminderNote` (chat only) tells the bot to check `list_tasks` on a greeting or when proposing next steps and mention at most `AppData.TaskReminderDigestCount` open tasks in one line, blocked first, never repeating a reminder and staying quiet mid-task. The user adds and changes tasks by hand in a workspace's dashboard (`(partials)/workspace-tasks-group.tsx`) and from the conversation header's menu (`(partials)/bot-tasks-modal.tsx`, loading its list when it opens so the page props stay small); both compose the shared `TaskForm`, `TaskListItem`, `TaskDetailPanel`, `TaskStatusChip` and `TaskFileRow` in `src/components/general/`. API in `api/admin/tasks/` (`list-tasks`, `get-task`, `save-task`, `delete-task`, `add-note`, `attach-file`, `detach-file`), each `userAuth` → `isReadOnlyUser` → `assertUserReachesTask` → `safeErrorMessage`. Live pushes for task changes are not built yet.
- **Not built yet:** task steps (nothing writes `bot_task_steps` yet), live `server:task-changed` pushes, and a `/admin/tasks` page.

### Installer and release

`install.sh` is the only supported install and update path: `curl -fsSL https://url.moduletrace.com/cobble | sudo bash`. It is idempotent, so the same script is the updater, and `--uninstall [--purge]` is the way back out. It installs Docker when missing (Linux), creates the `cobble` network, writes `$COBBLE_HOME/install.conf` and a generated `.env` (mode 600), and starts one `cobble` container. Existing secrets are never regenerated; the database is backed up to `$COBBLE_HOME/backups` before a container is replaced. The default home is `/opt/cobble`, configurable with `--home`, and everything that must survive an update lives in its `data/` directory (`db`, `media`, `computers`, `bots`, `workspaces`, `mcp`).

The bind mounts are built in the marked `selinux-detection` and `mount-specs` blocks, and `install.test.ts` runs those blocks in bash and parses the specs the way docker does. A mount spec is `source:destination[:options]`: docker takes the SELinux `z` and the read-only `ro` as comma separated options in that third field, so a destination is always a plain absolute path. Appending an option to the destination (`$DATA_DIR:/data,z`) makes docker mount a directory literally named `/data,z`: the app then writes its database to its own container layer, `container_matches` never matches, every update recreates the container without its data, `backup_database` silently skips, and `resolveHostPath` refuses to provision computers (`/data/computers/1 isn't on a volume`). `rescue_container_data` runs before a container is replaced and copies a container's unmounted `/data` onto the host, so repairing an install from the broken form does not lose its database.

The app serves plain HTTP only; TLS is the reverse proxy's or tunnel's job, and the installer, the image's `HEALTHCHECK` and the dev server all assume it (`Bun.serve` gets no `tls`, there is no certificate handling anywhere). `isSecureRequest` still reads the proxy's `X-Forwarded-Proto`, so the auth and CSRF cookies get `Secure` and the installed app, Web Push and voice keep their secure context behind HTTPS; the installer binds `0.0.0.0` by default, and `--bind 127.0.0.1` limits it to a local proxy. Updates drop any `TLS_*` or `HTTP_REDIRECT_PORT` entries an older install left in `.env` and recreate a container that still mounts `/app/certs`.

The container runs with `--network cobble`, `-p BIND:PORT`, `.env` at `/app/.env` read only, the host data directory at `/data`, the docker socket at `/var/run/docker.sock`, `MAIN_SERVER=1` and `PORT`, `--init`, `--cpus 1` and `--memory 2048m` (a light orchestration process: the heavy work happens in each computer's own container; overridable with `--cpus`/`--memory`), a `unless-stopped` restart policy and bounded json-file logs. `/data` must stay a bind mount: `resolveHostPath` turns paths under the app's own mounts into host paths for computer containers, and a named volume would break them. `DATA_DIR=/data` and `DB_DIR=/data/db` come from the Dockerfile; `entrypoint.sh` keeps bun-sqlite's schema record on the same volume.

Publishing is `scripts/publish-github.sh` and `scripts/publish-gitea.sh`: the shared flow (argument parsing, the release gate, tag preparation, the push, the release notes) lives in `scripts/lib/publish-common.sh` and `scripts/lib/publish-release.sh`, and only the release object differs (`gh release create` for GitHub, which is what fires the image build, and `curl` against the Gitea API with a `GITEA_TOKEN` or the token `tea login` stored). Both are marked-block scripts tested by `scripts/publish-tools.test.ts`, which lifts the blocks out with the `install.test.ts` approach. Images are built by GitHub Actions on a published release (`.github/workflows/release.yml`), for `linux/amd64` and `linux/arm64`, and pushed to `ghcr.io/moduletrace/cobble` with `latest`, version, `major.minor` and commit-sha tags. CI never touches the Gitea registry: it is on a VPN, which a runner cannot reach, so `scripts/mirror-gitea.sh` (`bun run mirror:gitea`) copies a release into `git.tben.me/moduletrace/cobble` afterwards, from the Gitea host over ssh, running skopeo there in a container and reading every destination tag's digest back against GHCR's; it only moves `latest` and `major.minor` while that release still owns them in GHCR, so mirroring an older one never rolls them back. The two run in parallel and neither can fail the other, which is the point: the installer defaults to GHCR, so a registry outage cannot fail a release. The version they are built with is passed as the `COBBLE_VERSION` build arg (the published tag, else `package.json`'s version read from `<version>`), which the image turns into `ENV COBBLE_VERSION` and `runningVersion` (`src/utils/running-version.ts`) reports on Settings > General; bump `package.json`'s `version` with a release, since a local build and a workflow-dispatch build fall back to it. `.github/workflows/ci.yml` runs `bun run typecheck` and `bun run test` on every push and pull request, and checks that `install.sh` parses. The Dockerfile pins Bun 1.4.3 to match CI. The repository is mirrored on Gitea, and the installer defaults to the GHCR image with `--image git.tben.me/moduletrace/cobble` as the Gitea alternative.

### Envs

Envs are set in Settings, under Environment (super admin only), not in `.env`. Table `envs` (private): `key`, encrypted `value`, `description`, `paradigm` (`EnvParadigms`: general, workspace, computer, bot) with `workspace_id`/`computer_id`/`bot_id`, `secret`, `expose_to_computers`. One per key and scope, enforced in `saveEnv` (`src/functions/backend/db/envs/`), since SQLite unique constraints treat NULL ids as distinct. Decrypted rows are cached per process (`grabAllEnvs`; `saveEnv`/`deleteEnv` clear it). A bot's or a workspace's own envs are also edited on its settings page (`settings/(partials)/envs/scoped-envs-group.tsx`, fed by `grabScopedEnvs`, `bot_envs` / `workspace_envs` page props), super admins only, which is the same table and the same `save-env`/`delete-env` routes (`scoped: true` makes them answer with just that scope, `EnvsMutationResult`, so the page updates in place).

- **App settings** (`EnvDict` in `src/dict/env-dict.ts`: `MAIN_COMPUTER_URL`, the `VAPID_*` Web Push keys and every provider preset's `key_env`) are what the app reads, through `grabEnv({ key })`: the general value, else `process.env`, so an existing `.env` still works. General only, never on computers. `ProcessOnlyEnvKeys` (`ENCRYPTION_KEY`/`SALT`, `PORT`, `DATA_DIR`, `DB_DIR`, `MAIN_SERVER`) can't be saved: they encrypt the table or are the container's own start-up settings.
- **Bot and computer envs** are any other key. `grabComputerEnvs` resolves one run's set (`resolveScopedEnvs`): the bot's own beat the computer's, which beat the workspace's, which beat general; workspaces are the non-archived ones the computer shares with the run's bot, oldest winning a tie. Every computer call site passes the run's `bot_id` (`openComputerTools`, `searchComputersTool`, `runScriptTool`, through `grabBotChatTools` / `run-bot-task`), and `grabRunEnvs` (`{{env.KEY}}` in an MCP server's or custom tool's templates) layers the bot's own over the workspace's over general. `openComputerTools` writes them to `~/.cobble/env` and `env.json` on the computer (owner only, rewritten every run) over the sandbox's REST API (`writeComputerEnv`); the sandbox's shells read no profile, so `withComputerEnv` makes `sandbox_execute_bash` source the file and `sandbox_execute_code` (Python, JavaScript) load the JSON, invisibly to the model. Bots get keys and descriptions in the task prompt, never values, and every computer tool result is passed through `redactSecrets` (each secret value shown as `[KEY]`).
- **API:** `api/admin/envs/` (`save-env`, `delete-env`), super admin only, logged by key, each returning `grabEnvsOverview` (secret values left out; a secret is only replaced, never read back), or `grabScopedEnvs` when the request came from a bot's or workspace's settings page.

### PWA and notifications

- **Generated assets:** `public/manifest.webmanifest`, `public/icons/*.png`, `public/offline.html` and `public/loading.html` are generated from `public/logos/logo.svg` and the theme tokens by `bun run gen:favicon` (`src/scripts/generate-brand-assets.ts` reads `src/styles/main.css`, so the manifest colors and the offline page never drift from the theme). The outputs are committed like the favicons. `src/data/pwa-data.ts` holds the shared paths, icon list, start path (`/admin/bots`) and shortcuts. The apple touch icon is opaque on paper (a transparent one renders black on iOS).
- **Head and install:** `src/pages/__root.tsx` `Head` links the manifest, sets the apple/mobile meta tags and renders `PwaInitScript`, which captures `beforeinstallprompt` before hydration (twui's hook reads it later), points `theme-color` at `--c-paper` (following the theme without repeating the color), and rewrites bunext's fixed viewport meta to add `viewport-fit=cover` and `interactive-widget=resizes-content`. `use-install-prompt` exposes installable/standalone/iOS state; the rail user menu shows Install app while installable, hides once installed, and opens `install-app-modal` with the Safari Share steps on iOS.
- **Service worker:** `public/sw.js`, served by `src/server.ts` through `handlePwaFile` with `no-store` and `Service-Worker-Allowed: /` (bunext's one-hour public cache would delay every deploy, including through Cloudflare). Navigations are network-only with `public/offline.html` as the fallback; authenticated HTML is never cached. `/public/`, icons and logos are stale-while-revalidate; `/api/`, `/ws`, `/media/`, `/computers/` and `/build-info.json` are never intercepted. `ServiceWorkerRegistrar` in `__root.tsx` registers it in production only (development HMR is never cached) and offers a reload when a new version waits; it never reloads on its own, so a running reply isn't cut off. In development it does the opposite (`clear-development-service-worker.ts`): it unregisters every worker and deletes every cache a production deploy left on the device, then reloads once into the uncontrolled tab, because the installed worker keeps serving its build's `/.bunext/public` files cache first while every cache-busting mechanism (the precache gate, `/build-info.json`, the loading page) is production only. Development also answers `/sw.js` with a stub that unregisters itself and drops the caches, so any other registration path (Web Push, the loading page) ends the same way.
- **Precache (production):** the whole of `.bunext/public` is downloaded once per build before the app is used. On start the server reads the build id from `.bunext/build-info.json` and lists every file in `.bunext/public` (`grabPrecacheManifest`, logged, read once). `handlePrecacheGate` (in `src/server.ts`, before bunext) answers a page navigation whose `AppData.BuildCookieName` cookie isn't that build with the generated `public/loading.html` at the same address (`no-store`, `{{build}}` filled in; `needsPrecachePage`: `Sec-Fetch-Mode: navigate` or `Sec-Fetch-Dest: document`, since a navigation the worker passes on arrives as `empty`, never `/api/`, `/.bunext/`, `/media`, `/computers/`, `/ws` or a path with an extension). The page reads the build's files from `GET /build-info.json` (`handle-build-info.ts`, `no-store`) and downloads them into `cobble-assets-<build_id>` itself (`AppData.PrecacheConcurrency` at a time), so it works whichever worker version is in charge. It then checks every file is in the cache, downloading any missing ones again up to `AppData.PrecacheVerifyPasses` more times, before it writes the marker (`PwaPrecacheMarkerPath`). It deletes the old build's cache, tells a new worker waiting behind an older one to take over (`SKIP_WAITING`, after `reg.update()`, for up to `AppData.PrecacheWorkerWaitMs`), sets the cookie and reloads into the app. Without Cache Storage, when files are still missing after the last pass, or once nothing has landed for `AppData.PrecacheStallMs`, it sets the cookie anyway, so the app runs from the network. The worker only serves `/.bunext/public/*` cache-first from the one `cobble-assets-*` cache. `sw.js` never changes per build, and React knows nothing of any of this. The loading page is generated with the offline page (`bun run gen:favicon`).
- **Phone shell:** below `md` the rail is a 280px slide-over with a scrim, Escape to close, and a close on every route change (`RailDrawerContext`); it is a 64px icon rail at `md` and 260px at `lg`. Rail internals use the `@container` `@rail:` variant (`--container-rail` in `src/styles/main.css`), not viewport `lg:`, so the drawer shows the full rail. `RailDrawerButton` sits in both pane headers. The shell is `h-dvh` with safe-area insets, and the headers/thread/composer use mobile padding.
- **Connection recovery:** twui's `useWebSocket` stops retrying after three attempts, so `LiveSocket` is remounted on `online`, `pageshow` and a hidden tab becoming visible (`use-connection-epoch`). `useOnlineStatus` drives the offline strip and disables sending: nothing is queued offline.
- **Web Push:** `push_subscriptions` (one row per endpoint and user; the endpoint is unique and a capability URL, never sent to the client) and `user_presence` (one row per user, refreshed by the socket keepalive in `src/websocket/index.ts`). VAPID keys are app settings in the `envs` table (`VAPID_PUBLIC_KEY` not secret, `VAPID_PRIVATE_KEY` secret), generated by `ensureVapidKeys` on the main server's first start; the subject is `SiteData.SiteURL`. `pushBotNotice` is the one trigger: bot messages (never the user's) and a needs-you face change without a message. It is skipped while `user_presence.last_seen_at` is within `AppData.PushPresenceWindowMs`, so an app open anywhere wins. `src/dict/push-dict.ts` owns payload limits and tags (one conversation replaces its own notification); sending never blocks or fails the job, 404/410 removes the device, and other failures are recorded on the subscription. API in `api/admin/push/` (subscribe, unsubscribe, test, remove-device, overview); Settings > Notifications (`notifications/`) has the per-device switch, a test send, and every device this user has. Permission is only asked from the switch (a denial is hard to undo, and iOS requires the gesture); iOS also needs the installed app. Development has no service worker, so the page says notifications need the installed production app.
- **Risks and notes:** push needs HTTPS (or localhost), so it depends on the proxy or tunnel in front. Cloudflare never caches `/sw.js` (no-store). The viewport rewrite is a workaround for bunext's fixed meta and stays defensive (no-op if the tag is gone); iOS Safari ignores `interactive-widget`, so its on-screen keyboard overlays the composer rather than resizing the page. Notification bodies are short message previews, so they can appear on a lock screen; there is no per-device mute beyond turning notifications off. Presence is per user, not per device: an open desktop suppresses a phone until it has been closed for `PushPresenceWindowMs` (deliberate). Notifications are best effort: no retry queue, and a failed send is logged and recorded on the subscription. Live events missed while disconnected are not replayed, except the conversation's reply catch-up for a pending reply; opening the conversation refetches. The drawer is not a focus trap.

### Bot files and folders

Bots keep files on the app's data volume, never on a computer: `DATA_DIR/bots/<bot_id>` for a bot's own, `DATA_DIR/workspaces/<workspace_id>` shared by the bots working there. The catalog is `bot_files` (scope `bot`|`workspace`, `owner_id`, `path`, `size`, `purpose`, `source`, `ingested_at`, `ingest_error`), its bytes are a private `media` row (paradigm `bot-file`) written by `recordBotFile` (`src/functions/backend/db/bot-files/record-bot-file.ts`), so they're served at `/media/<media id>` through `grabBotFileMedia` and deleted with the row.

Who reaches what: `grabStorageScopes` (`src/utils/storage-access.ts`) gives a worker its own folder plus the workspace's while it works there, and the coordinator every bot's and every workspace's. Enforced in the tools, never left to the prompt. `resolveBotPath` (`src/utils/resolve-bot-path.ts`) keeps every path inside its root (no absolute paths, `..`, NUL, symlink escapes) and the quota (`app_settings` key `bot_files_quota_mb`, set on Settings > General, `grabBotFileQuotaBytes`) is checked by `assertStorageRoom` before a write.

Tools in `src/harness/tools/storage/` (`grabBotFileTools`, every run: chat, tasks, routines): `list_files`, `read_file`, `write_file`, `search_files`, `move_file`, `delete_file`, `ingest_file`, `copy_file` (only when the run reaches more than one folder), `generate_image` and `make_pdf`. A file written with `attach` rides on the reply: the outcome's `attachments` become `bot_message_media` rows on the result (`completeBotJob`). Bots name files as `[file:<id>]` (`src/utils/file-ref.ts`, linked by `linkComputerRefs`, rendered by `file-ref-chip.tsx`). A chip looks the file up in the conversation's page props (`grabConversationFiles`, which must list the **whole** folder tree, not just the top level, or every file saved under `reports/` or `seo/...` renders as dead plain text) and fetches it by id (`api/admin/bots/files?file_id=`) when it is older than the page's first page. Chat uploads are copied into the conversation's folder (`storeChatAttachments`) so a later run can search them.

A task on a computer also gets `import_file` (`src/harness/tools/computer/import-file-tool.ts`): it copies a file or a folder off the computer into the run's files (a folder as a gzipped tar, unpacked into one `bot_files` row per file, source `tool`), ingests it at once, and answers with the `[file:ID]` refs plus the original's path. The bytes come back through `downloadComputerFile` (`download-computer-file.ts`), the sandbox's streaming `GET /v1/file/download` - **never through its shell**: `/v1/shell/exec` caps output at 30,047 characters, so `base64 -w0` silently cut every import at 22,528 bytes and a 1.12 MiB archive was saved as a 22 KB file that would not open, with the tool reporting success. The size is read first (nothing over `AppData.BotFileImportMaxBytes` is transferred), the download asks for `change_policy=abort` (409 if the file changes while it is sent) and the length is checked against the size read, so a cut-short transfer is reported instead of saved. `BotFilesNote` and `ComputerFilesNote` (`src/data/harness-prompts.ts`) tell the bot to save the deliverable before the long tail of a run, so a step limit can't cost the user the report.

Anything a bot leaves on a computer is not in `bot_files`, but the user can still open it: bots name a folder as `[folder:<volume>:<owner>:<path>]` (`src/utils/folder-ref.ts`, e.g. `[folder:computer:3:seo/coderank/raw]`), linked by the same `linkComputerRefs` and rendered by `folder-ref-chip.tsx`, which opens `/admin/files` on that folder and downloads nothing. A `[file:<id>]` ref (`src/utils/file-ref.ts`) is always a saved file, whose chip opens `/media/<id>`; the two are never guessed from a name's extension.

Ingestion (`bot_file_chunks`, `bot_file_vectors` a vec0 table, `file_ingest_queue`): `ingestBotFile` extracts text (`extractBotFileText`, reusing `fileToText`; images and audio only on request), chunks it (`chunkText`), embeds it through the `embedding` harness and fits every vector to the fixed `AppData.EmbeddingDimensions` (`fitVectorDimensions`, pad or cut then normalize). The `file-ingest` worker (`src/file-ingest/`, started from `server.ts` next to the bots queue) works the queue; `ingest-pending-files` sweeps up anything stranded. With no embedding model the chunks are still stored and `searchBotFiles` falls back to word matching, always saying which it did.

Two sqlite-vec limits the code works around: a vec0 table's metadata columns are NOT NULL (a raw insert fills `created_at`/`updated_at` itself), and a side-car column can't be filtered in the same query as `MATCH` (the search pulls the nearest `k` and filters scopes in code). Never pass `--vector` to `bunx bun-sqlite schema` in `entrypoint.sh`: it would wipe every vector on each boot. Changing `AppData.EmbeddingDimensions` needs a manual `--vector` run plus a re-ingest.

A third limit is SQLite's, and it bit this code: foreign keys are **off per connection** unless `enableForeignKeys` (`src/functions/backend/db/enable-foreign-keys.ts`) ran, and only `src/server.ts` runs it. A script or CLI process that deletes rows therefore never fires the schema's `ON DELETE CASCADE`, so `deleteBotFile` and `purgeStorage` delete `bot_file_chunks` themselves instead of leaving it to the cascade. A chunk left behind fails the next `bun run db:schema` with `Foreign key violations after schema sync: bot_file_chunks -> bot_files` (bun-sqlite runs `PRAGMA foreign_key_check` after a sync). Any new helper that deletes a parent row must delete its children explicitly for the same reason.

A bot's or workspace's folder is removed with it (`purgeStorage`, called from `deleteBot` and `deleteWorkspace`, which read the media ids first).

### Files browser (`/admin/files`)

Every folder on the data volume, browsable read-only: a bot's own, a workspace's shared one, a computer's home (`DATA_DIR/computers/<id>`, where everything a bot left in `~/…` actually is), the media library, an MCP server's install folder and the database. A `RailPages` entry beside Routines, Activity and Help.

- **Volumes:** `src/utils/storage-volumes.ts` (client-safe): `StorageVolumes` (`bots`, `workspaces`, `computers`, `media`, `mcp`, `db`) with each one's ref name (`bot`, `computer`, …), `parseStorageSpec({ volume, owner_id })` (null for anything unknown, so a bad id never becomes a path), and `storageSpecHref({ spec, path })` → `/admin/files?volume=<v>&owner=<id>&path=<p>`, the one place a link is built.
- **Access:** `resolveStorageSpec` (`src/functions/backend/storage/resolve-storage-spec.ts`) is the only gate, called before any path exists: a bot through `grabUserBot`, a workspace through `grabUserWorkspace`, a computer through `grabUserComputer` (so a member sees only their grants), `media` only their own (super admins and admins anyone's), `mcp` for crew managers, `db` for the super admin alone. `grabStorageSpecs` lists the same folders for the picker, with `reachableBotsSql` (`src/utils/crew-access/reachable-bots-sql.ts`, tested beside the other crew SQL) so the picker never offers one the resolver refuses.
- **Paths:** `resolveStoragePath` (`src/functions/backend/storage/resolve-storage-path.ts`) applies the `resolveBotPath` rules (no absolute paths, `..`, NUL, too deep, symlink escape) to any volume root.
- **Reading:** `listStorageEntries` (folders first, `AppData.StorageBrowserPageSize`, a missing folder shows empty, symlinks are skipped, `bot_files` rows joined by `(scope, owner_id, path)` so those files keep opening through `/media/<id>`) and `grabStorageFile` (a regular file under `AppData.StorageBrowserReadMaxBytes`). API in `src/pages/api/admin/storage/` (`list`, `file`, the latter with an ETag like `handleMediaServer`), zod in `src/schemas/storage-schema.ts`.
- **Page:** `src/pages/admin/files/index.server.ts` + `index.tsx` (in `SettingsShell`, `wide`), route-local code: `(sections)/files-browser.tsx` and `(sections)/files-pane.tsx` (state, `history.pushState`, so a folder link is shareable and the back button walks back up), `(partials)/files-volume-row.tsx`, `files-breadcrumb-row.tsx`, `files-search-input.tsx`, `storage-entry-row.tsx`, `(hooks)/use-storage-folder.ts` (the page's own copy first, the API on navigation). Read-only: deletion stays in the chat's files modal and the bot's tools.
- **Chips:** `[folder:…]` refs open it, and the bot files modal links into it ("Browse this folder in Files").

### Harness (the model layer)

`src/harness/` is the brain: every model call goes through it. Install-wide, managed by super admins at `/admin/settings/models`.

- **Harnesses** (`Harnesses` in `src/dict/harness-dict.ts`): `text` (chats, plans, calls tools, drives computers; always on), `vision` ("Media reading": images/scans/screenshots for a text model that can't see), `transcription` (speech to text: speech-to-text models only, never chat models that hear audio), `speech` (text to speech: text-to-speech models only), `image` (image generation), `embedding` (turns bot file chunks into vectors for `search_files`: embedding models only, never auto-picked, so it stays unavailable until a super admin picks one, and search falls back to words without it). Each has one `harness_settings` row (no row = automatic, on, with fallback): a `provider_id` (a `harness_providers` row the harness is limited to, for its pick and its automatic picks; null for any provider; with no row, the default provider, OpenRouter, `grabDefaultHarnessProviderId`), a pinned `model_id` (a `harness_models` row from that provider, not a foreign key) or null for automatic, `fallback` (also try the automatic picks), `enabled`, `options` (`voice`).
- **Providers** (`HarnessProviderKinds` in `src/dict/harness-provider-dict.ts`): OpenRouter (default, created by `ensureDefaultHarnessProvider`), OpenAI, Anthropic, Gemini, DeepSeek, Groq, Mistral, xAI, Ollama (local, its models count as free), any OpenAI-compatible server. Rows in `harness_providers` (a private table: the key is encrypted with `src/utils/crypt.ts`, never sent to the client; falls back to the preset's env, e.g. `OPENROUTER_API_KEY`, via `grabEnv`). One adapter today, `src/harness/providers/openai-compatible/` (`chat`, `transcribe`, `speak`, `generateImage`, `embed` (`POST /embeddings`, batched), `listModels`), chosen in `grab-harness-adapter.ts`; a provider needing a native API gets its own adapter there (`HarnessAdapter` in `src/harness/types.ts`).
- **Catalog:** `harness_models` per provider (modalities, tools support, voices, Artificial Analysis scores, `free`, `available`). The `sync-harness-models` cron (hourly, on start) refreshes every enabled provider (`syncAllHarnessModels`); unlisted models are kept but marked unavailable. OpenRouter lists every modality with capabilities; other providers list ids, so capabilities are guessed (`guess-model-modalities`). Free on OpenRouter means `:free` or `openrouter/free`, never zero token prices.
- **Automatic picks** (`rank-auto-models`): only for the `automatic` harnesses in `Harnesses` (text, vision, image); transcription and speech have none and are not available until a model is picked (the model select reads "Not available", no fallback checkbox). Free models that can do the job (on the setting's provider when it has one; a provider without free models needs a picked model) (`model-serves-harness`: text needs tools), by vendor `AutoModelVendorPreference` (DeepSeek, Qwen, NVIDIA, Liquid), then other vendors that look competent (`AppData.HarnessAutoMin*`), then routers; within a vendor by intelligence index, then full-size before small, newest first.
- **Calling:** `resolveHarnessCandidates` (pinned, then automatic) -> `withCandidates` tries them in order with process-wide cooldowns (`resolve/harness-cooldowns.ts`): rate limits and outages rest the model (until `Retry-After` when given), 401/402 rest the whole provider, 403/404 rest the model for an hour. Errors are `HarnessError` (`code` provider|unavailable|aborted, `retryable`), classified in `to-harness-error.ts`. Entry points: `text/generate-text.ts` (one turn), `text/run-text-loop.ts` (tool loop, sticks to one model, wraps up after `max_steps`, `AppData.HarnessMaxToolSteps` by default and the install's own saved limit for a task run, `grabHarnessMaxToolSteps` in `app-settings/grab-run-limits.ts` reading the `harness_max_tool_steps` app setting, and `grabReplyMaxToolSteps`/`bot_reply_max_tool_steps` for a chat reply in place of `AppData.BotReplyMaxToolSteps`; `src/utils/read-step-limit.ts` falls back to the code default for anything missing, below 1 or past `AppData.BotStepLimitMax`; trims the oldest tool output before each turn to fit the model's context, `fit-transcript-to-context.ts`), `media/read-image.ts`, `media/read-media.ts` (documents locally via `fileToText`, images and audio through their harnesses), `audio/transcribe-audio.ts`, `audio/speak-text.ts` (returns the provider's audio as a stream, falling back to the next model only until one answers), `image/generate-image.ts`, `embeddings/embed-text.ts` (batches, then `fit-vector-dimensions.ts` pads or cuts each vector to `AppData.EmbeddingDimensions` and normalizes it, since a provider's width varies). A text model that can't take an image or audio part gets it read by the vision/transcription harness first (`prepare-messages-for-model.ts`).
- **Per-bot models:** in a workspace, its `workspace_harness_settings` row wins (`source: "workspace"`). Otherwise a worker bot inherits the defaults per harness unless it has a `bot_harness_settings` row (same fields as `harness_settings`, unique per bot and harness). `grabHarnessSetting({ harness, bot_id })` returns the bot's row (`source: "bot"`) or the default; the coordinator always gets the default. Every entry point takes `bot_id` and passes it on (text loop -> media reading/transcription), so a bot's run uses its own models throughout. Saved by `saveBotHarnessSetting` (`inherit` deletes the row); the page data is `grabBotHarnessOverview`. A bot's chat header has a ghost model icon (`(partials)/header-icon-button.tsx`; the text model and provider show under the bot's name) that opens a modal with every harness (`(partials)/bot-models-modal.tsx`); the modal and `/admin/bots/[bot_id]/models` both use `(partials)/bot-harness-form.tsx`. On the coordinator the modal edits the defaults themselves (`api/admin/harness/save-settings`, the same settings as Settings, under Models, without the providers): super admins only, the icon is disabled for everyone else. Every one of these wraps `settings/models/(partials)/harness-settings-form.tsx`: per harness a provider select first (`harness-provider-select.tsx`: any provider, one provider, or on a bot "Same as the default"), then the model select narrowed to it. Removing a provider (`deleteHarnessProvider`) puts default and bot settings that picked it or its models back on any provider (`save` prop; "Same as the default" shows when the overview item has `default`).
- **Tools:** `HarnessTool` (built per run with its bot bound): own tools validate with a zod `schema`; tools described elsewhere pass a JSON schema as `parameters` and validate themselves. `run-tool-call.ts` reports bad calls and failures back to the model as `Error: ...`, and caps every tool call at `AppData.HarnessToolTimeoutMs` (aborted, its result handed back as an error the model can work around). Bot tools in `src/harness/tools/storage/`: `list_files`, `read_file`, `write_file`, `search_files`, `move_file`, `delete_file`, `ingest_file`, `copy_file`, `generate_image`, `make_pdf` (`grabBotFileTools`, every run, see Bot files and folders). Bot tools in `src/harness/tools/bot/`: `update_memory`, `update_instructions` (`grabBotTools`, every run), `start_task` and, for the coordinator with workers, `delegate_task` (`grabBotChatTools`, chat replies only; a task handed over in the coordinator's own chat runs in the worker's own chat unless it names a `workspace_id`, which puts it in that workspace: the worker's workspace instructions and memory, the workspace's context, files and computers apply, and it needs membership, added first via `addBotToWorkspace` when missing; handed over from inside a workspace it always runs there and a different `workspace_id` is refused; the job's `requested_by_bot_id` is the coordinator, and `relayJobToRequester` posts the worker's result or failure in its conversation, pushed as `relay` by `pushBotNotice`), `ask_user_to_take_over` (see Take-over), plus for the coordinator `create_workspace` and `update_workspace` (a new workspace with its shared context, or changing one; both also in its tasks, so a task that gathers material can write it into the workspace), `create_bot` (add a worker with its name, role and instructions, and join it to a workspace straight away) and, in its own chat, `add_bot_to_workspace` (put an existing worker in one of the user's workspaces) and `remove_bot_from_workspace` (take it out, keeping its conversation there), `assign_computer` (grant or revoke a worker's direct access, `add/remove-bot-to/from-computer`; access through a workspace stays and the result says so), `update_bot` (change a worker's name, role or core instructions, `src/functions/backend/db/bots/update-bot.ts`), `find_bot` and `find_workspace` (look the crew, or the live workspaces, up by id, name, role/description or instructions/context), `delete_bot` (a worker with its conversations, tasks, schedules and workspace places; the coordinator can't go) and `delete_workspace` (its conversations, tasks, schedules and links; its bots and computers stay). `search_computers` (`src/harness/tools/computer/search-computers-tool.ts`, chat replies and tasks) runs one read-only command on all of a bot's computers at once over the sandbox REST API (`run-computer-command.ts`), each output under its `[computer:ID]`, secrets redacted, capped by `AppData.ComputerSearch*`.
- **MCP:** `src/harness/mcp/`: `connectMcpServer` (Streamable HTTP, SSE fallback), `grabMcpTools` (each server tool becomes a `HarnessTool` with its JSON schema), `mcpResultToText` (text as is, images read by the vision harness, `isError` thrown). Computers are reached this way: `openComputerTools` (`src/harness/tools/computer/`) connects to the computer's AIO sandbox at `grabComputerMcpUrl` (the `MAIN_COMPUTER_URL` app setting for the main computer, for a dev server outside docker, e.g. `http://localhost:8170`, else `http://<hostname>:<port>/mcp`, or its IP address when the hostname can't be reached), leaves out `ComputerMcpExcludedTools`, and sets the computer's envs (see Envs).
- **MCPs and custom tools (extensions):** installs add their own MCP servers (`mcp_servers`, Settings, under MCPs) and tools (`custom_tools`, Settings, under Tools). A bot gets the ones given to it (`bot_mcp_servers`, `bot_custom_tools`) and, in a workspace, that workspace's (`workspace_mcp_servers`, `workspace_custom_tools`); `botExtensionsSql` (`src/utils/bot-extensions-sql.ts`, tested) resolves a run's, `grabBotExtensions` reads them. `all_bots` ("every bot by default") only decides what bots are given: turning it on gives it to every bot (`attachToAllBots`, the coordinator included) and every new bot gets it (`attachDefaultExtensions`, from `createBot` and `ensureCoordinatorBot`); turning it off takes it from no one, and a bot that drops one keeps it dropped (`src/scripts/attach-every-bot-extensions.ts`, safe to rerun, moved installs from the old "reaches every bot" meaning). Super admins define them. Super admins and admins (the super admin alone for the coordinator, `assertBotExtensionManager`) give and take them per bot on the bot's settings page and under its chat's composer (`composer-tools.tsx`: one "N tools" button (`composer-tools-modal.tsx`) counting everything it can use there, which opens a modal listing them (a workspace's not removable) with the add picker, so ten of them don't fill the composer; beside it `composer-computer-button.tsx` opens the bot's current computer in the viewer), and per workspace on the workspace's settings page; the settings pages share `settings/(partials)/extensions-group.tsx`, all through `useExtensions` (`src/hooks/`), `grabBotExtensionsOverview` / `grabWorkspaceExtensionsOverview` (`bot_extensions` / `workspace_extensions` page props, both on `grabExtensionsOverview`) and `api/admin/bots/` and `api/admin/workspaces/` `add-extension`, `remove-extension` (`setBotExtension`, `setWorkspaceExtension`); the MCPs and Tools pages' Access modal sets bots and workspaces in bulk (`setExtensionAccess`, an admin never touches the coordinator's or `all_bots`). Helpers in `src/functions/backend/db/extensions/`, API in `api/admin/mcps/` and `api/admin/tools/`, zod in `src/schemas/extension-schema.ts`.
    - **Skills** are the third extension kind (`skills`, Settings, under Skills; `bot_skills` / `workspace_skills`, same `all_bots` rules, `ExtensionTables.kind = "skill"`). A skill is a runbook: `name`, a unique `slug` a bot loads it by, a one-line `description` (when it applies) and the markdown `body`, capped by `AppData.SkillBodyMaxChars`. Unlike an MCP server or a tool it has no `status`/`last_error`: nothing connects or runs, and it never reports a failure (`ExtensionRow` and `ComposerToolRow` key the state line off `kind == "skill"`). A run sees only the slug and description; `openBotSkills` (`src/harness/tools/skills/`, fed by the `skills` that `grabBotExtensions` already reads) adds one `load_skill` tool that returns the body, cut at `AppData.SkillBodyShownChars` with a line saying so, and throws the valid slugs for an unknown one. Slugs are checked by `checkSkillSlug` (`src/utils/check-skill-slug.ts`, tested); `load_skill` is in `ReservedToolNames` so a custom tool can't shadow it. The settings page is `api/admin/skills/` (`save-skill`, `delete-skill` super admin only, `set-skill-access`), `grabSkillsOverview`; `deleteSkill` deletes the two link tables itself, since foreign keys are off outside the server process. `openBotExtensions` (`src/harness/tools/extensions/`) turns them into harness tools for chat replies and tasks (built-ins win a name clash; `close()` in the runners' `finally`; `note` tells the bot about a server it can't use). Templates (`fillTemplate`, `src/utils/fill-template.ts`): `{{args.x}}` and `{{env.KEY}}`, envs from `grabRunEnvs` (the bot's own beat the workspace's, which beat general; app settings never, no need to be set on computers); secret values are redacted from every result.
    - **Custom tools** (`CustomToolKinds` in `src/dict/custom-tool-dict.ts`): a name (`checkCustomToolName`: snake_case, never a built-in's, `ReservedToolNames`), a description, a JSON-schema `parameters` and a `config` per kind, built by `buildCustomTool` (`src/harness/tools/custom/`): `http` (`runHttpTool`: the app fetches, arguments URL-encoded in the URL, no body template sends the arguments as JSON), `script` (`runScriptTool`: bash/python/javascript on the run's computer, the task's or in chat the bot's current one, over the sandbox REST API in a subshell, computer envs set, arguments as JSON in `COBBLE_ARGS` and on stdin; left out without a computer), `prompt` (`runPromptTool`: one `generateText` turn with the bot's models). Super admins try one from the page (`testCustomTool`, the main computer for scripts): a saved one from its row (Try), or the form as it is, saved or not (`draft`, nothing recorded), from the Test panel in its modal (`custom-tool-test-panel.tsx`). Every run (a bot's call or a try; not bad arguments or a cancel) is saved as the tool's `status`, `last_error` and `last_checked_at` (`recordCustomToolRun`); editing resets it to `untested`.
    - **MCP servers** (`McpTransports` in `src/dict/mcp-dict.ts`): `remote` (URL plus templated headers, `connectMcpServer`) or `stdio` (a command run with the host's shell, `hostShell`, in `DATA_DIR/mcp/<id>` (`grabMcpDir`), after its `setup_command`, which `runMcpSetup` reruns whenever it changes (marker file) or on Reinstall; its env is only its own `env` map plus the shell basics, `grabMcpStdioEnv`, never the app's). Their tools are listed into `tools_cache` (`syncMcpServer`: on save, Refresh, and hourly for remote ones by the `sync-mcp-servers` cron) and named `<slug>__<tool>` (`mcpToolName`); the modal's Test connection checks the form as it is, saved or not, outside the pool (`testMcpDraft`, `testMcpServer`, `api/admin/mcps/test-mcp`, a new stdio server's setup runs in `mcp/0`); a run that can't connect, or connects again after failing, updates its `status` too (`recordMcpServerConnection`); runs build tools from the cache (`mcpHarnessTool`) and connect on the first call (`openMcpServer`). stdio servers live in a per-process pool (`src/harness/mcp/mcp-stdio-pool.ts`), stopped after `AppData.McpStdioIdleMs` idle, on save or delete.
- **Check before and after acting:** every reply and task gets `ActionCheckNote` (`src/data/harness-prompts.ts`): look for an existing record before creating one (update it instead), and verify every action took effect before saying it's done. The tools back it: `update_memory` and `update_instructions` read the save back (`checkSavedText`, `grabBotSavedText`) and fail if it didn't take; `create_bot` refuses a name already on the crew, `create_schedule` a twin of one of the bot's schedules (`findDuplicateCronJob`: same place and same title, or same timing and brief), and `start_task`/`delegate_task` a task with the same title still open in that session (`grabOpenBotTask`); `create_workspace` already refuses a name in use.
- **Replies and tasks:** `run-bot-reply.ts` answers from the last `BotReplyHistoryMessages` messages (`botMessagesToHarness`) with `ChatReplyNote`, the bot's computers (`compose-computers-note.ts`: what they are and that they're reached through `start_task`) and the chat tools, at most `grabReplyMaxToolSteps()` steps (the `bot_reply_max_tool_steps` app setting, else `AppData.BotReplyMaxToolSteps`), and finishes silently if the bot already had the last word (`enqueueBotReply` reuses a pending reply, so a burst of messages gets one). `run-bot-task.ts` runs the brief (`compose-job-brief.ts`) with `TaskRunNote`, the bot tools, the file tools and the job's computer tools, closing the MCP connection after. Both add `BotFilesNote` (where a bot's files live, that they persist, and to search before saying it doesn't know) and collect `attachments` for files the bot wrote with `attach`, which `completeBotJob` links to the result message. Retryable harness errors go back to the queue's backoff, the rest are `BotJobFatalError`.
- **API:** `api/admin/harness/` (`save-provider` also syncs, `sync-provider`, `delete-provider`, `save-settings`), super admin only, each returning the fresh `grabHarnessOverview()` page data.

### Token usage

Every model call is written down in `token_usages` as it happens, so a message can show what it cost and `/admin/usage` can total the install's spending by day, by model and by provider.

- **Whose call it was:** `src/harness/usage/harness-usage-scope.ts` is an `AsyncLocalStorage` scope (`user_id`, `bot_id`, `workspace_id`, `session_id`, `job_id`, `message_id`) that a run enters once, in `run-bot-job.ts` (`runWithHarnessUsageScope`), so everything underneath - the text loop's turns, a session summary, a transcription, an image, an embedding - is attributed without threading the ids through every harness function. The store is merged, not replaced: a nested run keeps the outer ids and overrides only what it knows better, and a `null` is a real value that clears a field (a bot's own chat has no workspace). `withUsageScope` (`with-usage-scope.ts`) wraps one call the app makes on a user's behalf outside any run (the chat's transcribe and speak routes), and `runWithFallbackScope` (`run-with-fallback-scope.ts`) attributes work whose run may be missing - a file the ingest queue embeds has no run, so it is charged to its owner (`grabUsageOwnerUserId`, the workspace's creator else the bot's) - while `mode: "own"` forces the given scope (the session namer, whose job is already complete and must not pay for it). Without an owner nothing is written: a row no one can be shown is not worth a row.
- **Recording:** `record-harness-usage.ts` writes one row and never throws (a model that answered must not lose its answer to a failed insert; it logs with `logError` instead). `build-usage-call.ts` turns the candidate that really answered into the row's provider and model (`provider_id`/`provider_name` and the catalog `model_id` plus the provider's `model_slug`), and providers that report no usage still get a row, so a call is never invisible. Every entry point calls it in `withCandidates`'s `on_success`: `generate-text.ts`, `read-image.ts`, `transcribe-audio.ts`, `speak-text.ts` (billed per character, so the text's length stands in for what was spoken), `generate-image.ts`, `embed-text.ts`.
- **Messages:** a reply or task job's calls carry the job's id as they happen, since its message doesn't exist yet; `complete-bot-job.ts` then runs `UPDATE token_usages SET message_id = ? WHERE job_id = ? AND message_id IS NULL` once the message is inserted, so the thread can show what the answer cost. Calls that produce no message (naming a session, embedding a file, transcribing a dictation) keep a null `message_id` and are still counted in the totals.
- **Message usage:** `grab-message-usage.ts` sums a batch of messages' calls (one `GROUP BY message_id`) and `grab-bot-messages.ts` attaches it as `BotMessageWithAttachments.usage` (`TokenUsageSummary`: prompt, completion, total, cost, calls). `formatTokens` (`src/utils/format-tokens.ts`) writes a count as a person reads it (842 / 1.2K / 842K / 1.2M). `MessageUsageChip` (`bots/[bot_id]/(partials)/message-usage-chip.tsx`) sits in the actions row under a bot message beside its timestamp and play button, showing the total with the breakdown in its `title`, and renders nothing when a message has no calls.
- **Usage page:** `/admin/usage`, a `RailPages` entry like Routines and Activity rather than a settings sub page, since it is the install's own record rather than a setting (super admins and admins only, `canManageCrew`; the dict entry's `can_view` hides it for anyone else, and `grab-usage-server-props.ts` redirects them to the bots). Helpers in `src/functions/backend/db/usage/`: `grab-usage-overview.ts` runs `grab-usage-totals.ts` (one total and the same split per harness), `grab-usage-days.ts` -> `bucket-usage-days.ts` (one bucket per day, empty days at zero), `grab-usage-models.ts` (per provider, model slug and harness, busiest first, joined to `harness_models` for a display name), plus the previous period of the same length and a rolling week. The window comes from `grab-usage-window.ts` (`UsageDefaultDays` 30, `UsageMaxDays` 365, clamped). API `api/admin/usage` (GET only) for another period, zod in `src/schemas/usage-schema.ts`.
- **Days are local, not UTC:** `grabUsageDays` groups in SQL with `date(created_at / 1000, 'unixepoch', 'localtime')`, matching the local midnights `grabUsageWindow` builds and the `YYYY-MM-DD` `localDate` writes. Grouping by the bare UTC day put a late evening's calls in the previous bucket and silently dropped them whenever no bucket carried that name (the chart's total disagreed with the totals card); `bucket-usage-days.test.ts` pins this. Never name a local day with `toISOString().slice(0, 10)`.
- **The page:** `index.server.ts` -> `(functions)/grab-usage-server-props.ts` (crew managers only, the period in the address), `index.tsx` in `SettingsShell` with `wide`, `(sections)/usage-section.tsx` (state in the address through `history.pushState`, `use-usage-overview.ts` loading another period), `(partials)/usage-totals.tsx` (tokens, model calls, cost, previous period with a change label), `usage-day-chart.tsx` + `usage-day-bar.tsx` (bars from static `h-*` classes in `(data)/usage-bar-heights.ts`, since inline styles are not allowed), `usage-harness-breakdown.tsx` + `usage-harness-row.tsx`, and `usage-models-table.tsx` + `usage-model-row.tsx` (calls, in, out, tokens, **price per thousand tokens** worked out from cost and tokens, which is what makes one model's cost comparable with another's).

## Admin Route Hierarchy

Live routes under `/admin` (see `AdminSections`): `/admin/bots`, `/admin/workspaces` (the index opens the first workspace), `/admin/routines`, `/admin/logs`, `/admin/help` and `/admin/settings`. `/admin` redirects to `/admin/bots`; a workspace's dashboard is `/admin/workspaces/<id>`. The routes in detail:

- `/admin/bots` (live) - the Cobble chat shell, plus `[bot_id]/`
- `/admin/routines` (live) - every schedule the user's bots have (see Schedules), opened from "Routines" in the rail's user popover
- `/admin/usage` (live) - what the crew's model calls spent, per day, per kind of call and per model, for crew managers (see Token usage), opened from "Usage" in the rail's user popover
- `/admin/logs` (live) - Activity, the audit log: super admins only (others are redirected; the popover's "Activity" item shows to super admins only). Rendered in `SettingsShell` with `wide`
- `/admin/help` (live) - familiar problems and their fixes, one `SettingsGroup` per topic (`(data)/help-topics.ts`, markdown rendered with the chat's `MessageMarkdownComponents`, the topic id as the group's anchor). First topic: signing in to Google and other strict sites with Cookie-Editor over the Desktop view (`HelpSignInTopicId` in `src/dict/help-dict.ts`; `ask_user_to_take_over` points bots to it)
- Routines, Files, Usage, Activity and Help are the `RailPages` (`src/dict/rail-pages-dict.ts`), opened from the rail's user popover (which lists them, then Settings and Sign out). On them the rail shows its own aside (`(partials)/rail-pages-list.tsx`): a back link to the bots, then those pages, like settings. A page carries `super_admin_only` or a `can_view` rule (Usage is `canManageCrew`), and `canSeeRailItem` (`src/utils/rbac/can-see-rail-item.ts`, tested) decides both the popover and the aside, so a page is never offered in one and hidden in the other
- `/admin/settings/backups` - super admin only (listed in `SettingsPages`), two groups. Database snapshots: create, list, restore, download and delete in `db/.backups` (`bun-sqlite backup`); "Restore from file" beside "Create backup" opens `upload-restore-modal.tsx` (upload a SQLite database file, which is saved to `.backups` then restored with `bun-sqlite import <file>`, since `restore` is interactive). Whole-data archives: `write-data-backup-archive.ts` tars all of `DATA_DIR` (database, media, computer homes, bot and workspace files, MCP installs) into `BACKUP_ARCHIVE_DIR` as `cobble-data-<epoch-ms>.tar.gz` (via `tar -czf` into a `.part` name, renamed on success so a cut-off run is never listed, with a `--transform` putting every member under one root folder named after the archive), keeping the newest `AppData.DataBackupsKept` and never touching `db/.backups`; its own "Restore from file" uploads a tar.gz (`upload-restore-data.ts`, gzip header checked first). `restore-data-backup.ts` extracts an archive to a temp dir, requires exactly one root folder holding a `db` folder (`findArchiveRoot`, since an upload is saved under a fresh timestamp and its inner name no longer matches the file name), takes a safety archive before touching anything (`write-data-backup-archive.ts`, not the locked `create-data-backup.ts`), empties `DATA_DIR` (the snapshots included) and copies the archive's in, then `restart-app.ts` restarts the process through the docker socket after the response lands (`process.exit`, the container's restart policy brings it back), because the open SQLite handle points at the replaced file. Row actions are icon-only (`RowIconButton`, `src/components/general/row-icon-button.tsx`): restore, download, delete. `BACKUP_ARCHIVE_DIR` is a `ProcessOnlyEnvKeys` start-up path (the installer mounts `$COBBLE_HOME/backups-archives` there; outside docker it falls back to `~/.cobble/backups`). Every backup and restore, of either kind, runs one at a time via `with-backup-run-lock.ts`. Helpers in `src/functions/backend/backups/`, API routes in `api/admin/backups/`
- `/admin/settings/database` - super admin only (listed in `SettingsPages`), a read-only browser of the install's SQLite database, no writes anywhere in it. `grabDatabaseTables` reads the catalog once (every table and its columns, SQLite's own `sqlite_*` tables and sqlite-vec's `bot_file_vectors_*` shadow tables left out) and adds each table's `COUNT(*)` and its size from `dbstat` (index pages folded back onto their table). `grabTableRecords` then returns one page (`AppData.DatabaseBrowserPageSize`, 50) of the open table's records newest key first, every value already cut in SQL at `AppData.DatabaseBrowserCellMaxChars` (400) so a table of long texts never ships a megabyte of JSON, and a `truncated` flag per cell; the search matches over the concatenation of every column's text (`buildSearchClause`: every word, or the whole phrase), which is what also works on a virtual table, where `MATCH` and `LIKE` over a vector blob do not. A record's key is its single primary key column, else its rowid, else nothing (a virtual table with no primary key reads as plain cells); clicking a cell's row opens `grabTableRecord` for that key, which returns the record whole up to `AppData.DatabaseBrowserValueMaxChars` (20000), the answer to a cell cut at three lines. `clampPage` puts a page past the end on the last one. The page's state (table, search, mode, page) lives in the address through `databaseHref`, so a table or a search can be linked to; the open table's first page comes with the page props and anything else loads through the API. Helpers in `src/functions/backend/database/` (`build-table-catalog-sql`, `build-table-sizes-sql`, `build-records-sql`, `build-search-clause`, `resolve-table-key`, `build-record-row`, `quote-sql-identifier`), API `api/admin/database/records` and `record` (GET only, `grabSuperAdminUser`), zod in `src/schemas/database-schema.ts`, page under `src/pages/admin/settings/database/`
- `/admin/settings` (live) - redirects to `SettingsDefaultPath` (`/admin/settings/general`). Sub pages: `general/` (theme: System/Light/Dark per device via `useThemePreference`, stored in `localStorage[AppData.ThemeStorageKey]`, the same key `ThemeInitScript` and twui's `ColorSchemeSelector` read; applies instantly, no save bar; for super admins the prompt appendix (`prompt-section.tsx`, `system-prompt-append-form.tsx`) saving `api/admin/settings/system-prompt-append` into the `bot_system_prompt_append` app setting, and the run limits (`run-limits-section.tsx`, `run-limits-form.tsx`) saving `api/admin/settings/run-limits` into the `harness_max_tool_steps` and `bot_reply_max_tool_steps` app settings (`grabRunLimitsOverview`, `run_limits` page prop, 0 clearing a limit back to its code default); and, for `canManageCrew`, the bot file quota: `storage-section.tsx`, `storage-quota-form.tsx` saving `api/admin/settings/storage-quota` into `app_settings`; and, for everyone, the running version: `version-section.tsx`, `version-line.tsx` showing `pageProps.running_version` from `runningVersion` (`src/utils/running-version.ts`: `COBBLE_VERSION`, the release tag the image was built with, else `package.json`'s version, in development too, with `(development)` after it and no such suffix in production), `profile/` (current user profile), `notifications/` (this device's Web Push switch, a test send, and every device the user subscribed), `change-password/` (set/change own password) `models/` (super admin only, `super_admin_only` in the dict hides it from the rail: model providers and which model each harness uses) and `environment/` (super admin only: app settings and the envs bots and computers use, per scope), each wrapped in `SettingsShell` (`(partials)/settings-shell.tsx`: 64px pane header like the conversation header, then one centered column that opens with the breadcrumb trail, `src/components/general/admin-breadcrumbs.tsx`: twui's `Breadcrumbs` reading `Home`, the sections and the page off `pageProps.url.pathname`, the last crumb named by the page's own `title` so a record reads `Scout` rather than `3` and `/admin/logs` reads `Activity`; `hasBreadcrumbTrail` drops it only on the crew of bots, which is home, and nothing outside `/admin` gets one). Content is built from `SettingsGroup` blocks (label + description left, controls right, hairline-separated, no cards) and ends with the sticky `SettingsSaveBar` (Save enabled only when dirty, Discard, one-line status). Saves don't reload the page. Keep save notices in component state: `useFormInit` clears `status` whenever the form changes. `bots/` (the crew: chat, edit with `BotForm`, models, delete a worker via `api/admin/bots/delete-bot`), `workspaces/` (open, create through the Add flow, delete), `computers/` (state and resources, live; super admins create, edit with `AddComputerForm`, retry, delete), `mcps/`, `tools/` and `skills/` (see MCPs, custom tools and skills: super admins add, edit, refresh or try, delete; admins only set Access, `extension-access-modal.tsx`; add, edit, access and try open in modals (`AddModalShell`). A skill has no status: nothing connects or runs, so `skill-row.tsx` puts its slug, who gets it (`extension-access-label.ts`) and its description where a server's or tool's status line goes), each MCP server or tool row with a status line under it (`extension-status-line.tsx`: connected or not, last run worked or failed, when, and why), those five for `canManageCrew` only (`grab-crew-settings-server-props.ts`, `can_view` in the dict) and `users/` (for `canListUsers`: super admins and admins; the dict's `can_view` gates it in the rail; add with a password (`canCreateUser`), edit (`canUpdateUser`), both with the one `UserForm` (`admin/settings/users/(partials)/user-form/`, `useFormInit`, photo included, saved in one request), revoke (adds `revoked`, `canRevokeUser`), delete (`canRevokeUser`'s rules, never yourself), pick their user types from `UserTypes` (`user-types-select.tsx`: only the ones `filterAssignableUserTypes` allows, never `UneditableUserTypes`; at least one, since login needs a type; `saveUser` checks each grant and removal with `userTypeAssignment`; the super admin's types are never shown or changed there, and they can't be revoked; the password is entered twice, `SaveUserRequestSchema` checks they match), delete with everything theirs; `api/admin/users/save-user`, `delete-user`). The lists share `SettingsList`, `SettingsListRow`, `SettingsRowAction` and `use-settings-list`. Sub pages are declared once in `src/dict/settings-pages-dict.ts`; on every `/admin/settings/*` page the rail swaps the crew for a back link plus that list. Add a sub page by adding its folder and a dict entry

Every `/admin/*` page renders in the bots layout (`src/layouts/bots/`); there is no separate admin layout or aside. The rail header's Add icon (super admins and admins only, like the rail's "+" buttons) opens the Add flow (`(partials)/add-flow.tsx`, `AddFlowContext`): a bot, a workspace or (super admins) a computer, each form in a modal stacked over the choice, so closing it goes back there (the Bots and Workspaces labels have a "+" that opens the form directly; in a workspace's rail, "Workspace bots" has a "+" that opens `link-bots-modal.tsx` instead: the crew's workers not in it yet (`api/admin/workspaces/linkable-bots`), each added with `add-bot`, which pushes `server:bot-created`, and "New bot" stacking the bot form over it); on a workspace's pages new bots and computers join it. The rail shows the crew (`(partials)/rail-crew-list.tsx`, needs `bots` and `workspaces` in page props): the coordinator, the latest `AppData.RailRecentBots` workers (plus any open, unread or needing you, `pick-rail-bots`), "See all bots" with a few stones opening every bot in a modal, then Workspaces with a "+". On `/admin/workspaces/*` it shows the workspace's crew instead (`(partials)/rail-workspace-list.tsx`: back link, the workspace, then only the bots attached to it, "Workspace bots"; the coordinator stays in the main crew rail, and on a workspace bot's page the back link returns to the workspace page), except on `/admin/settings/*` where it shows "Back to bots" plus the settings sub pages (`(partials)/rail-settings-list.tsx`). Page heroes use the shared `AdminHero` component (`src/components/general/admin-hero.tsx`).

## Page Directory Rules

For route-local code under `src/pages`, use non-routed meta folders:

- `(sections)` - page sections
- `(partials)` - components used by sections
- `(data)` - page-local data
- `(functions)` - page-local functions
- `(utils)` - page-local utilities
- `(hooks)` - page-local hooks

Keep route files (e.g. `index.tsx`) thin - compose from route-local meta folders.

Dynamic routes use `[param]` syntax (e.g. `[id]`, `[user_type]`, `[key]`). Catch-all routes use `[[...paths]]`.

Server functions use `index.server.ts` alongside `index.tsx` (e.g. `single-document/[id]/index.server.ts`).

## Shared Code Organization

- **`src/hooks/`** - Shared React hooks: `use-admin-crud-get`, `use-fetch-get`, `use-form-init`, `use-local-user`, `use-users-get`, `use-server-event`, `use-live-bots`
- **`src/functions/`** - Shared functions split by target:
    - `src/functions/backend/` - Server-only: auth (`backend/auth/`), DB queries (`backend/db/` incl. `bots/`), email client + templates (`email/`), media upload, `logger.ts` (audit logging)
    - `src/functions/frontend/` - Client-safe: CRUD handlers, fetch handlers, media upload/delete handlers
    - `src/functions/server/` - Server utilities (media server handler etc.)
- **`src/utils/`** - Utility modules: cookies, crypt, ejson, money, safe-compare, text-to-html, and more. Most have corresponding `.test.ts` files.
- **`src/utils/rbac/`** - RBAC engine: `user-type-assignment` (matrix interpreter), `is-pass-through-user`, `can-*` permission helpers, `filter-assignable-user-types` - with `bun:test` suites driven by fixture matrices
- **`src/dict/`** - Dictionaries/constants: main links (`main-links-dict.ts`), user types + assignment matrices, media types, document statuses (`document-dict.ts`), websocket rooms, log categories, company data
- **`src/data/`** - Static data: `site-data.ts` (site name/slug - the only place brand literals live), `app-data.ts` (app constants - cookie names, limits, defaults, Paystack endpoint)
- **`src/types/`** - Type definitions: `index.ts` (incl. WebSocket event/room types), `sql-joins.ts`, `paystack.ts`, `search-api.ts`
- **`src/scripts/`** - CLI scripts: `seed-super-admin.tsx` (idempotent super admin seed) + `generate-brand-assets.ts` (favicon/brand asset generator)
- **`llm-code-assistant/`** - `query-db.ts` (read-only SQL CLI for coding assistants)

## Component Systems

| Directory                    | Description                                                                                                                                                 |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `@moduletrace/twui`          | **The** design system - external package. Contains layout primitives, elements, editors (TinyMCE), form components, hooks, SVGs, and utilities. Import from `@moduletrace/twui` or a deep path such as `@moduletrace/twui/layout/Button` |
| `src/components/general/`    | Shared page components built on twui: `logo`, `logo-icon`, `wordmark`, `bot-stone` (+ `bot-stone-face`, `stone-eye-*`), `user-avatar` (round photo or initials, `sm|md|lg`), `code-editor` (twui `AceEditor` for commands and code, loading the ACE scripts on demand via `loadAceScripts`), `icon-tile`, `computer-view` (+ `computer-view-frame`, `computer-view-mode-button`), `inner-page-hero-section`, `main-hero-section`, `admin-hero`, `login-shell`, `tagline-section`, media/image helpers |

## UI & Styling Rules (MANDATORY)

- **Always build UI with the twui component library and the shared `src/components/general/` components.** Do not hand-roll HTML for layout, typography, or controls.
  - Layout: `Section`, `Container`, `Stack`, `Row`, `Center`, `Span`, `P`, `H1`-`H5`, `Button`, `Divider`, `Img`, `Link`, `ArrowedLink`, `Main`, `Header`, `Footer`, `Spacer`
  - Elements: `Card`, `Paper`, `Tag`, `HeaderNav` (+ `MainLinks` dict), `LinkList`, `Tabs`, `Table`, `Breadcrumbs`, `Modal`, `Dropdown`, `CheckBulletPoints`, `EmptyContent`, `Loading`, `LoadingBlock`, `Toggle`, `Toast`
  - Forms: `Form`, `Input`, `Select`, `Textarea`, `Checkbox`, `Radios`, `SearchSelect`, `FileUpload`, `ImageUpload`
  - Shared: `Logo`, `LogoIcon`, `InnerPageHeroSection`, `AdminHero`, `LoginShell` (auth pages/forms), `TaglineSection`
- **UNCHANGEABLE RULE - twui components always win over raw JSX elements.** Use twui components (`H1`-`H5`, `P`, `Span`, `List`, `Img`, `Row`, `Stack`, `Container`, `Section`, `Card`, `Button`, `Tag`, `Link`, etc.) before basic JSX elements (`<h1>`-`<h6>`, `<p>`, `<span>`, `<ul>`, `<li>`, `<img>`, `<a>`, `<button>`, `<div>`-as-card, etc.). Never introduce or reintroduce a raw JSX element where a twui equivalent exists. Override twui defaults via `className` rather than reaching for the raw element.
- **Never create new CSS classes, CSS files, or inline `style={{...}}` hacks.** Style exclusively by composing existing component props/classNames, existing twui classNames (`twui-*`), semantic theme-token utilities (`bg-dark`, `bg-light`, `text-dark`, `text-primary`, `bg-primary`, `text-accent`, `bg-accent`, `text-secondary`, `border-slate-200`, etc.), and utility classes already used elsewhere in this codebase. Build up from what exists; extend component code only when the existing API cannot express the need.
- **Chunk Tailwind classes with `tailwind-merge`, never one long line.** A `className` is `twMerge(...)` (`import { twMerge } from "tailwind-merge"`) with one chunk per claim: base layout and spacing first, then one string per conditional, and a caller's override last, so a later chunk always beats an earlier one instead of depending on source order in the CSS. Never concatenate conditional classes with a template literal or a bare `+`, and never leave a single 20-class string where the parts mean different things: `className={twMerge("w-full shrink-0 flex-col gap-0! items-stretch rounded-default py-1", selected && "bg-ink/5")}`.
- **Theming:** colors, radii, and fonts change only in `src/styles/main.css`: Cobble tokens as CSS vars on `:root` (light) and `.dark`, exposed in the `@theme inline` block as `paper`, `surface`, `sunken`, `ink`, `ink-secondary`, `ink-disabled`, `hairline`, `hairline-strong`, `on-ink`, `hue-<name>`, plus `shadow-popover` and `--m-fast/base/slow` motion vars (zeroed under reduced motion). Font is Lato (stand-in). Never hardcode hex colors, radii, or font stacks inside components. Hues are bot identity only, never state, and the `--color-hue-*` tokens remain for the fixed palettes that use them (the files browser's volume icons, `BotHues`' own hexes).
    - **The one exception:** a bot's colour is user data, not a theme value, so it reaches `BotStone` (`src/components/general/bot-stone.tsx`) and `UnreadDot` as an SVG `fill` attribute - an arbitrary hex has no token to point at. Nothing else in the app may take a colour that way, and `resolveBotColor`/`normalizeBotColor` (`src/utils/bot-color.ts`) are the only place a colour string is validated.
- **Branding:** `docs/readme/hero-light.png`, `hero-dark.png`, `workspace-light.png` and `workspace-dark.png` are screenshots of the current app (chat and workspace dashboard, 1440px at 2x, downscaled to 1600px wide), committed and picked per theme by GitHub's `#gh-light-mode-only` / `#gh-dark-mode-only` fragment. They are not generated: when the UI changes, retake them from a seeded dev instance (`bun run seed`, `bun run seed:bots`) and keep the same names, so the README never shows an older dashboard. Never use `assets/handoff/assets/screens/*.png` for anything: that design is outdated. `public/logos/logo.svg` is the single source of truth for the logo mark; `logo-icon.tsx` renders it inline (via `BotStone`) and `logo.tsx` renders the stone beside the vector `Wordmark`. Whenever the logo changes, run `bun run gen:favicon` to regenerate `public/favicon.ico`, `public/favicon.png`, `public/apple-touch-icon.png`, and `public/logos/logo.png` (the last is used in emails). Font + favicon links live in `__root.tsx` `Head()`.
- **Site name:** brand literals live only in `src/data/site-data.ts`. Page meta titles, emails, and UI copy must interpolate `SiteData["SiteName"]` - never hardcode a brand name in page files.

## Conventions

- Prefer one function/component per file over monolithic files with multiple exports.
- **Modularization is non-negotiable: prioritize modularization over monolithic functions/components.** Repeated UI blocks must be extracted into their own reusable components driven by props (e.g. an info row component taking `{ icon, text }`), never repeated inline markup.
- **Every rendered array must have a separate component for each array item.** Always extract the item JSX into its own component and render it via `items.map((item) => <ItemComponent {...item} />)` - never inline the item's JSX inside the map callback.
- **No em dashes (U+2014) anywhere** in the repo: code, comments, docs, markdown, or user-facing strings. Use a regular hyphen (`-`) or restructure the sentence instead.
- Always use object parameters (`{ foo, bar }`) instead of comma-separated positional parameters.
- Prefer existing patterns over new abstractions.
- **Do not patch `@moduletrace/twui`** - it is a general, publishable library shared across projects. Do not edit the installed package in `node_modules`; compose or override with `src/components/general/` components and `src/styles/main.css` instead.
- `.env` is gitignored but exists locally with secrets. Do not commit it. It is also excluded by `.dockerignore`; production passes it with `docker run --env-file`.
- **Rate limiting:** `src/utils/rate-limit.ts` (in-memory fixed window, per process). Unauthenticated endpoints that send email or check secrets (`password-login`, `signup`, `contact`) must be rate limited.
- **Validation:** validate untrusted request bodies with the zod schemas in `src/schemas/` via `parseWithSchema`. Throw `AppError` for messages that are safe to show; catch blocks return `safeErrorMessage` so internal errors never reach the client.
- **HTML columns:** any `html: true` column that accepts user input must be sanitized with `sanitizeRichTextHtml` (or `sanitizeUserBio` for `users.bio`) before it is stored or rendered with `dangerouslySetInnerHTML`.
- **Outgoing email outside production** goes through `resolveEmailRecipient` (`src/functions/backend/email/resolve-email-recipient.ts`) and is redirected to `DEV_EMAIL_RECIPIENT`. Never hardcode recipient addresses.
- **Never trust ids from the request body for ownership or sessions.** Use the session `user.id`, and gate acting on other users with the `src/utils/rbac/` helpers (`canUpdateUser` etc.).
- `secrets/` directory is also gitignored.
- `external/` binary directories (`**/models`, `**/bin`, `**/x-86`, `**/arm-64`) are gitignored.