Commit Graph
45 Commits
Author SHA1 Message Date
tbenandClaude Opus 5.5 93896646c7 Let the coordinator and workspace admins hand over tasks and routines
delegate_task takes a task_id to hand over a task already on a list: it
moves to the worker (a workspace's task stays on its list), goes to
doing, and the job is linked back so its end is noted on the task.
hand_over_schedule moves a bot's own routine to another bot, keeping its
workspace. Users get Hand over in the Tasks UI and both routine lists.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 11:23:26 +01:00
tbenandClaude Opus 5.5 373210c2a5 Make the skill runbook limits a general setting
The longest runbook a skill may save and how much of it load_skill hands
back are now set on Settings > General (skill_body_max_chars,
skill_body_shown_chars), defaulting to AppData.SkillBodyMaxChars and
SkillBodyShownChars. Saving, confirming and proposing a skill check the
install's limit; a skill saved before it was lowered can still be toggled.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 09:43:00 +01:00
tbenandClaude Opus 5.5 ea62dd11f0 Add bot types with a permission matrix, and workspace admin bots
A bot's powers came from bots.kind alone, so only the coordinator could
hand work to other bots. bot_types now works like user_types: a row with
no workspace holds everywhere (the coordinator), one with a workspace
only there (a workspace admin), and no row is a worker. bots.kind stays
the legacy marker of the one coordinator for who sees it.

The matrices in src/dict/bot-types-dict.ts say what each type may do,
whom it may direct and who grants it, read by one generic interpreter
(rbac/type-assignment.ts) that userTypeAssignment now wraps too. Every
run reads its powers once (grabBotRunPowers) and the chat tools, task
runs and task tools gate on them instead of kind; a run's powers are its
type's and its user's together.

In the workspace it leads, an admin hands tasks to the workers there
(results relayed back to it), sets their instructions, adds tasks to
their lists and keeps the workspace's context when its user may. Super
admins and admins grant it from the workspace's settings or the chat
header's menu, and the coordinator with set_bot_type. Admins are marked
with a filled green shield beside their name across the workspace, kept
live by pushBotUpdated's admin_workspace_ids.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 09:04:14 +01:00
tbenandClaude Opus 5.5 86becd50bc Keep every bot's computer work in its own workspace folder
Computers now mount only /home/gem/workspace from DATA_DIR/computers/<id>,
so only the workspace persists and goes into the data archive. An older
computer's whole home is moved aside to <id>-home-<ts> (kept, never
archived) and its container recreated once by sync-computers.

Each run works in bots/<bot>/users/<user> (also its cwd and HOME), with
bots/<bot> for work every user shares and workspaces/<id> shared by a
workspace's bots; the coordinator reaches the whole workspace. Computer
tools are gated by withComputerDirs: bash and code start there, file tool
paths and file: URLs outside the run's folders are refused, and so is sudo.
search_computers, import_file and script tools start there too.

The files browser shows members only the bots' folders they reach, their
own user folder in each and their workspaces', with names on the id folders.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 08:24:49 +01:00
tbenandClaude Opus 5.5 fefc295d15 Let the coordinator manage skills, MCP servers and tools, behind a confirm card
- Coordinator tools in its own chat (super admin only): list_extensions,
  set_extension_access, delete_extension, and update_skill /
  update_mcp_server / update_custom_tool for what can't change where
  anything is sent (on/off, names, arguments, left-out tools)
- Adding a skill, MCP server or tool, or changing a skill's words, a
  server's URL or headers or a tool's kind or config, goes through
  propose_skill / propose_mcp_server / propose_custom_tool: stored in
  extension_proposals with the bot's background check, shown as a
  [proposal:ID] chip that opens a card with Confirm and Reject
- The card's facts are worked out by the app from what Confirm saves: the
  host, every {{env.KEY}} and whether it's saved, the config; for a skill
  its links, hosts, tool names, mentions of secrets and a diff against
  the saved runbook
- Confirming decides once and checks again; a server or tool missing a key
  is saved switched off, and the outcome is posted to the bot in chat
- Never from a bot: stdio servers, a URL whose host a template decides,
  envs in a prompt tool, a key's value

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 06:08:09 +01:00
tbenandClaude Opus 5.5 b0fe619ae8 Let bots take in messages sent while they reply
A reply read the session once at the start, so a message sent while it
ran was never seen, and the follow-up reply queued for it then finished
silently: the message was saved before the reply, so the thread ended
with the bot. Those messages went unanswered.

The text loop now reads new user messages after every round of tools and
before an answer goes out, and the bot decides from what each one means
whether it changes the work (folded in, or the unsent answer redrafted)
or is a separate request, which it acknowledges at once with the new
send_message tool and does after the current work. Messages sent that
way are marked interim, so the thinking row stays up while the bot
works. A reply job records the newest user message it read, and a queued
reply only stays silent when a finished reply already read everything.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 04:49:10 +01:00
tbenandClaude Opus 5.5 c13b9ea826 Show how each bot message was generated
Every bot message gets an info button that opens its details: the
provider and model behind it, tokens and cost, the job that ran it, each
model used and every call in order. Each model call now records how it
was made in token_usages.settings (generation settings sent, whose
settings they were, pinned or automatic model, fallbacks), shown per
model in the modal.

Live replies now carry their token usage, from the job's push and the
live relay alike, so it shows without a reload.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 04:33:47 +01:00
tbenandClaude Opus 5.5 cdb93ac1bc Fix file access, XSS and access-control bugs from a security scan
- Media rows can no longer be created through the CRUD API, updates only
  take description fields on the caller's own row, and every media read
  or delete goes through mediaFilePath, which refuses paths outside
  DATA_DIR or inside the database folder.
- Uploads strip the file name and extension to safe characters and keep
  only title, description and summary from the request's media object.
- Folder imports no longer follow symlinks (walkRegularFiles), and the
  source path is quoted in the computer's shell.
- User and bot files are served with nosniff, and HTML, SVG, XML and
  scripts are sandboxed by CSP.
- Sandbox cookies are kept to the computer's prefix, and websocket
  upgrades on /ws and the computer proxy must come from the app's own
  address (README and Help note nginx's Host header).
- user_types CRUD writes go through the assignment matrix; members only
  read their own user record; read-only users can read CRUD again;
  grabUsers builds valid SQL for array search terms and bad pages; an
  admin media DELETE needs an id; read-only users can't delete media.
- Rate limits and the audit log use the connection's address, trusting
  forwarded headers only from a proxy, and login limits are split per
  address, per account from that address and per account overall.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-25 03:38:07 +01:00
tben 4261535741 Tell every bot its own id, in every run
The bot's id never reached its prompt: `composeBotPrompt` only took its
instructions, the appendix, its memory and its workspace, and
`grabBotPrompt` never passed `bot.id` or `bot.name` on. A bot could only
learn its id by accident, from a tool description that named it, so
"which bot am I" was unanswerable in most contexts.

- Open the prompt with who the bot is (`composeIdentity`): its name,
  whether it's the coordinator or a worker, its role, and its id, said
  to be how tools and refs point back at it. `grabBotPrompt` now passes
  the bot, which covers chat replies, tasks, routines and delegated
  tasks, since both runners build their prompt there.
- Turn the notes' example refs into functions bound to the running bot:
  `chatReplyNote`, `botFilesNote` and `coordinatorReplyNote` take its id,
  so the folder-ref example no longer points at a hardcoded `owner_id: 5`
  (another bot's folder) nor `[bot:3]`. ComputerFilesNote keeps the
  computer it's given.
- Stop `create_task` from hiding the worker's own id in its own chat.

Tests: four new `composeBotPrompt` cases pin the identity block (worker,
coordinator, unnamed, no bot at all), and a new `harness-prompts.test.ts`
checks each note's own-folder and self refs point at exactly the running
bot, for three ids.
2026-10-23 17:40:28 +01:00
tben d9aea8e9c8 Record what every model call spends, and show it
Nothing anywhere recorded what a model call cost: the harness used a
provider's usage numbers only to decide whether a turn had finished, then
dropped them. So a reply could not say what it took to make, one model
could not be compared with another on price, and there was no way to ask
what a day, a week or a month of the crew's work came to.

- `token_usages` records one row per call: user, bot, workspace, session,
  job and message, the harness, the provider and model that actually
  answered, the prompt and completion counts, and the cost when the
  provider reports one.
- `harness-usage-scope.ts` carries those ids in an AsyncLocalStorage
  scope that `run-bot-job.ts` enters once, so a run's turns, its session
  summary, its transcriptions, images and embeddings are all attributed
  without threading ids through every harness function. Nested runs merge
  rather than replace, and a null genuinely clears a field (a bot's own
  chat has no workspace).
- Work with no run behind it is charged to its owner instead:
  `run-with-fallback-scope.ts` reaches the workspace's creator else the
  bot's, and deliberately does not override a real run, so an ingest a
  bot asked for is not misattributed to the bot's owner.
- All six harnesses record through `withCandidates`'s `on_success`.
  Recording never throws: a model that answered must not lose its answer
  to a failed insert.
- A job's calls land before its message exists, so `complete-bot-job.ts`
  back-fills `message_id` once the message is inserted.
- The message's actions row shows what it cost, beside the timestamp and
  the play button, with the breakdown in its title.
- `/admin/usage` totals the install's spending for 7, 30 or 90 days: the
  total against the previous period, a day-by-day chart, a split per kind
  of call, and a table per model with the price per thousand tokens
  worked out from cost and tokens, which is what makes two models
  comparable. Crew managers only, since it is the whole install's work
  rather than one user's conversation. It sits with Routines and Activity
  in the user popover rather than under Settings, because it is a record
  of what happened, not something to configure.

Days are grouped in local time (`date(created_at / 1000, 'unixepoch',
'localtime')`), matching the local midnights the window builds and the
`YYYY-MM-DD` buckets it looks up. Grouping by the bare UTC day put a late
evening's calls under a name no bucket carried, so they vanished and the
chart's total silently disagreed with the totals card; a test pins it.

`canSeeRailItem` now decides who sees a popover or rail entry. The two
each filtered with their own inline rule and only one understood
`can_view`, so a gated page could be offered in the popover to everyone.
2026-10-23 15:41:41 +01:00
tben 0048324d8a Let a delegated task choose the workspace it runs in
delegate_task had no workspace argument: a task handed over from the
coordinator's own chat always ran in the worker's own chat, on that
worker's memory, files and directly assigned computers alone, even when
the work belonged to a workspace. The coordinator could not say where the
work goes, and the prompt never said the two differ.

- delegate_task takes an optional workspace_id: the worker runs the task
  in that workspace's context, with its own instructions and memory
  there, the project's files and the workspace's computers, and the
  result lands in its conversation in that workspace.
- A worker missing from that workspace is added first (pushBotCreated)
  and the reply says so.
- Handed over from inside a workspace the task always runs there; a
  different workspace_id is refused instead of ignored.
- The coordinator prompt, CoordinatorReplyNote and the tool description
  now spell the distinction out.
2026-10-23 14:25:21 +01:00
tben f2802c1bf3 Let bots set a picture from a file they can see
A coordinator asked to set an uploaded logo as a workspace picture could
not: the picture tools take a media id, and nothing ever gave the bot one.
Attachments reached the model as "[Attached image \"logo.png\"]", every
file tool answers with a [file:ID] ref, and an SVG is deliberately stored
as a document rather than an image.

- copyMediaAsPicture copies an approved image into a new picture row
  (bot-image/workspace-image). The source keeps its row and bytes, so a
  message that attached it still loads and changing a picture can never
  take a file away from a conversation.
- grabPictureSource resolves a picture by [file:ID] (the same scope gate
  every file tool uses) or by the user's own upload; parseFileRef reads
  the [file:12] form.
- The new set_picture tool ships on every run: a bot sets its own face,
  the coordinator can also dress a worker. update_workspace gained
  picture_file_id, and setBotPicture/setProfileImage share the copy and
  the delete.
- removeReplacedPicture deletes a replaced picture only when it really is
  one, never a chat upload.
- uploadAndRecordMedia writes images in their own format: PNG stays PNG
  with its transparency, WebP stays WebP, GIF becomes PNG, everything
  else JPEG, and the row records the real mime type (it used to say jpeg
  for every image).
- Pictures change live: BotUpdatedData carries image_media_id and a new
  server:workspace-updated pushes workspace pictures, relayed between
  processes.
- Attachments are labelled with their media id, and BotFilesNote tells a
  bot that a [file:ID] can become a picture.
- grabRunScopes is now the single source of a run's reachable folders,
  shared by the file tools, imports and the picture tools.
2026-10-23 05:24:55 +01:00
tben d3357cb7df Stop storing raw file bytes in media.text_content
`text_content` is an extracted-text cache, but nothing stopped it from
holding a dump of bytes:

- `fileToText` ended its switch with `default: return buffer.toString("utf-8")`
  and defaulted a missing extension to "txt", so any binary was decoded.
- Both extraction callers gated on `media_type == "document"`, and
  `attachmentFileKind` classes everything non-image/audio/video as a
  document: an SVG, a zip, a spreadsheet, a .bin.
- Nothing capped the length, so a 20 MB attachment could leave 20 MB of
  text in the database.
- `storeChatAttachments` copied the text onto a second `media` row, and
  `SELECT m.*` pulled the column into page props and API responses.

On a real install an SVG held 6852 characters of markup and a
`.designcraft` file held 142 KB of raw bytes.

Changes:

- `file-to-text.ts` lists the text extensions explicitly and returns
  undefined for anything else; no extension is no text.
- `can-extract-text.ts` is the one allowlist (reusing
  `BotTextFileExtensions`), used by both upload routes and by
  `uploadAndRecordMedia`; `extension-of.ts` is shared with the storage
  catalog instead of duplicated.
- `BotAttachmentTextStoreMaxChars` (200k) caps the column via
  `cap-text-content.ts`.
- `store-chat-attachments.ts` no longer copies the text; ingestion
  re-reads the bytes once.
- `media-columns.ts` selects every media column but `text_content`, which
  now has to be asked for (`grab-messages-media`'s reply path and
  ingestion do); the media CRUD routes sanitize rows they write.
- `ingest-bot-file.ts` decides on size alone, since a cached slice made
  an oversized file look already read.
- `bun run media:prune-text` cleans rows that were already stored
  (rerunnable, with a VACUUM hint).

Verified on the dev database: both bad rows cleared (146 KB), rerun
reports nothing to do.
2026-10-23 05:22:04 +01:00
tben a5eb2b6a95 Let the coordinator set a workspace's picture
CI / check (push) Canceled after 0s
update_workspace now takes an image media id (or null to remove the
picture). updateWorkspace writes image_media_id through
resolveWorkspacePicture, which only accepts the caller's own image and
deletes the picture it replaced, so a bot can never show someone else's
file or drop a picture it is still using.
2026-10-22 21:10:06 +01:00
tben 2398314703 Give the coordinator a fuller prompt, and show bot instructions in find_bot
The coordinator's default prompt was a page of job descriptions. It now
states the working rules it kept getting wrong: a queued task is not a
running one, a tool result is the only proof an action happened, an
approved text is written whole, and an explicit constraint such as "no
computer" is binding. The prompt is attached at creation only, so this
reaches new installs and a coordinator whose prompt is still empty.

find_bot now returns each match's instructions, so the coordinator can
judge a worker by what it was told rather than by its name.

(The previous prompt is kept as a commented-out block at the bottom of the
file.)
2026-10-22 20:11:03 +01:00
tben 08d02e6033 Give bots skills: runbooks they load when a job matches
A skill is a runbook: a name, a unique slug, a one-line description of when
it applies, and the markdown body. A run only ever sees the slug and
description (one load_skill tool carries them), and reads the whole body
only when it calls the tool, so a long runbook costs nothing until needed.

Skills are the third extension kind beside MCP servers and custom tools, on
the same botExtensionsSql path, so per-bot, per-workspace and all-bots
grants work the same way, and attachDefaultExtensions hands an every-bot
skill to new bots. load_skill is a reserved tool name, so a custom tool
cannot shadow it, and a built-in wins a clash.

Managed on Settings > Skills; ExtensionRow links each kind to its own page
and a skill never reports a failure, since nothing connects or runs.
2026-10-22 12:31:42 +01:00
tben 1a6249afd3 Let a bot take any colour
A bot's identity was one of seven hues, and the hue picked both its colour
and its stone shape. Now its colour is free and its stone is its own:

- `bots.color` holds any `#rrggbb`, `bots.stone` the flat-cut shape it's
  drawn on. `bots.hue` stays as the palette column they replaced, read only
  as a fallback; `src/scripts/move-bot-colour-to-owners.ts` backfills an
  install from it (safe to rerun).
- `src/utils/bot-color.ts` is the only place a colour is validated or
  resolved (`resolveBotColor`/`resolveBotStone` carry the old hue forward,
  `botColorNeedsPaperFace` flips the eyes to paper on a very dark body), and
  `src/utils/bot-stone-props.ts` gives every row its props, so a row saved
  before this still renders as it did.
- `BotStone` takes the colour as the body's SVG `fill`, the one documented
  exception to the token rule: an arbitrary hex has no token to point at.
  `UnreadDot` follows it.
- The picker keeps the palette tiles (still shape and colour together) and
  adds a native swatch and a hex field, so a custom colour keeps its stone.
  Same place as before: the header avatar's modal and Settings > Bots, for
  crew managers. The coordinator stays hue-less.
- A recolour reaches other open tabs through `pushBotUpdated`.
2026-10-22 11:55:36 +01:00
tben aebd6b8ccc Serve every media file privately at /media/<id>
There is no public media any more. `/media/public/<id>` served a media
row with no session at all, so anyone with the id could read a file.
The public/private split is gone: the path is `/media/<id>` (and
`/media/<id>/thumbnail`), every request is signed in first, and
`grabUserMedia` decides whether the user may see it: their own upload,
a bot's or workspace's picture or file they reach, or a file on a task
they reach. Anything else is a 404 before a path is built.

The `media.is_private` column and the two per-user directories went
with it: uploads live in `media/<user_id>/` and there is nothing to
pick between. The files browser's media volume is now the viewer's
own folder whatever their role.

Bots also say which of the two things they mean. A `[file:ID]` ref is a
saved file and its chip opens the bytes; a folder is a new
`[folder:volume:owner:path]` ref (`src/utils/folder-ref.ts`) whose chip
opens the files browser there and downloads nothing. The old `[path:...]`
ref and the extension sniffing that guessed file from folder are gone,
so `import_file` names the folder holding a file it copied rather than
the file itself.

The prompts say what the Assistant got wrong: list before importing or
saving, one copy per artifact, never split a file into parts unless
asked, never claim the store is empty without a fresh list, and never
invent a size cap. A re-import now says the path already existed
instead of silently overwriting and reporting a fresh copy.
2026-10-22 10:10:53 +01:00
tben 96735de48f Let a bot and a workspace set their own envs
Envs had three scopes, general, workspace and computer. A bot had no
scope of its own, so the only way to give one worker a key was to set it
on a computer it happened to share.

The envs table gets a bot_id, and resolution now layers bot over
computer over workspace over general, so a bot's own key wins wherever it
runs. Every computer call site passes the run's bot_id, and grabRunEnvs
does the same for the {{env.KEY}} a custom tool or an MCP server's headers
read.

A bot's and a workspace's settings page can now edit their own envs, in
place, through the same save-env and delete-env routes: sending `scoped`
makes them answer with just that scope instead of the whole install's
overview. Super admins only, like the Environment page.
2026-10-22 08:30:40 +01:00
tben f1b7ba535f Stream computer files instead of reading them through the shell
import_file sent a computer's bytes back with base64 over /v1/shell/exec,
whose stdout is capped at 30,047 characters. Every import was therefore
cut at 22,528 bytes and saved anyway, with the tool reporting success: a
1.12 MiB archive became a 22 KB file that would not open, and a crawler's
crawl.json lost half its pages.

The bytes now come from the sandbox's streaming GET /v1/file/download.
The size is read first so nothing over BotFileImportMaxBytes is
transferred, the download asks for change_policy=abort so a file that
changes mid-send is refused rather than half-saved, and the received
length is checked against the size read, so a cut-short transfer fails
loudly. Folders go the same way, as a gzipped tar unpacked into one row
per file.

Bots also name a path as [path:volume:owner:path] now, so what a bot
leaves on a computer is a link into the files browser instead of text.
2026-10-22 07:37:19 +01:00
tben 7cb26c4e36 Add tasks: the crew's shared to-do list
Bots and workspaces each keep a list of work tracked until it's done, and
the bots move it along themselves as they work.

- tasks (one collection per bot or per workspace), task_notes (the
  progress log) and task_media (a deliverable or a reference file, its
  bytes a private media row served at /media/private/<id>)
- statuses todo/doing/blocked/done/cancelled, with blocked_reason while
  only the user can unblock it
- visibility as SQL in src/utils/task-access: a user's own bots and
  workspaces, a run's own list plus its workspaces, everything for the
  coordinator; every tool and endpoint resolves through it
- list_tasks, create_task and update_task on every run, so a chat reply,
  a task and a routine all read the list, log progress and file what they
  produced
- TasksNote on every run and TaskReminderNote in chat, so a greeting or a
  proposed next step brings pending work up in one measured line
- tasks group on the workspace dashboard and a tasks modal in the
  conversation header, sharing one form, row and detail panel
- a bot's or workspace's own chat uploads are read before the delete so
  none are stranded; bot files keep their folder

Also corrects the README's RAM guidance and table alignment.
2026-10-22 05:57:50 +01:00
tben 8edff8988a Run models on sensible generation defaults out of the box
A fresh install had no generation settings until someone set one, so every
model ran on whatever its provider happened to use. Each harness now has a
default (text 0.3 temperature, vision 0.2, image the provider's own), shown
in the model settings modal so what a model runs on is never a guess.

The defaults are only sent to a model whose catalog lists the field, so a
provider that says nothing about its models still gets no parameter it
might reject; anything saved on a harness or bot wins over the default.
2026-10-22 04:56:10 +01:00
tben c2553a4746 Add per-model generation settings to the model selector
Each text and vision harness setting now carries the generation settings
its model takes (temperature, sampling limits, penalties, seed, reasoning
effort), edited in a modal under the model pick so the harness block stays
short. Nothing is set by default, so models keep their provider's own
values until a super admin or admin changes them.

A setting is only offered and only sent when the catalog says the model
takes it: OpenRouter's supported_parameters is now stored on
harness_models and refreshed by the hourly sync. Other providers list no
parameters, so their models get them all.
2026-10-22 04:52:55 +01:00
tben d17a4a7bdd Add bots files handling logic 2026-10-21 17:56:09 +01:00
tben 9778d0b9ff Keep a bot's workspace memory from reading as the workspace's record
A workspace bot asked to "update the workspace record" wrote to its own
memory there, called it the workspace's record, and then, when the user
said nothing had changed, denied making any write at all: it checked
nothing and confessed to work it had done.

- Say in the prompt that the workspace's context is the user's, and that
  memory in a workspace is the bot's own notes, readable in its settings
- Say the same in update_memory and update_instructions results, with
  where the user reads it and that the workspace's own fields didn't
  change
- Have ActionCheckNote look at what actually happened before agreeing
  with a user that nothing was done, and never deny work that happened
2026-10-20 14:59:56 +01:00
tben 3b355d68bc Keep bot jobs and harness calls from hanging 2026-10-20 12:59:24 +01:00
tbenandClaude Opus 5.5 535adb6e16 Make routines act on their own and check every bot action
Scheduled runs are pre-approved: they carry out their brief instead of
drafting for confirmation. A run that can't do its brief calls
report_run_failed and is retried 5 and 10 minutes later; only then does
the user get one "failed after N tries" message, and the routine keeps
why in last_error.

Every reply and task now gets ActionCheckNote: look for an existing
record before creating one, and verify an action took effect before
saying it's done. Memory and instructions saves are read back, and
create_bot, create_schedule, start_task and delegate_task refuse
duplicates.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-20 08:45:49 +01:00
tben e2093b0df6 Update frontend UI/UX 2026-10-20 04:59:08 +01:00
tbenandClaude Opus 5.5 d1b5b6d68a Add replies and emoji reactions to chat messages
Every message has a row under it to reply or react. Replying quotes the
message above the composer, and the sent message stores
reply_to_message_id (a plain id, not a foreign key, so deleting the
quoted message only drops the quote). send-message accepts a quote from
any session of the same conversation, so a rollover or a thread that
moved on still sends; the conversation drops a picked quote when its
session changes. The bot reads the quote as "[Replying to ...]" before
the user's turn, file-only replies included, and relayed messages carry
it too.

Reactions are a fixed set (BotMessageReactions) in
bot_message_reactions, one per message, emoji and user or bot. Users
toggle their own through api/admin/bots/react-message, and every bot
gets react_to_message for the session it answers (the user's latest
message by default). Both push the fresh list as server:bot-reaction,
with mine worked out for the session's user.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 21:56:31 +01:00
tbenandClaude Opus 5.5 bc499a7466 Let admins hear a voice sample before picking it
The Speech section of every model settings form now has "Hear a
sample", which streams a short sentence (AppData.VoicePreviewText) in
the form's model and voice before they're saved, through
GET api/admin/harness/preview-voice (super admins and admins, that
model only via grabModelCandidate). pickVoice is shared with
speakText, and explainAudioError reads why a streamed player failed
for both read-aloud and the sample.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 21:32:49 +01:00
tbenandClaude Opus 5.5 163d9a434c Add dictation and streamed read-aloud to the chat
- A mic in the composer records, transcribes with the bot's
  transcription model (api/admin/bots/transcribe) and writes the text
  into the input at the cursor to edit before sending. Escape cancels;
  recordings aren't stored.
- Every bot message has a read-aloud button that plays
  GET api/admin/bots/speak-message, streamed from the provider as the
  audio is made, so it starts in a couple of seconds. One message plays
  at a time; only the user's own sessions can be read.
- Both buttons stay visible but disabled while the bot has no model for
  them (bot_voice page prop).
- Transcription and speech no longer pick free models automatically:
  they're not available until a model is picked, and Settings says so.
  Transcription lists speech-to-text models only, so the chat-model
  transcript path and its missing-audio check are gone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 21:21:04 +01:00
tben 139b3e443d Give the coordinator working crew management tools
- Resolve the crew and workspaces fresh inside each tool run, so a bot or
  workspace created earlier in the same reply is usable: create_bot,
  assign_computer, add_bot_to_workspace, delegate_task and
  set_bot_workspace_instructions.
- Add coordinator-only update_bot, find_bot, find_workspace, delete_bot,
  delete_workspace and remove_bot_from_workspace tools.
- Push server:workspace-removed when a workspace is deleted, and
  workspace_ids on server:bot-updated so a workspace's crew drops a bot
  that was taken out of it.
2026-10-19 20:34:51 +01:00
tben b0c9c84b2b Give workspace computers to attached bots in their own chat
A computer linked to a workspace now shows in the composer (and is usable
by tasks and replies) when chatting with any bot attached to that
workspace, not only in the workspace conversation. Members see those
computers through their assigned workers too, and assign_computer still
describes direct grants only.
2026-10-19 20:03:38 +01:00
tbenandClaude Opus 5.5 6294b911e6 Give bots tools and MCP servers per bot and workspace, with status and tests
- Bots and workspaces get tools and MCP servers from their settings
  pages; bots also from chips under the chat composer, removable there
- "Every bot" now gives one to every bot now and every new bot; each can
  still drop it (attach-every-bot-extensions.ts moves old installs over)
- Status lines: MCP servers record whether they connect, tools whether
  their last run worked
- Test connection in the MCP form and a Test panel in the tool form check
  the form as it is, without saving
- Hide Google sign-in on the login form

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 15:35:13 +01:00
tbenandClaude Opus 5.5 c6ec7d77a2 Add MCP servers and custom tools for bots
Two new settings pages, MCPs and Tools. Super admins define them; super
admins and admins attach them to bots, to workspaces (every bot there),
or to every bot. Bots get them in chat replies and tasks.

- Custom tools: a JSON-schema of arguments plus an HTTP request, a script
  on the bot's computer, or a prompt, templated with {{args.x}} and
  {{env.KEY}}, secrets redacted from results.
- MCP servers: remote (URL and headers) or stdio commands run on the host
  with a setup command and an isolated env, in a per-process pool. Tools
  are listed into a cache and connect on first call.
- Add, edit, access and try open in modals.
- Markdown, JSON and command fields use twui's AceEditor (CodeEditor),
  its scripts loaded on demand.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 15:01:39 +01:00
tben 550aaeabac Let the coordinator create workspaces and write findings in tasks
create_workspace gives the coordinator a way to start a workspace with its shared context, and the tool list is shared, so a new id can be used by create_bot or add_bot_to_workspace in the same reply.

A coordinator's tasks now get create_workspace and update_workspace too: the research task that had the material could only report it, never write it into the workspace, so the coordinator promised an update it had no turn to make.

A workspace created from chat is pushed as server:workspace-created, from the tool and from the live relay, so the rail shows it without a reload.
2026-10-19 14:32:47 +01:00
tben fccaa6e280 Show a newly created bot in the rail at once
The rail was built from page props and only knew bots it already had, so a
bot the coordinator created did not appear until a reload.

create_bot and add_bot_to_workspace now push server:bot-created
(BotCreatedData) to everyone who sees the bot: its rail row, the
workspaces it is in, and whether it belongs in that user's own chat list.
useLiveBots inserts it, after the pinned coordinator outside a workspace
and only when the open workspace is one it joined. liveRelay pushes created
bots too, so a bot added by another process still shows.
2026-10-19 12:17:56 +01:00
tben 981a180d19 Let the coordinator create bots and put them in workspaces
The coordinator had no tool to add a worker, so a request to create one
fell back to update_workspace and wrote a bot that did not exist into the
workspace context. create_bot now adds a worker with its name, role and
instructions and joins it to a workspace, and add_bot_to_workspace puts an
existing worker in one. Both are coordinator-only and chat-replies-only,
and createBot still enforces the crew-manager check.

The coordinator's prompts now say how a bot gets created, and that one
which does not exist is never written into workspace context.

pick-free-hue moves to utils so the tool can pick a hue without importing
from layouts.
2026-10-19 12:11:04 +01:00
tbenandClaude Opus 5.5 e45d54d30b Share one crew across the install, keep user memory per user
Bots, workspaces and computers now belong to the install. Users reach
workers only through bot_users (assigned by a super admin or admin) and
workspace_users; the coordinator and main computer are one per install,
and only the super admin sees the coordinator. Conversations are per
user (bot_sessions.user_id), jobs run for the session's user, and live
updates go only to who should see them.

A bot's core memory is now general knowledge shared by every user it
works for; what it learns about a user lives in bot_user_memories.

One-off scripts move existing installs over: move-bot-memory-to-owners
(run first), then share-crew.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 11:37:14 +01:00
tbenandClaude Opus 5.5 c44048dbab Add bot schedules (routines), open Activity, add Help and a Desktop computer view
Bots create, list, update and delete schedules (cron_jobs, Bun.cron in UTC);
a per-minute cron queues due runs as tasks. Routines, Activity (super admins)
and Help get their own rail aside. Help covers familiar problems, starting
with signing in to Google through Cookie-Editor over the new Desktop (noVNC)
view.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 10:52:32 +01:00
tbenandClaude Opus 5.5 f24c0d7e82 Run computers with resources and health checks, add settings for bots, workspaces, computers and users
- Computers get CPUs and memory (default 1 CPU, 2 GB), set at creation and
  editable; new resources stop, remove and recreate the container, its home
  dir kept. Every computer container has a docker health check and stays
  "creating" until it passes; only healthy computers are reached, by
  hostname, else by their IP on the network (dev outside docker).
- Docker is required: the app creates the cobble network on startup and
  exits without docker.
- Production container: deploy/run-app-container.sh mounts .env, the data
  dir and the docker socket; the database (DB_DIR), media, computers' homes
  and bun-sqlite's live schema now live on the data volume.
- Settings pages for bots, workspaces, computers and users, sharing list
  components; bots can be deleted; users are created with a password and
  confirmation, edited and deleted through RBAC-gated APIs.
- User types trimmed to super_admin, admin, member, read_only and revoked;
  admins manage members. UserAvatar shows profile photos from the user row.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 10:26:44 +01:00
tbenandClaude Opus 5.5 0ce61cdb49 Add workspaces, conversation sessions, app-managed computers and pictures
- Rail: Add icon with stacked form modals (bot, workspace, computer),
  coordinator, recent bots with "See all bots", Workspaces section, and
  a workspace mode listing only its crew
- Workspaces: page for context, bots (per-workspace instructions and
  memory), computers (default, link, unlink), models and delete;
  conversations scoped per bot per workspace
- Sessions: conversations split into searchable, reopenable sessions
  with carry-over summaries; sessions and new-session header icons
- Computers: created and provisioned over the docker socket, home dir
  kept at DATA_DIR/computers/<id> after delete; super admins only
- Models: workspace settings win over a bot's own, then the defaults
- Pictures for bots and workspaces; bots editable from the header
- Coordinator tools: update_workspace, set_bot_workspace_instructions

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 08:39:33 +01:00
tbenandClaude Opus 5.5 56daf0c462 Open computers from the chat, let bots ask for a take-over, show task progress
- Serve each computer to its owner under /computers/<id>/ (HTTP and
  websockets proxied to the AIO sandbox) and show it in a shared
  ComputerView with browser, terminal and code modes, opened from the
  computers modal or a take-over button in the thread.
- ask_user_to_take_over lets a bot hand a computer to the user for a
  login or captcha; the message gets a take-over button and needs-you,
  and the viewer hands it back with a chat message.
- While a task runs the thread shows "<bot> is working..." with the
  task's latest step; steps read as plain words, not tool names.
- A coordinator with no workers no longer claims to delegate, and bots
  describe started tasks in the first person.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 05:35:44 +01:00
tbenandClaude Opus 5.5 7b580d782e Add computer switching, crew reporting, job steps and markdown chat
- Bots have a current computer (bots.computer_id), switched from the chat
  header beside the model button or by clicking a computer the bot names
  ([computer:ID]); tasks run on it by default
- search_computers runs one read-only command on all of a bot's computers;
  chat replies know their computers (compose-computers-note)
- The coordinator can use every computer and grants or revokes crew access
  with assign_computer
- Delegated jobs record requested_by_bot_id; the worker's result or failure
  is reported back in the coordinator's conversation
- bots_queue_updates records each model turn of a job, pushed live and shown
  in the working-on panel
- Live relay: every process pushes new messages, job steps and bot changes
  from the database to its own sockets, so work done elsewhere shows live
- Bot messages render as markdown (react-markdown + remark-gfm, no raw HTML)
  with twui components; rail previews are plain text
- Fix header button overflow in the model and computer selects
- Bump @moduletrace/twui to 0.1.2

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 04:44:35 +01:00
tbenandClaude Opus 5.5 6a2665dae9 Add the harness, envs, per-bot models and chat attachments
- Harness: providers, model catalog synced hourly, automatic free picks,
  cooldowns, tool loop with context trimming, MCP computer tools
- Bots queue runs replies and tasks through the harness; jobs wait
  (attempt refunded) while every model is resting
- Envs: Settings > Environment, scoped general/workspace/computer,
  written to computers and redacted from tool output
- Models: Settings > Models, per-harness provider select defaulting to
  OpenRouter, and a models modal on each bot's chat (the coordinator's
  edits the defaults)
- Chat: thinking row while a reply runs, file attachments (upload,
  paste, drop) that replies read as images, audio and document text
- Fix media uploads failing after a user's first file, and private
  files being served from the public media path

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 03:38:38 +01:00