A ref quoted alone in a code span now links like any ref, and refs
inside longer code stay literal instead of turning into broken link
text. A file chip opens a file from another bot's folder for a user
who reaches that folder, by the same rule as its bytes, so the
asked bot's files open from the asking bot's conversation. Bots are
told to keep refs out of backticks, and the coordinator to pass them
on as given.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Knowledgebases give bots RAG over documents, scoped like extensions: to
particular bots, to a workspace, or to every bot, with their own button
beside Tools in the composer. Documents are chunked and embedded by the
file-ingest worker with the install's embedding model, which is now an
install-wide setting rather than a per-bot one.
- Word search runs on an FTS5 index ranked by BM25, with a scored LIKE
fallback for scripts written without spaces.
- Changing the embedding model re-indexes documents automatically.
- Ingest jobs wait without spending attempts while every embedding model
rests, fail at once on errors retrying can't fix, and keep passages
searchable by words when the model refuses.
- Settings > Knowledgebases shows each knowledgebase's and document's
index state: spinner, warning, grey check (words only) or green check.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
- Media rows can no longer be created through the CRUD API, updates only
take description fields on the caller's own row, and every media read
or delete goes through mediaFilePath, which refuses paths outside
DATA_DIR or inside the database folder.
- Uploads strip the file name and extension to safe characters and keep
only title, description and summary from the request's media object.
- Folder imports no longer follow symlinks (walkRegularFiles), and the
source path is quoted in the computer's shell.
- User and bot files are served with nosniff, and HTML, SVG, XML and
scripts are sandboxed by CSP.
- Sandbox cookies are kept to the computer's prefix, and websocket
upgrades on /ws and the computer proxy must come from the app's own
address (README and Help note nginx's Host header).
- user_types CRUD writes go through the assignment matrix; members only
read their own user record; read-only users can read CRUD again;
grabUsers builds valid SQL for array search terms and bad pages; an
admin media DELETE needs an id; read-only users can't delete media.
- Rate limits and the audit log use the connection's address, trusting
forwarded headers only from a proxy, and login limits are split per
address, per account from that address and per account overall.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Nothing anywhere recorded what a model call cost: the harness used a
provider's usage numbers only to decide whether a turn had finished, then
dropped them. So a reply could not say what it took to make, one model
could not be compared with another on price, and there was no way to ask
what a day, a week or a month of the crew's work came to.
- `token_usages` records one row per call: user, bot, workspace, session,
job and message, the harness, the provider and model that actually
answered, the prompt and completion counts, and the cost when the
provider reports one.
- `harness-usage-scope.ts` carries those ids in an AsyncLocalStorage
scope that `run-bot-job.ts` enters once, so a run's turns, its session
summary, its transcriptions, images and embeddings are all attributed
without threading ids through every harness function. Nested runs merge
rather than replace, and a null genuinely clears a field (a bot's own
chat has no workspace).
- Work with no run behind it is charged to its owner instead:
`run-with-fallback-scope.ts` reaches the workspace's creator else the
bot's, and deliberately does not override a real run, so an ingest a
bot asked for is not misattributed to the bot's owner.
- All six harnesses record through `withCandidates`'s `on_success`.
Recording never throws: a model that answered must not lose its answer
to a failed insert.
- A job's calls land before its message exists, so `complete-bot-job.ts`
back-fills `message_id` once the message is inserted.
- The message's actions row shows what it cost, beside the timestamp and
the play button, with the breakdown in its title.
- `/admin/usage` totals the install's spending for 7, 30 or 90 days: the
total against the previous period, a day-by-day chart, a split per kind
of call, and a table per model with the price per thousand tokens
worked out from cost and tokens, which is what makes two models
comparable. Crew managers only, since it is the whole install's work
rather than one user's conversation. It sits with Routines and Activity
in the user popover rather than under Settings, because it is a record
of what happened, not something to configure.
Days are grouped in local time (`date(created_at / 1000, 'unixepoch',
'localtime')`), matching the local midnights the window builds and the
`YYYY-MM-DD` buckets it looks up. Grouping by the bare UTC day put a late
evening's calls under a name no bucket carried, so they vanished and the
chart's total silently disagreed with the totals card; a test pins it.
`canSeeRailItem` now decides who sees a popover or rail entry. The two
each filtered with their own inline rule and only one understood
`can_view`, so a gated page could be offered in the popover to everyone.
A profile photo, a bot's picture and a workspace's picture all come from
one condition now (sharedPictureMediaSql), so a face is never blank just
because its owner isn't in the viewer's rail. Anything that isn't a
picture stays behind its own check.
A coordinator asked to set an uploaded logo as a workspace picture could
not: the picture tools take a media id, and nothing ever gave the bot one.
Attachments reached the model as "[Attached image \"logo.png\"]", every
file tool answers with a [file:ID] ref, and an SVG is deliberately stored
as a document rather than an image.
- copyMediaAsPicture copies an approved image into a new picture row
(bot-image/workspace-image). The source keeps its row and bytes, so a
message that attached it still loads and changing a picture can never
take a file away from a conversation.
- grabPictureSource resolves a picture by [file:ID] (the same scope gate
every file tool uses) or by the user's own upload; parseFileRef reads
the [file:12] form.
- The new set_picture tool ships on every run: a bot sets its own face,
the coordinator can also dress a worker. update_workspace gained
picture_file_id, and setBotPicture/setProfileImage share the copy and
the delete.
- removeReplacedPicture deletes a replaced picture only when it really is
one, never a chat upload.
- uploadAndRecordMedia writes images in their own format: PNG stays PNG
with its transparency, WebP stays WebP, GIF becomes PNG, everything
else JPEG, and the row records the real mime type (it used to say jpeg
for every image).
- Pictures change live: BotUpdatedData carries image_media_id and a new
server:workspace-updated pushes workspace pictures, relayed between
processes.
- Attachments are labelled with their media id, and BotFilesNote tells a
bot that a [file:ID] can become a picture.
- grabRunScopes is now the single source of a run's reachable folders,
shared by the file tools, imports and the picture tools.
`text_content` is an extracted-text cache, but nothing stopped it from
holding a dump of bytes:
- `fileToText` ended its switch with `default: return buffer.toString("utf-8")`
and defaulted a missing extension to "txt", so any binary was decoded.
- Both extraction callers gated on `media_type == "document"`, and
`attachmentFileKind` classes everything non-image/audio/video as a
document: an SVG, a zip, a spreadsheet, a .bin.
- Nothing capped the length, so a 20 MB attachment could leave 20 MB of
text in the database.
- `storeChatAttachments` copied the text onto a second `media` row, and
`SELECT m.*` pulled the column into page props and API responses.
On a real install an SVG held 6852 characters of markup and a
`.designcraft` file held 142 KB of raw bytes.
Changes:
- `file-to-text.ts` lists the text extensions explicitly and returns
undefined for anything else; no extension is no text.
- `can-extract-text.ts` is the one allowlist (reusing
`BotTextFileExtensions`), used by both upload routes and by
`uploadAndRecordMedia`; `extension-of.ts` is shared with the storage
catalog instead of duplicated.
- `BotAttachmentTextStoreMaxChars` (200k) caps the column via
`cap-text-content.ts`.
- `store-chat-attachments.ts` no longer copies the text; ingestion
re-reads the bytes once.
- `media-columns.ts` selects every media column but `text_content`, which
now has to be asked for (`grab-messages-media`'s reply path and
ingestion do); the media CRUD routes sanitize rows they write.
- `ingest-bot-file.ts` decides on size alone, since a cached slice made
an oversized file look already read.
- `bun run media:prune-text` cleans rows that were already stored
(rerunnable, with a VACUUM hint).
Verified on the dev database: both bad rows cleared (146 KB), rerun
reports nothing to do.
There is no public media any more. `/media/public/<id>` served a media
row with no session at all, so anyone with the id could read a file.
The public/private split is gone: the path is `/media/<id>` (and
`/media/<id>/thumbnail`), every request is signed in first, and
`grabUserMedia` decides whether the user may see it: their own upload,
a bot's or workspace's picture or file they reach, or a file on a task
they reach. Anything else is a 404 before a path is built.
The `media.is_private` column and the two per-user directories went
with it: uploads live in `media/<user_id>/` and there is nothing to
pick between. The files browser's media volume is now the viewer's
own folder whatever their role.
Bots also say which of the two things they mean. A `[file:ID]` ref is a
saved file and its chip opens the bytes; a folder is a new
`[folder:volume:owner:path]` ref (`src/utils/folder-ref.ts`) whose chip
opens the files browser there and downloads nothing. The old `[path:...]`
ref and the extension sniffing that guessed file from folder are gone,
so `import_file` names the folder holding a file it copied rather than
the file itself.
The prompts say what the Assistant got wrong: list before importing or
saving, one copy per artifact, never split a file into parts unless
asked, never claim the store is empty without a fresh list, and never
invent a size cap. A re-import now says the path already existed
instead of silently overwriting and reporting a fresh copy.
ON DELETE CASCADE only fires when PRAGMA foreign_keys is on, and only
src/server.ts turns it on. A script or CLI process deleting a bot file
therefore left every bot_file_chunks row behind, and the next
`bun run db:schema` failed with
Foreign key violations after schema sync: bot_file_chunks -> bot_files
deleteBotFile and purgeStorage now delete the chunks themselves, so the
cascade is no longer load-bearing and a maintenance script behaves like
the server.
grab-conversation-files asked listBotFiles for the top level only, and a
bot saves into folders (reports/, seo/coderank/, tmp/...). The page
therefore carried none of the files a reply named, so every [file:ID]
chip rendered as dead plain text and the files could not be opened at
all: the bot had done the work and the user had no way to reach it.
The props now take the whole tree, and a chip that still isn't among
them (a message older than the page's first page of files) fetches its
one file by id, cached per id so a deleted one doesn't refetch on every
render.
tbenandClaude Opus 5.5