Serve every media file privately at /media/<id>

There is no public media any more. `/media/public/<id>` served a media
row with no session at all, so anyone with the id could read a file.
The public/private split is gone: the path is `/media/<id>` (and
`/media/<id>/thumbnail`), every request is signed in first, and
`grabUserMedia` decides whether the user may see it: their own upload,
a bot's or workspace's picture or file they reach, or a file on a task
they reach. Anything else is a 404 before a path is built.

The `media.is_private` column and the two per-user directories went
with it: uploads live in `media/<user_id>/` and there is nothing to
pick between. The files browser's media volume is now the viewer's
own folder whatever their role.

Bots also say which of the two things they mean. A `[file:ID]` ref is a
saved file and its chip opens the bytes; a folder is a new
`[folder:volume:owner:path]` ref (`src/utils/folder-ref.ts`) whose chip
opens the files browser there and downloads nothing. The old `[path:...]`
ref and the extension sniffing that guessed file from folder are gone,
so `import_file` names the folder holding a file it copied rather than
the file itself.

The prompts say what the Assistant got wrong: list before importing or
saving, one copy per artifact, never split a file into parts unless
asked, never claim the store is empty without a fresh list, and never
invent a size cap. A re-import now says the path already existed
instead of silently overwriting and reporting a fresh copy.
This commit is contained in:
2026-10-22 10:10:53 +01:00
parent ffd32c8b27
commit aebd6b8ccc
37 changed files with 466 additions and 399 deletions
+5 -5
View File
@@ -118,7 +118,7 @@ Inspect live data with `bun llm-code-assistant/query-db.ts "<sql>"` - a read-onl
- **Take-over:** a bot asks the user to take over a computer with `ask_user_to_take_over` (`src/harness/tools/computer/ask-to-take-over-tool.ts`, chat replies and tasks, defaulting to the task's computer). It writes `[take-over:ID:mode]` (`src/utils/take-over-ref.ts`; `withTakeOverRefs` adds it if the bot left it out and flags the message needs-you), linked by `link-computer-refs.ts` and rendered by `(partials)/take-over-chip.tsx`. The viewer then offers "Hand back to <bot>", which sends a chat message so the bot carries on; a task stops and reports instead of waiting. - **Take-over:** a bot asks the user to take over a computer with `ask_user_to_take_over` (`src/harness/tools/computer/ask-to-take-over-tool.ts`, chat replies and tasks, defaulting to the task's computer). It writes `[take-over:ID:mode]` (`src/utils/take-over-ref.ts`; `withTakeOverRefs` adds it if the bot left it out and flags the message needs-you), linked by `link-computer-refs.ts` and rendered by `(partials)/take-over-chip.tsx`. The viewer then offers "Hand back to <bot>", which sends a chat message so the bot carries on; a task stops and reports instead of waiting.
- **Health:** `containers.status` is docker's state with `running` narrowed by the health check (`starting`, `unhealthy`; `inspectDockerContainer`). Only a `running` computer is reached: `grabComputerMcpUrl` returns `ready: false` otherwise (re-reading a stale state from docker first, `refreshComputerContainer`, which `sync-computers` uses too), and callers say why (`computerNotReadyMessage`). - **Health:** `containers.status` is docker's state with `running` narrowed by the health check (`starting`, `unhealthy`; `inspectDockerContainer`). Only a `running` computer is reached: `grabComputerMcpUrl` returns `ready: false` otherwise (re-reading a stale state from docker first, `refreshComputerContainer`, which `sync-computers` uses too), and callers say why (`computerNotReadyMessage`).
- **Schedules (routines in the UI):** `cron_jobs` (per user and bot, `workspace_id` null for its own chat, `title`, `instructions` (the brief), `schedule` (a `Bun.cron` expression, UTC), `computer_id` (not a foreign key; null runs on the bot's current one), `enabled`, `max_runs` (1 for a one-off), `run_count`, `next_run_at`, `last_run_at`, `last_job_id`, `last_error`). Helpers in `src/functions/backend/db/cron-jobs/` (`create-cron-job`, `update-cron-job`, `delete-cron-job`, `grab-cron-jobs`, `grab-user-cron-job`, `fire-cron-job`), all ownership-checked, and with `bot_id` limited to that bot's own. `nextCronRun` (`src/utils/next-cron-run.ts`, `Bun.cron.parse`) validates a schedule and refuses one that runs more often than `AppData.CronJobMinIntervalMs`; a bot has at most `AppData.CronJobMaxPerBot`. Every bot gets `create_schedule`, `list_schedules`, `update_schedule`, `delete_schedule` (`grabBotTools`, every run; descriptions via `describe-cron-job.ts`). The `run-cron-jobs` cron fires due ones (`fireCronJob`: claims the run atomically by moving `next_run_at`, skips it while the last run's job is still open, queues the brief as a task with `enqueueBotJob` in the bot's active session there, turns off after `max_runs` or when there's no next run, keeps why it couldn't in `last_error`, logs each run under `cron`). A run missed while the server was down runs once, then carries on from now. A run acts on its own (`ScheduledRunNote`: the brief is already approved, so it sends rather than drafts); a run that can't do its brief calls `report_run_failed` (scheduled runs only), which throws `ScheduledRunFailedError` so the queue retries it, `AppData.ScheduledRunRetryMs` apart and doubling (`scheduledRunRetryAt`); once out of attempts the user gets one "failed after N tries" message and the schedule's `last_error` keeps why (`recordCronJobFailure`). `/admin/routines` lists them (pause/resume, edit, delete; `api/admin/routines/update-routine`, `delete-routine`); bots create them from chat. - **Schedules (routines in the UI):** `cron_jobs` (per user and bot, `workspace_id` null for its own chat, `title`, `instructions` (the brief), `schedule` (a `Bun.cron` expression, UTC), `computer_id` (not a foreign key; null runs on the bot's current one), `enabled`, `max_runs` (1 for a one-off), `run_count`, `next_run_at`, `last_run_at`, `last_job_id`, `last_error`). Helpers in `src/functions/backend/db/cron-jobs/` (`create-cron-job`, `update-cron-job`, `delete-cron-job`, `grab-cron-jobs`, `grab-user-cron-job`, `fire-cron-job`), all ownership-checked, and with `bot_id` limited to that bot's own. `nextCronRun` (`src/utils/next-cron-run.ts`, `Bun.cron.parse`) validates a schedule and refuses one that runs more often than `AppData.CronJobMinIntervalMs`; a bot has at most `AppData.CronJobMaxPerBot`. Every bot gets `create_schedule`, `list_schedules`, `update_schedule`, `delete_schedule` (`grabBotTools`, every run; descriptions via `describe-cron-job.ts`). The `run-cron-jobs` cron fires due ones (`fireCronJob`: claims the run atomically by moving `next_run_at`, skips it while the last run's job is still open, queues the brief as a task with `enqueueBotJob` in the bot's active session there, turns off after `max_runs` or when there's no next run, keeps why it couldn't in `last_error`, logs each run under `cron`). A run missed while the server was down runs once, then carries on from now. A run acts on its own (`ScheduledRunNote`: the brief is already approved, so it sends rather than drafts); a run that can't do its brief calls `report_run_failed` (scheduled runs only), which throws `ScheduledRunFailedError` so the queue retries it, `AppData.ScheduledRunRetryMs` apart and doubling (`scheduledRunRetryAt`); once out of attempts the user gets one "failed after N tries" message and the schedule's `last_error` keeps why (`recordCronJobFailure`). `/admin/routines` lists them (pause/resume, edit, delete; `api/admin/routines/update-routine`, `delete-routine`); bots create them from chat.
- **Tasks (the crew's to-do list):** `tasks` is the tracker, one collection per bot (`bot_id`) and one per workspace (`workspace_id`; exactly one is set, enforced in `taskScopeValues`, since SQLite treats NULL in a unique constraint as distinct - the `envs` precedent). It is not `bot_tasks` or `bots_queue`: those record a queued job, this records the work. Fields: `title`, `description` (the brief, markdown), `status` (`TaskStatuses` in `src/dict/task-dict.ts`: `todo|doing|blocked|done|cancelled`), `blocked_reason` (only while blocked), `origin` (`user|bot`), `created_by_bot_id`/`created_by_user_id` (not foreign keys), `completed_at`. `task_notes` is the progress log (author `user|bot`, `bot_id`, `text`); `task_media` links a file to a task as a `deliverable` or a `reference` (`task_id`, `media_id` FK, `bot_file_id` not a FK, `kind`, `name`, `added_by_bot_id`), the bytes being a private `media` row (paradigm `task-media`) served at `/media/private/<id>` through `grabTaskMediaFile` (registered in `handle-media-server.ts` beside `grabBotFileMedia`). Visibility is SQL in `src/utils/task-access/` (`taskScopesSql` for a user: `chatBotsSql` bots plus `workspacesSql` workspaces; `botTaskScopesSql` for a run: its own plus every workspace it works in, everything for the coordinator; tested against SQLite). A workspace's tasks are visible to every bot working there and to the coordinator. Helpers in `src/functions/backend/db/tasks/`: `create-task` (`assertCrewManager` for a manual add, `TaskMaxPerScope` cap), `grab-tasks` (filters, joins the bot's face, the workspace and the latest note), `grab-task`, `update-task` (stamps `completed_at`, clears `blocked_reason` when it leaves blocked), `delete-task`, `add-task-note`, `attach-task-file`/`detach-task-file` (a bot file resolved through `grabBotFile` so a run can't link outside its folders; a chat upload must be the user's own media; `TaskMediaMaxPerTask`), `grab-task-media`, `grab-task-media-file`, `find-duplicate-task`, `purge-task-media` (`grabTaskMediaIds`, read before `deleteBot`/`deleteWorkspace` so a task's own chat uploads aren't stranded; a bot file keeps its folder). Every run gets `list_tasks`, `create_task`, `update_task` (`src/harness/tools/bot/tasks/`, from `grabBotTools`): a worker may only add to its own list or the workspace it works in, the coordinator to any worker's, any workspace's, or its own, and `update_task` takes `status`, `blocked_reason`, a `note` and `deliverable_file_ids`/`reference_file_ids` (`[file:ID]` refs) in one call. `TasksNote` (every reply and task) says how to keep the list honest; `TaskReminderNote` (chat only) tells the bot to check `list_tasks` on a greeting or when proposing next steps and mention at most `AppData.TaskReminderDigestCount` open tasks in one line, blocked first, never repeating a reminder and staying quiet mid-task. The user adds and changes tasks by hand in a workspace's dashboard (`(partials)/workspace-tasks-group.tsx`) and from the conversation header's menu (`(partials)/bot-tasks-modal.tsx`, loading its list when it opens so the page props stay small); both compose the shared `TaskForm`, `TaskListItem`, `TaskDetailPanel`, `TaskStatusChip` and `TaskFileRow` in `src/components/general/`. API in `api/admin/tasks/` (`list-tasks`, `get-task`, `save-task`, `delete-task`, `add-note`, `attach-file`, `detach-file`), each `userAuth` → `isReadOnlyUser` → `assertUserReachesTask` → `safeErrorMessage`. Live pushes for task changes are not built yet. - **Tasks (the crew's to-do list):** `tasks` is the tracker, one collection per bot (`bot_id`) and one per workspace (`workspace_id`; exactly one is set, enforced in `taskScopeValues`, since SQLite treats NULL in a unique constraint as distinct - the `envs` precedent). It is not `bot_tasks` or `bots_queue`: those record a queued job, this records the work. Fields: `title`, `description` (the brief, markdown), `status` (`TaskStatuses` in `src/dict/task-dict.ts`: `todo|doing|blocked|done|cancelled`), `blocked_reason` (only while blocked), `origin` (`user|bot`), `created_by_bot_id`/`created_by_user_id` (not foreign keys), `completed_at`. `task_notes` is the progress log (author `user|bot`, `bot_id`, `text`); `task_media` links a file to a task as a `deliverable` or a `reference` (`task_id`, `media_id` FK, `bot_file_id` not a FK, `kind`, `name`, `added_by_bot_id`), the bytes being a private `media` row (paradigm `task-media`) served at `/media/<id>` through `grabTaskMediaFile` (registered in `handle-media-server.ts` beside `grabBotFileMedia`). Visibility is SQL in `src/utils/task-access/` (`taskScopesSql` for a user: `chatBotsSql` bots plus `workspacesSql` workspaces; `botTaskScopesSql` for a run: its own plus every workspace it works in, everything for the coordinator; tested against SQLite). A workspace's tasks are visible to every bot working there and to the coordinator. Helpers in `src/functions/backend/db/tasks/`: `create-task` (`assertCrewManager` for a manual add, `TaskMaxPerScope` cap), `grab-tasks` (filters, joins the bot's face, the workspace and the latest note), `grab-task`, `update-task` (stamps `completed_at`, clears `blocked_reason` when it leaves blocked), `delete-task`, `add-task-note`, `attach-task-file`/`detach-task-file` (a bot file resolved through `grabBotFile` so a run can't link outside its folders; a chat upload must be the user's own media; `TaskMediaMaxPerTask`), `grab-task-media`, `grab-task-media-file`, `find-duplicate-task`, `purge-task-media` (`grabTaskMediaIds`, read before `deleteBot`/`deleteWorkspace` so a task's own chat uploads aren't stranded; a bot file keeps its folder). Every run gets `list_tasks`, `create_task`, `update_task` (`src/harness/tools/bot/tasks/`, from `grabBotTools`): a worker may only add to its own list or the workspace it works in, the coordinator to any worker's, any workspace's, or its own, and `update_task` takes `status`, `blocked_reason`, a `note` and `deliverable_file_ids`/`reference_file_ids` (`[file:ID]` refs) in one call. `TasksNote` (every reply and task) says how to keep the list honest; `TaskReminderNote` (chat only) tells the bot to check `list_tasks` on a greeting or when proposing next steps and mention at most `AppData.TaskReminderDigestCount` open tasks in one line, blocked first, never repeating a reminder and staying quiet mid-task. The user adds and changes tasks by hand in a workspace's dashboard (`(partials)/workspace-tasks-group.tsx`) and from the conversation header's menu (`(partials)/bot-tasks-modal.tsx`, loading its list when it opens so the page props stay small); both compose the shared `TaskForm`, `TaskListItem`, `TaskDetailPanel`, `TaskStatusChip` and `TaskFileRow` in `src/components/general/`. API in `api/admin/tasks/` (`list-tasks`, `get-task`, `save-task`, `delete-task`, `add-note`, `attach-file`, `detach-file`), each `userAuth` → `isReadOnlyUser` → `assertUserReachesTask` → `safeErrorMessage`. Live pushes for task changes are not built yet.
- **Not built yet:** task steps (nothing writes `bot_task_steps` yet), live `server:task-changed` pushes, and a `/admin/tasks` page. - **Not built yet:** task steps (nothing writes `bot_task_steps` yet), live `server:task-changed` pushes, and a `/admin/tasks` page.
### Installer and release ### Installer and release
@@ -152,7 +152,7 @@ Envs are set in Settings, under Environment (super admin only), not in `.env`. T
### Bot files and folders ### Bot files and folders
Bots keep files on the app's data volume, never on a computer: `DATA_DIR/bots/<bot_id>` for a bot's own, `DATA_DIR/workspaces/<workspace_id>` shared by the bots working there. The catalog is `bot_files` (scope `bot`|`workspace`, `owner_id`, `path`, `size`, `purpose`, `source`, `ingested_at`, `ingest_error`), its bytes are a private `media` row (paradigm `bot-file`) written by `recordBotFile` (`src/functions/backend/db/bot-files/record-bot-file.ts`), so they're served at `/media/private/<media id>` through `grabBotFileMedia` and deleted with the row. Bots keep files on the app's data volume, never on a computer: `DATA_DIR/bots/<bot_id>` for a bot's own, `DATA_DIR/workspaces/<workspace_id>` shared by the bots working there. The catalog is `bot_files` (scope `bot`|`workspace`, `owner_id`, `path`, `size`, `purpose`, `source`, `ingested_at`, `ingest_error`), its bytes are a private `media` row (paradigm `bot-file`) written by `recordBotFile` (`src/functions/backend/db/bot-files/record-bot-file.ts`), so they're served at `/media/<media id>` through `grabBotFileMedia` and deleted with the row.
Who reaches what: `grabStorageScopes` (`src/utils/storage-access.ts`) gives a worker its own folder plus the workspace's while it works there, and the coordinator every bot's and every workspace's. Enforced in the tools, never left to the prompt. `resolveBotPath` (`src/utils/resolve-bot-path.ts`) keeps every path inside its root (no absolute paths, `..`, NUL, symlink escapes) and the quota (`app_settings` key `bot_files_quota_mb`, set on Settings > General, `grabBotFileQuotaBytes`) is checked by `assertStorageRoom` before a write. Who reaches what: `grabStorageScopes` (`src/utils/storage-access.ts`) gives a worker its own folder plus the workspace's while it works there, and the coordinator every bot's and every workspace's. Enforced in the tools, never left to the prompt. `resolveBotPath` (`src/utils/resolve-bot-path.ts`) keeps every path inside its root (no absolute paths, `..`, NUL, symlink escapes) and the quota (`app_settings` key `bot_files_quota_mb`, set on Settings > General, `grabBotFileQuotaBytes`) is checked by `assertStorageRoom` before a write.
@@ -160,7 +160,7 @@ Tools in `src/harness/tools/storage/` (`grabBotFileTools`, every run: chat, task
A task on a computer also gets `import_file` (`src/harness/tools/computer/import-file-tool.ts`): it copies a file or a folder off the computer into the run's files (a folder as a gzipped tar, unpacked into one `bot_files` row per file, source `tool`), ingests it at once, and answers with the `[file:ID]` refs plus the original's path. The bytes come back through `downloadComputerFile` (`download-computer-file.ts`), the sandbox's streaming `GET /v1/file/download` - **never through its shell**: `/v1/shell/exec` caps output at 30,047 characters, so `base64 -w0` silently cut every import at 22,528 bytes and a 1.12 MiB archive was saved as a 22 KB file that would not open, with the tool reporting success. The size is read first (nothing over `AppData.BotFileImportMaxBytes` is transferred), the download asks for `change_policy=abort` (409 if the file changes while it is sent) and the length is checked against the size read, so a cut-short transfer is reported instead of saved. `BotFilesNote` and `ComputerFilesNote` (`src/data/harness-prompts.ts`) tell the bot to save the deliverable before the long tail of a run, so a step limit can't cost the user the report. A task on a computer also gets `import_file` (`src/harness/tools/computer/import-file-tool.ts`): it copies a file or a folder off the computer into the run's files (a folder as a gzipped tar, unpacked into one `bot_files` row per file, source `tool`), ingests it at once, and answers with the `[file:ID]` refs plus the original's path. The bytes come back through `downloadComputerFile` (`download-computer-file.ts`), the sandbox's streaming `GET /v1/file/download` - **never through its shell**: `/v1/shell/exec` caps output at 30,047 characters, so `base64 -w0` silently cut every import at 22,528 bytes and a 1.12 MiB archive was saved as a 22 KB file that would not open, with the tool reporting success. The size is read first (nothing over `AppData.BotFileImportMaxBytes` is transferred), the download asks for `change_policy=abort` (409 if the file changes while it is sent) and the length is checked against the size read, so a cut-short transfer is reported instead of saved. `BotFilesNote` and `ComputerFilesNote` (`src/data/harness-prompts.ts`) tell the bot to save the deliverable before the long tail of a run, so a step limit can't cost the user the report.
Anything a bot leaves on a computer is not in `bot_files`, but the user can still open it: bots name a path as `[path:<volume>:<owner>:<path>]` (`src/utils/path-ref.ts`, e.g. `[path:computer:3:seo/coderank/raw]`), linked by the same `linkComputerRefs` and rendered by `path-ref-chip.tsx`, which opens `/admin/files` on that folder. Anything a bot leaves on a computer is not in `bot_files`, but the user can still open it: bots name a folder as `[folder:<volume>:<owner>:<path>]` (`src/utils/folder-ref.ts`, e.g. `[folder:computer:3:seo/coderank/raw]`), linked by the same `linkComputerRefs` and rendered by `folder-ref-chip.tsx`, which opens `/admin/files` on that folder and downloads nothing. A `[file:<id>]` ref (`src/utils/file-ref.ts`) is always a saved file, whose chip opens `/media/<id>`; the two are never guessed from a name's extension.
Ingestion (`bot_file_chunks`, `bot_file_vectors` a vec0 table, `file_ingest_queue`): `ingestBotFile` extracts text (`extractBotFileText`, reusing `fileToText`; images and audio only on request), chunks it (`chunkText`), embeds it through the `embedding` harness and fits every vector to the fixed `AppData.EmbeddingDimensions` (`fitVectorDimensions`, pad or cut then normalize). The `file-ingest` worker (`src/file-ingest/`, started from `server.ts` next to the bots queue) works the queue; `ingest-pending-files` sweeps up anything stranded. With no embedding model the chunks are still stored and `searchBotFiles` falls back to word matching, always saying which it did. Ingestion (`bot_file_chunks`, `bot_file_vectors` a vec0 table, `file_ingest_queue`): `ingestBotFile` extracts text (`extractBotFileText`, reusing `fileToText`; images and audio only on request), chunks it (`chunkText`), embeds it through the `embedding` harness and fits every vector to the fixed `AppData.EmbeddingDimensions` (`fitVectorDimensions`, pad or cut then normalize). The `file-ingest` worker (`src/file-ingest/`, started from `server.ts` next to the bots queue) works the queue; `ingest-pending-files` sweeps up anything stranded. With no embedding model the chunks are still stored and `searchBotFiles` falls back to word matching, always saying which it did.
@@ -177,9 +177,9 @@ Every folder on the data volume, browsable read-only: a bot's own, a workspace's
- **Volumes:** `src/utils/storage-volumes.ts` (client-safe): `StorageVolumes` (`bots`, `workspaces`, `computers`, `media`, `mcp`, `db`) with each one's ref name (`bot`, `computer`, …), `parseStorageSpec({ volume, owner_id })` (null for anything unknown, so a bad id never becomes a path), and `storageSpecHref({ spec, path })` → `/admin/files?volume=<v>&owner=<id>&path=<p>`, the one place a link is built. - **Volumes:** `src/utils/storage-volumes.ts` (client-safe): `StorageVolumes` (`bots`, `workspaces`, `computers`, `media`, `mcp`, `db`) with each one's ref name (`bot`, `computer`, …), `parseStorageSpec({ volume, owner_id })` (null for anything unknown, so a bad id never becomes a path), and `storageSpecHref({ spec, path })` → `/admin/files?volume=<v>&owner=<id>&path=<p>`, the one place a link is built.
- **Access:** `resolveStorageSpec` (`src/functions/backend/storage/resolve-storage-spec.ts`) is the only gate, called before any path exists: a bot through `grabUserBot`, a workspace through `grabUserWorkspace`, a computer through `grabUserComputer` (so a member sees only their grants), `media` only their own (super admins and admins anyone's), `mcp` for crew managers, `db` for the super admin alone. `grabStorageSpecs` lists the same folders for the picker, with `reachableBotsSql` (`src/utils/crew-access/reachable-bots-sql.ts`, tested beside the other crew SQL) so the picker never offers one the resolver refuses. - **Access:** `resolveStorageSpec` (`src/functions/backend/storage/resolve-storage-spec.ts`) is the only gate, called before any path exists: a bot through `grabUserBot`, a workspace through `grabUserWorkspace`, a computer through `grabUserComputer` (so a member sees only their grants), `media` only their own (super admins and admins anyone's), `mcp` for crew managers, `db` for the super admin alone. `grabStorageSpecs` lists the same folders for the picker, with `reachableBotsSql` (`src/utils/crew-access/reachable-bots-sql.ts`, tested beside the other crew SQL) so the picker never offers one the resolver refuses.
- **Paths:** `resolveStoragePath` (`src/functions/backend/storage/resolve-storage-path.ts`) applies the `resolveBotPath` rules (no absolute paths, `..`, NUL, too deep, symlink escape) to any volume root. - **Paths:** `resolveStoragePath` (`src/functions/backend/storage/resolve-storage-path.ts`) applies the `resolveBotPath` rules (no absolute paths, `..`, NUL, too deep, symlink escape) to any volume root.
- **Reading:** `listStorageEntries` (folders first, `AppData.StorageBrowserPageSize`, a missing folder shows empty, symlinks are skipped, `bot_files` rows joined by `(scope, owner_id, path)` so those files keep opening through `/media/private/<id>`) and `grabStorageFile` (a regular file under `AppData.StorageBrowserReadMaxBytes`). API in `src/pages/api/admin/storage/` (`list`, `file`, the latter with an ETag like `handleMediaServer`), zod in `src/schemas/storage-schema.ts`. - **Reading:** `listStorageEntries` (folders first, `AppData.StorageBrowserPageSize`, a missing folder shows empty, symlinks are skipped, `bot_files` rows joined by `(scope, owner_id, path)` so those files keep opening through `/media/<id>`) and `grabStorageFile` (a regular file under `AppData.StorageBrowserReadMaxBytes`). API in `src/pages/api/admin/storage/` (`list`, `file`, the latter with an ETag like `handleMediaServer`), zod in `src/schemas/storage-schema.ts`.
- **Page:** `src/pages/admin/files/index.server.ts` + `index.tsx` (in `SettingsShell`, `wide`), route-local code: `(sections)/files-browser.tsx` and `(sections)/files-pane.tsx` (state, `history.pushState`, so a folder link is shareable and the back button walks back up), `(partials)/files-volume-row.tsx`, `files-breadcrumb-row.tsx`, `files-search-input.tsx`, `storage-entry-row.tsx`, `(hooks)/use-storage-folder.ts` (the page's own copy first, the API on navigation). Read-only: deletion stays in the chat's files modal and the bot's tools. - **Page:** `src/pages/admin/files/index.server.ts` + `index.tsx` (in `SettingsShell`, `wide`), route-local code: `(sections)/files-browser.tsx` and `(sections)/files-pane.tsx` (state, `history.pushState`, so a folder link is shareable and the back button walks back up), `(partials)/files-volume-row.tsx`, `files-breadcrumb-row.tsx`, `files-search-input.tsx`, `storage-entry-row.tsx`, `(hooks)/use-storage-folder.ts` (the page's own copy first, the API on navigation). Read-only: deletion stays in the chat's files modal and the bot's tools.
- **Chips:** `[path:…]` refs open it, and the bot files modal links into it ("Browse this folder in Files"). - **Chips:** `[folder:…]` refs open it, and the bot files modal links into it ("Browse this folder in Files").
### Harness (the model layer) ### Harness (the model layer)
+1 -6
View File
@@ -158,11 +158,6 @@ const schema: BUN_SQLITE_DatabaseSchemaType = {
dataType: "INTEGER", dataType: "INTEGER",
options: [0, 1], options: [0, 1],
}, },
{
fieldName: "is_private",
dataType: "INTEGER",
options: [0, 1],
},
{ {
fieldName: "archived", fieldName: "archived",
dataType: "INTEGER", dataType: "INTEGER",
@@ -688,7 +683,7 @@ const schema: BUN_SQLITE_DatabaseSchemaType = {
}, },
{ {
tableName: "task_media", tableName: "task_media",
tableDescription: `A file linked to a task: the deliverable it produced, or something it needs to do the work. The bytes are a private \`media\` row (paradigm \`task-media\`), served at \`/media/private/<id>\` to whoever reaches the task`, tableDescription: `A file linked to a task: the deliverable it produced, or something it needs to do the work. The bytes are a private \`media\` row (paradigm \`task-media\`), served at \`/media/<id>\` to whoever reaches the task`,
fields: [ fields: [
{ {
fieldName: "task_id", fieldName: "task_id",
-1
View File
@@ -146,7 +146,6 @@ export type BUN_SQLITE_COBBLE_MEDIA = {
text_content?: string; text_content?: string;
is_primary?: 0 | 1 | ""; is_primary?: 0 | 1 | "";
is_rag_ingested?: 0 | 1 | ""; is_rag_ingested?: 0 | 1 | "";
is_private?: 0 | 1 | "";
archived?: 0 | 1 | ""; archived?: 0 | 1 | "";
} }
@@ -90,7 +90,7 @@ describe("botMessagesToHarness with a file-only reply", () => {
author: "user", author: "user",
text: "", text: "",
reply_to: { id: 4, author: "bot", text: "Send me the logo." }, reply_to: { id: 4, author: "bot", text: "Send me the logo." },
attachments: [{ id: 9, name: "logo.png", media_type: "image", url: "/media/private/9" }], attachments: [{ id: 9, name: "logo.png", media_type: "image", url: "/media/9" }],
}, },
], ],
}), }),
@@ -104,7 +104,7 @@ describe("botMessagesToHarness with a file-only reply", () => {
}); });
describe("botMessagesToHarness with files", () => { describe("botMessagesToHarness with files", () => {
const attachments = [{ id: 9, name: "shot.png", media_type: "image" as const, url: "/media/private/9" }]; const attachments = [{ id: 9, name: "shot.png", media_type: "image" as const, url: "/media/9" }];
test("puts a message's loaded files after its text", () => { test("puts a message's loaded files after its text", () => {
const parts = new Map([[1, [{ type: "image" as const, url: "data:image/jpeg;base64,AA", name: "shot.png" }]]]); const parts = new Map([[1, [{ type: "image" as const, url: "data:image/jpeg;base64,AA", name: "shot.png" }]]]);
+3 -3
View File
@@ -15,8 +15,8 @@ type Props = {
}; };
/** /**
* One file on a task: its name, opening the bytes at * One file on a task: its name, opening the bytes at `/media/<id>` (private,
* `/media/private/<id>`, and its file ref (built by `fileRef`) when it * so only someone who reaches the task can load it), and its file ref (built by `fileRef`) when it
* lives in a bot's or workspace's folder (so the file's real home is * lives in a bot's or workspace's folder (so the file's real home is
* reachable too). * reachable too).
*/ */
@@ -40,7 +40,7 @@ export default function TaskFileRow({ file, onDetach, busy }: Props) {
<FileText size={16} className="shrink-0 text-ink-secondary" aria-hidden /> <FileText size={16} className="shrink-0 text-ink-secondary" aria-hidden />
)} )}
<Link <Link
href={`/media/private/${file.media_id}`} href={`/media/${file.media_id}`}
target="_blank" target="_blank"
rel="noreferrer" rel="noreferrer"
title={file.name || "Open the file"} title={file.name || "Open the file"}
+14 -5
View File
@@ -1,6 +1,6 @@
import { AppData } from "@/src/data/app-data"; import { AppData } from "@/src/data/app-data";
import botRef from "@/src/utils/bot-ref"; import botRef from "@/src/utils/bot-ref";
import pathRef from "@/src/utils/path-ref"; import folderRef from "@/src/utils/folder-ref";
/** /**
* Prompts the helper harnesses send. The text model gets their output in * Prompts the helper harnesses send. The text model gets their output in
@@ -33,7 +33,7 @@ You are chatting with the user. Answer their latest messages directly and briefl
- To add a worker to the crew, use create_bot once the user has asked for it or agreed, and add_bot_to_workspace to place it. A bot exists only when create_bot made it: never describe one as if it did, and never write it into a workspace's context. - To add a worker to the crew, use create_bot once the user has asked for it or agreed, and add_bot_to_workspace to place it. A bot exists only when create_bot made it: never describe one as if it did, and never write it into a workspace's context.
- Never claim you did something you didn't do. - Never claim you did something you didn't do.
- The user can mention a bot as [bot:ID] and a workspace as [workspace:ID]; a [Mentions: ...] line after their message says who and what those are. Write the same refs when you name them back. - The user can mention a bot as [bot:ID] and a workspace as [workspace:ID]; a [Mentions: ...] line after their message says who and what those are. Write the same refs when you name them back.
- Documents, images and reports you make are files: save them with write_file, make_pdf or generate_image, and send one to the user with it ("attach"). When you name a file, write its ref, e.g. [file:12], so the user can open it. - Documents, images and reports you make are files: save them with write_file, make_pdf or generate_image, and send one to the user with it ("attach"). When you name a file, write its [file:ID] ref, e.g. [file:12], so the user can open it. When you name a folder, write a folder ref instead, e.g. ${folderRef({ volume: "bot", owner_id: 5, path: "reports" })}: that opens the folder in Files and downloads nothing.
## Keep the work moving ## Keep the work moving
@@ -50,14 +50,23 @@ You have a folder of your own, kept with the app (not on a computer), and it sta
- list_files shows what's in it; read_file opens one (by its [file:ID] ref or path); write_file saves a text file, make_pdf a PDF, generate_image an image. Move and delete with move_file and delete_file when something is out of date, so the folder stays useful. - list_files shows what's in it; read_file opens one (by its [file:ID] ref or path); write_file saves a text file, make_pdf a PDF, generate_image an image. Move and delete with move_file and delete_file when something is out of date, so the folder stays useful.
- Text files are chunked and searchable as soon as they're saved. search_files finds the passages that answer a question, by meaning when an embedding model is set and by words otherwise. Search your files before saying you don't know something they may hold, and when the user asks about a document they gave you. - Text files are chunked and searchable as soon as they're saved. search_files finds the passages that answer a question, by meaning when an embedding model is set and by words otherwise. Search your files before saying you don't know something they may hold, and when the user asks about a document they gave you.
- Uploaded files land in the same folder, so what the user sent you earlier is searchable later too. - Uploaded files land in the same folder, so what the user sent you earlier is searchable later too.
- When you name a file, write its [file:ID] ref: the user sees a link to it. They can also browse everything under Files, in the menu. - A file and a folder are two different refs, and you never guess which one from the name:
- A file you saved is a [file:ID] ref (from write_file, list_files, search_files or import_file). The user sees a chip that opens the file.
- A folder is a folder ref, e.g. ${folderRef({ volume: "bot", owner_id: 5, path: "reports" })}: the user sees a chip that opens that folder in Files, and nothing is downloaded. Use the volume of the folder you mean ("bot" for your own, "workspace" for a workspace's shared one).
- Never write a path as plain text and expect the user to find it; never write a [file:ID] ref for something that is a folder.
- A file you only described is not saved. Save it first, then say it's saved. - A file you only described is not saved. Save it first, then say it's saved.
- Save what the user will want before the long tail of a run. A step limit ends a run where it stands: write the report first with write_file (or copy it off the computer with import_file), then carry on verifying.`; - Save what the user will want before the long tail of a run. A step limit ends a run where it stands: write the report first with write_file (or copy it off the computer with import_file), then carry on verifying.
- One copy of a thing is enough. Before you write or import something, list_files (or search_files) for it: if it's already there, update it (write_file over the same path) instead of saving a second copy under a new name. Never split a file into parts unless the user asked for parts.
- Your files don't disappear. If you can't find something, list_files or search_files before saying anything is missing, and never claim the folder is empty without having just listed it.
- Report what the tools actually returned. Don't invent limits, sizes or caps you haven't seen in a tool result: if an import or a write failed, say what the tool said.`;
/** /**
* Added to a run that has a computer: what it leaves there is not in its * Added to a run that has a computer: what it leaves there is not in its
* files, but the user can still open it. `can_import` is false in a chat * files, but the user can still open it. `can_import` is false in a chat
* reply, where the import only happens inside a task. * reply, where the import only happens inside a task.
*
* A folder ref is always a folder, so a file left on the computer is named
* by the folder that holds it, never by the file's own path.
*/ */
export const ComputerFilesNote = ({ export const ComputerFilesNote = ({
computer_id, computer_id,
@@ -69,7 +78,7 @@ export const ComputerFilesNote = ({
What you make on the computer (downloads, screenshots, a crawl, raw evidence, logs) lives in its home, not in your files. It survives restarts of the computer, but it is not searchable and it goes if the computer is rebuilt. What you make on the computer (downloads, screenshots, a crawl, raw evidence, logs) lives in its home, not in your files. It survives restarts of the computer, but it is not searchable and it goes if the computer is rebuilt.
- The user can open any of it: name a path on the computer as a path ref, e.g. ${pathRef({ volume: "computer", owner_id: computer_id, path: "seo/coderank/raw" })}. Point at the folder for evidence they may want to see. - The user can open any of it: name a folder on the computer as a folder ref, e.g. ${folderRef({ volume: "computer", owner_id: computer_id, path: "seo/coderank/raw" })}. It opens that folder in Files, so it downloads nothing. Point at the folder holding the evidence they may want to see.
${ ${
can_import can_import
? `- import_file copies a file or a folder off the computer into your own files, in one go, and the copy is searchable straight away. Import the finished work (the report, the spreadsheet, the final document), attach it when it is what they asked for, and name it with its [file:ID] ref.\n- The raw evidence can stay on the computer: import it only when the user needs it kept, or when the computer may be rebuilt.\n` ? `- import_file copies a file or a folder off the computer into your own files, in one go, and the copy is searchable straight away. Import the finished work (the report, the spreadsheet, the final document), attach it when it is what they asked for, and name it with its [file:ID] ref.\n- The raw evidence can stay on the computer: import it only when the user needs it kept, or when the computer may be rebuilt.\n`
@@ -220,7 +220,6 @@ async function insertMediaRow({
media_type: media_type as any, media_type: media_type as any,
media_write_path: write_path, media_write_path: write_path,
mime_type: mime, mime_type: mime,
is_private: 1,
}, },
], ],
}); });
@@ -43,18 +43,6 @@ export default async function grabMedia(
values.push(...query.media_paradigms, query.media_paradigms.length); values.push(...query.media_paradigms, query.media_paradigms.length);
} }
if (query?.is_media_private) {
sql += ` AND m.is_private = 1\n`;
if (!user?.id) {
throw new Error(`Private media requires authenticated user`);
} else {
sql += ` AND m.user_id = ?\n`;
values.push(user.id);
}
} else {
sql += ` AND COALESCE(m.is_private, 0) <> 1\n`;
}
if (query?.id) { if (query?.id) {
sql += ` AND m.id = ?\n`; sql += ` AND m.id = ?\n`;
values.push(query.id); values.push(query.id);
@@ -0,0 +1,56 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import type { BUN_SQLITE_COBBLE_MEDIA, BunSQLiteTables } from "@/db/types/db";
import grabSharedPictureMedia from "@/src/functions/backend/db/access/grab-shared-picture-media";
import grabBotFileMedia from "@/src/functions/backend/db/bot-files/grab-bot-file-media";
import grabTaskMediaFile from "@/src/functions/backend/db/tasks/grab-task-media-file";
type Params = {
media_id: number | string;
user_id: number | string;
};
/**
* One media row, only if this user may load its bytes. Every media URL is
* `/media/<id>` and every one of them is private, so this is the single
* gate in front of the file on disk:
*
* - the user's own uploads (`media.user_id`),
* - a bot's or workspace's picture, to whoever reaches that bot or
* workspace (`grabSharedPictureMedia`),
* - a bot's or workspace's file, to whoever reaches the folder it lives in
* (`grabBotFileMedia`),
* - a file attached to a task, to whoever reaches the task
* (`grabTaskMediaFile`).
*
* Null when it is not theirs to see, so a request never turns into a path
* before this has passed.
*/
export default async function grabUserMedia({
media_id,
user_id,
}: Params): Promise<BUN_SQLITE_COBBLE_MEDIA | undefined> {
if (!media_id || !user_id) return undefined;
const own = await BunSQLite.select<
BUN_SQLITE_COBBLE_MEDIA,
(typeof BunSQLiteTables)[number]
>({
table: "media",
query: {
query: {
user_id: {
value: user_id,
},
},
},
targetId: media_id,
});
return (
own.singleRes ||
(await grabSharedPictureMedia({ media_id, user_id })) ||
(await grabBotFileMedia({ media_id, user_id })) ||
(await grabTaskMediaFile({ media_id, user_id })) ||
undefined
);
}
@@ -62,7 +62,6 @@ export default async function setProfileImage({ user_id, target, id, data_url }:
media_mime_type: ext, media_mime_type: ext,
media_paradigms: [paradigm], media_paradigms: [paradigm],
media: { title: `${record.name || target} picture` }, media: { title: `${record.name || target} picture` },
is_private: true,
user: { id: Number(user_id) }, user: { id: Number(user_id) },
}); });
@@ -34,7 +34,6 @@ type Params = {
media?: BUN_SQLITE_COBBLE_MEDIA; media?: BUN_SQLITE_COBBLE_MEDIA;
media_name?: string; media_name?: string;
media_mime_type?: string; media_mime_type?: string;
is_private?: boolean;
image_width?: number; image_width?: number;
image_height?: number; image_height?: number;
thumbnail_width?: number; thumbnail_width?: number;
@@ -53,7 +52,6 @@ export default async function uploadAndRecordMedia({
media, media,
media_mime_type, media_mime_type,
media_name, media_name,
is_private,
image_width, image_width,
image_height, image_height,
thumbnail_width, thumbnail_width,
@@ -64,17 +62,10 @@ export default async function uploadAndRecordMedia({
let target_file_name = `${media_name || now}`; let target_file_name = `${media_name || now}`;
let target_file_thumbnail_name = `${media_name || now}-thumb`; let target_file_thumbnail_name = `${media_name || now}-thumb`;
const { const { USER_MEDIA_RELATIVE_DIR: MEDIA_RELATIVE_DIR, DATA_DIR } =
USER_MEDIA_PRIVATE_RELATIVE_DIR, grabDirNames({
USER_MEDIA_PUBLIC_RELATIVE_DIR, user,
DATA_DIR, });
} = grabDirNames({
user,
});
const MEDIA_RELATIVE_DIR = is_private
? USER_MEDIA_PRIVATE_RELATIVE_DIR
: USER_MEDIA_PUBLIC_RELATIVE_DIR;
if (!MEDIA_RELATIVE_DIR) { if (!MEDIA_RELATIVE_DIR) {
throw new Error(`No target media directory found!`); throw new Error(`No target media directory found!`);
@@ -236,7 +227,6 @@ export default async function uploadAndRecordMedia({
* lets NULLs repeat. * lets NULLs repeat.
*/ */
is_primary: is_primary ? 1 : (null as any), is_primary: is_primary ? 1 : (null as any),
is_private: is_private ? 1 : 0,
media_write_path: media_write_path, media_write_path: media_write_path,
media_thumbnail_write_path: media_thumbnail_write_path:
media_type === "image" ? media_thumbnail_write_path : undefined, media_type === "image" ? media_thumbnail_write_path : undefined,
@@ -6,6 +6,14 @@ type Params = {
user?: BUN_SQLITE_COBBLE_USERS_JOIN; user?: BUN_SQLITE_COBBLE_USERS_JOIN;
}; };
/**
* Where a media file is served: `/media/<id>`, always, with
* `/media/<id>/thumbnail` for an image's smaller copy.
*
* There is no public media and no `public`/`private` segment to pick: every
* media URL is private, so the request is checked against the signed-in
* user when it arrives (`grabUserMedia`).
*/
export default function grabMediaURL({ media: passed_media, user }: Params) { export default function grabMediaURL({ media: passed_media, user }: Params) {
const media = passed_media || {}; const media = passed_media || {};
@@ -13,19 +21,11 @@ export default function grabMediaURL({ media: passed_media, user }: Params) {
media.id = user.profile_media_id; media.id = user.profile_media_id;
} }
let media_url = `/media`;
if (media.is_private) {
media_url += `/private`;
} else {
media_url += `/public`;
}
if (!media.id) { if (!media.id) {
return {}; return {};
} }
media_url += `/${media.id}`; const media_url = `/media/${media.id}`;
const media_thumbnail_url = const media_thumbnail_url =
media.media_type == "image" || user?.id media.media_type == "image" || user?.id
@@ -25,8 +25,8 @@ type Params = {
* *
* Who reaches what: a bot, workspace or computer through the same SQL the * Who reaches what: a bot, workspace or computer through the same SQL the
* crew uses (`grabUserBot`, `grabUserWorkspace`, `grabUserComputer`, so a * crew uses (`grabUserBot`, `grabUserWorkspace`, `grabUserComputer`, so a
* member sees only their own grants); their own media folder (super * member sees only their own grants); their own media folder, and only
* admins and admins anyone's); an MCP server's install folder for crew * their own whatever their role; an MCP server's install folder for crew
* managers; the database for the super admin alone. * managers; the database for the super admin alone.
*/ */
export default async function resolveStorageSpec({ user_id, spec }: Params) { export default async function resolveStorageSpec({ user_id, spec }: Params) {
@@ -75,7 +75,12 @@ async function storageRoot({
: null; : null;
case "media": case "media":
if (role == "member" && Number(spec.owner_id) != Number(user_id)) return null; /**
* A person's uploads are theirs alone, whatever their role:
* the folder is the media library behind `/media/<id>`, not a
* crew-shared one.
*/
if (Number(spec.owner_id) != Number(user_id)) return null;
return (await hasUser({ user_id: spec.owner_id })) return (await hasUser({ user_id: spec.owner_id }))
? path.join(DATA_DIR, "media", String(spec.owner_id)) ? path.join(DATA_DIR, "media", String(spec.owner_id))
@@ -121,13 +126,8 @@ async function storageLabel({
(await grabUserComputer({ computer_id: spec.owner_id, user_id }))?.name || (await grabUserComputer({ computer_id: spec.owner_id, user_id }))?.name ||
`Computer ${spec.owner_id}` `Computer ${spec.owner_id}`
); );
case "media": { case "media":
if (role == "member" || Number(spec.owner_id) == Number(user_id)) return "My media"; return "My media";
const person = await grabPersonName({ user_id: spec.owner_id });
return person ? `${person}'s media` : `Media of user ${spec.owner_id}`;
}
case "mcp": case "mcp":
return (await grabMcpServerName({ server_id: spec.owner_id })) || `MCP server ${spec.owner_id}`; return (await grabMcpServerName({ server_id: spec.owner_id })) || `MCP server ${spec.owner_id}`;
case "db": case "db":
@@ -154,22 +154,6 @@ async function hasUser({ user_id }: { user_id: number | string }) {
return Boolean(res.payload?.[0]?.id); return Boolean(res.payload?.[0]?.id);
} }
async function grabPersonName({ user_id }: { user_id: number | string }) {
const res = await BunSQLite.sql<{
first_name?: string | null;
last_name?: string | null;
username?: string | null;
}>({
sql: `SELECT first_name, last_name, username FROM users WHERE id = ? LIMIT 1`,
values: [Number(user_id)],
});
const person = res.payload?.[0];
if (!person) return "";
return [person.first_name, person.last_name].filter(Boolean).join(" ") || person.username || "";
}
async function hasMcpServer({ server_id }: { server_id: number | string }) { async function hasMcpServer({ server_id }: { server_id: number | string }) {
const res = await BunSQLite.sql<{ id: number }>({ const res = await BunSQLite.sql<{ id: number }>({
sql: `SELECT id FROM mcp_servers WHERE id = ? LIMIT 1`, sql: `SELECT id FROM mcp_servers WHERE id = ? LIMIT 1`,
@@ -1,12 +1,8 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import type { BUN_SQLITE_COBBLE_MEDIA, BunSQLiteTables } from "@/db/types/db";
import grabDirNames from "@/src/utils/grab-dir-names"; import grabDirNames from "@/src/utils/grab-dir-names";
import { existsSync } from "node:fs"; import { existsSync } from "node:fs";
import path from "node:path"; import path from "node:path";
import userAuth from "../../backend/auth/user-auth"; import userAuth from "../../backend/auth/user-auth";
import grabSharedPictureMedia from "@/src/functions/backend/db/access/grab-shared-picture-media"; import grabUserMedia from "../../backend/db/media/grab-user-media";
import grabBotFileMedia from "@/src/functions/backend/db/bot-files/grab-bot-file-media";
import grabTaskMediaFile from "@/src/functions/backend/db/tasks/grab-task-media-file";
const { DATA_DIR } = grabDirNames(); const { DATA_DIR } = grabDirNames();
@@ -14,70 +10,37 @@ type Params = {
req: Request; req: Request;
}; };
/**
* Serves a media file's bytes: `/media/<id>`, and `/media/<id>/thumbnail`
* for an image's smaller copy.
*
* There is no public media. Every URL is private, so the request has to be
* signed in and the user has to reach the file (`grabUserMedia` decides
* that: their own upload, a bot's or workspace's file or picture they can
* chat with, a file on one of their tasks). Anything else is answered with
* a 404, never with the file.
*/
export default async function handleMediaServer({ export default async function handleMediaServer({
req, req,
}: Params): Promise<Response> { }: Params): Promise<Response> {
const url = new URL(req.url); const url = new URL(req.url);
const [root, privacy, media_id, thumbnail] = url.pathname const [_root, media_id, thumbnail] = url.pathname
.split("/") .split("/")
.filter((p) => p.match(/./)); .filter((p) => p.match(/./));
if (!privacy) { if (!media_id) {
throw new Error(`No media privacy params passed`); throw new Error(`No media id passed`);
} }
let media: BUN_SQLITE_COBBLE_MEDIA | undefined = undefined; const { user } = await userAuth({ req });
if (privacy == "private") { if (!user?.logged_in_status) {
const { user, user_types } = await userAuth({ req }); throw new Error(`Not logged in`);
if (!user?.logged_in_status) {
throw new Error(`Not logged in`);
}
const media_res = await BunSQLite.select<
BUN_SQLITE_COBBLE_MEDIA,
(typeof BunSQLiteTables)[number]
>({
table: "media",
query: {
query: {
user_id: {
value: user.id,
},
},
},
targetId: media_id,
});
media =
media_res.singleRes ||
(media_id ? await grabSharedPictureMedia({ media_id, user_id: user.id! }) : null) ||
(media_id ? await grabBotFileMedia({ media_id, user_id: user.id! }) : null) ||
(media_id ? await grabTaskMediaFile({ media_id, user_id: user.id! }) : null) ||
undefined;
} else {
const media_res = await BunSQLite.select<
BUN_SQLITE_COBBLE_MEDIA,
(typeof BunSQLiteTables)[number]
>({
table: "media",
targetId: media_id,
});
media = media_res.singleRes || undefined;
/**
* A private file is only served through `/media/private`, to its
* owner (or, for a bot's or workspace's picture, to whoever sees
* it).
*/
if (media?.is_private) {
media = undefined;
}
} }
const media = await grabUserMedia({ media_id, user_id: user.id! });
if (!media?.id) { if (!media?.id) {
throw new Error(`Media record not found!`); throw new Error(`Media record not found!`);
} }
@@ -98,12 +61,14 @@ export default async function handleMediaServer({
const file = Bun.file(full_media_location); const file = Bun.file(full_media_location);
const etag = `"${file.size}-${file.lastModified}"`; const etag = `"${file.size}-${file.lastModified}"`;
const cache_control =
privacy == "private" ? "private, no-cache" : "public, no-cache";
const headers = new Headers({ const headers = new Headers({
ETag: etag, ETag: etag,
"Cache-Control": cache_control, /**
* Private: the response depends on who is signed in, so no shared
* cache may hold it.
*/
"Cache-Control": "private, no-cache",
}); });
if (req.headers.get("if-none-match") === etag) { if (req.headers.get("if-none-match") === etag) {
+30 -10
View File
@@ -4,10 +4,10 @@ import path from "node:path";
import { z } from "zod"; import { z } from "zod";
import { AppData } from "@/src/data/app-data"; import { AppData } from "@/src/data/app-data";
import enqueueFileIngest from "@/src/file-ingest/enqueue-file-ingest"; import enqueueFileIngest from "@/src/file-ingest/enqueue-file-ingest";
import recordBotFile from "@/src/functions/backend/db/bot-files/record-bot-file"; import recordBotFile, { grabBotFileRow } from "@/src/functions/backend/db/bot-files/record-bot-file";
import grabComputerMcpUrl from "@/src/functions/backend/db/computers/grab-computer-mcp-url"; import grabComputerMcpUrl from "@/src/functions/backend/db/computers/grab-computer-mcp-url";
import computerNotReadyMessage from "@/src/utils/computer-not-ready-message"; import computerNotReadyMessage from "@/src/utils/computer-not-ready-message";
import pathRef from "@/src/utils/path-ref"; import folderRef from "@/src/utils/folder-ref";
import type { StorageScope } from "@/src/utils/storage-access"; import type { StorageScope } from "@/src/utils/storage-access";
import type { HarnessTool } from "../../types"; import type { HarnessTool } from "../../types";
import downloadComputerFile, { shellQuote } from "./download-computer-file"; import downloadComputerFile, { shellQuote } from "./download-computer-file";
@@ -39,10 +39,12 @@ type Args = {
* a whole folder (taken as a gzipped tar and unpacked into one row per * a whole folder (taken as a gzipped tar and unpacked into one row per
* file). The copies are kept with the app, stay after the computer is * file). The copies are kept with the app, stay after the computer is
* rebuilt, and are searchable at once; the originals stay on the computer, * rebuilt, and are searchable at once; the originals stay on the computer,
* where the message's path ref points at them. * where the message's folder ref points at the folder holding them.
* *
* The bytes come back through the sandbox's own shell * The bytes come back over the sandbox's streaming download endpoint
* (`runComputerCommand`), base64 encoded, which every sandbox has. * (`downloadComputerFile`), never through its shell, whose output is
* capped: a `base64` round trip through `/v1/shell/exec` silently cut every
* import at 30,047 characters.
*/ */
export default function importFileTool({ export default function importFileTool({
computer_id, computer_id,
@@ -56,7 +58,7 @@ export default function importFileTool({
return { return {
name: "import_file", name: "import_file",
description: `Copy a file or folder from this computer into your own files, so it is kept with the app and stays after the computer is rebuilt. Give its path on the computer, e.g. "seo/coderank/raw/crawl.json" or "seo/coderank/raw"; a folder comes in as it is, in one go. Up to ${max_mb} MB.${ description: `Copy a file or folder from this computer into your own files, so it is kept with the app and stays after the computer is rebuilt. Give its path on the computer, e.g. "seo/coderank/raw/crawl.json" or "seo/coderank/raw"; a folder comes in as it is, in one go. Up to ${max_mb} MB. Check with list_files and the computer's own tools what you already imported: importing the same thing twice just writes over the copy, so do it once.${
own?.scope == "workspace" ? " The copies land in the workspace's shared files." : "" own?.scope == "workspace" ? " The copies land in the workspace's shared files." : ""
} Imports are searchable at once. Name a copy with its [file:ID] ref. Import the report you wrote before the last long steps of a run, so it is never lost to the step limit.`, } Imports are searchable at once. Name a copy with its [file:ID] ref. Import the report you wrote before the last long steps of a run, so it is never lost to the step limit.`,
schema: z.object({ schema: z.object({
@@ -153,6 +155,12 @@ async function importOneFile({
if (!download.ok) return importFailure({ reason: download.reason, size: download.size, path: target.source }); if (!download.ok) return importFailure({ reason: download.reason, size: download.size, path: target.source });
const previous = await grabBotFileRow({
scope: target.scope.scope,
owner_id: target.scope.owner_id,
path: name,
});
const saved = await recordBotFile({ const saved = await recordBotFile({
bot_id: target.bot_id, bot_id: target.bot_id,
scope: target.scope.scope, scope: target.scope.scope,
@@ -171,11 +179,13 @@ async function importOneFile({
} }
return [ return [
`Copied ~/${target.source} into your files as ${saved.path} [file:${saved.id}], ${formatSize({ bytes: download.size })} byte for byte.`, previous?.id
`The original is still on the computer: ${pathRef({ ? `You already had ${saved.path} (same ref, no second copy): its bytes are now this file, ${formatSize({ bytes: download.size })} byte for byte.`
: `Copied ~/${target.source} into your files as ${saved.path} [file:${saved.id}], ${formatSize({ bytes: download.size })} byte for byte.`,
`The original is still on the computer, in ${folderRef({
volume: "computer", volume: "computer",
owner_id: target.computer_id, owner_id: target.computer_id,
path: target.source, path: sourceFolder({ path: target.source }),
})}`, })}`,
"Name the copy with its [file:ID] ref when you tell the user about it.", "Name the copy with its [file:ID] ref when you tell the user about it.",
].join("\n"); ].join("\n");
@@ -215,7 +225,7 @@ async function importFolder({
`Copied ${saved.length} file(s) from ~/${target.source} into your files under ${name}/:`, `Copied ${saved.length} file(s) from ~/${target.source} into your files under ${name}/:`,
...saved.slice(0, 20).map((file) => `- ${file.path} [file:${file.id}]`), ...saved.slice(0, 20).map((file) => `- ${file.path} [file:${file.id}]`),
saved.length > 20 ? `(and ${saved.length - 20} more)` : "", saved.length > 20 ? `(and ${saved.length - 20} more)` : "",
`The originals are still on the computer: ${pathRef({ `The originals are still on the computer, in ${folderRef({
volume: "computer", volume: "computer",
owner_id: computer_id, owner_id: computer_id,
path: target.source, path: target.source,
@@ -226,6 +236,16 @@ async function importFolder({
.join("\n"); .join("\n");
} }
/**
* The folder holding a path, for a folder ref: a ref is always a folder, so
* a file's ref points at the directory the user can open to see it.
*/
function sourceFolder({ path: source }: { path: string }) {
const segments = source.split("/").filter(Boolean);
return segments.slice(0, -1).join("/");
}
/** /**
* Whether the path is a file, a folder, or not there at all. * Whether the path is a file, a folder, or not there at all.
*/ */
+31 -3
View File
@@ -1,6 +1,7 @@
import { z } from "zod"; import { z } from "zod";
import { AppData } from "@/src/data/app-data"; import { AppData } from "@/src/data/app-data";
import listBotFiles from "@/src/functions/backend/db/bot-files/list-bot-files"; import listBotFiles from "@/src/functions/backend/db/bot-files/list-bot-files";
import folderRef from "@/src/utils/folder-ref";
import type { HarnessTool } from "../../types"; import type { HarnessTool } from "../../types";
import { fileRef, formatBytes, type StorageToolContext } from "./storage-tool-context"; import { fileRef, formatBytes, type StorageToolContext } from "./storage-tool-context";
@@ -13,12 +14,15 @@ type Args = {
/** /**
* Lists a bot's files: one folder's, or everything under it, or whatever * Lists a bot's files: one folder's, or everything under it, or whatever
* matches a name. Each file comes with its ref so the bot can read it. * matches a name. Each file comes with its `[file:ID]` ref so the bot can
* read it, and the answer names the folder it is in with a folder ref, so a
* bot that points at "everything in reports/" writes something the user can
* actually open.
*/ */
export default function listFilesTool(context: StorageToolContext): HarnessTool<Args> { export default function listFilesTool(context: StorageToolContext): HarnessTool<Args> {
return { return {
name: "list_files", name: "list_files",
description: `List the files you have saved, newest first: one folder's with "folder", everything with "recursive", or the ones whose name or path has "search". Each line has its path, size and ref (e.g. [file:12]) to read it.`, description: `List the files you have saved, newest first: one folder's with "folder", everything with "recursive", or the ones whose name or path has "search". Each line has its path, size and [file:ID] ref (e.g. [file:12]) to read it. The answer also names the folder as a folder ref, which opens it in Files without downloading anything: a file is a [file:ID] ref, a folder is a [folder:...] ref.`,
schema: z.object({ schema: z.object({
folder: z.string().max(AppData.BotFilePathMaxChars).optional(), folder: z.string().max(AppData.BotFilePathMaxChars).optional(),
search: z.string().max(120).optional(), search: z.string().max(120).optional(),
@@ -56,7 +60,31 @@ export default function listFilesTool(context: StorageToolContext): HarnessTool<
? `\n(Showing the first ${AppData.BotFileListMaxItems}. Narrow it with "folder" or "search".)` ? `\n(Showing the first ${AppData.BotFileListMaxItems}. Narrow it with "folder" or "search".)`
: ""; : "";
return `${files.length} file(s):\n${lines.join("\n")}${more}`; return `${files.length} file(s):\n${lines.join("\n")}${more}\n\nThe folder: ${folderRefFor({ files, folder, scopes: context.scopes })}`;
}, },
}; };
} }
/**
* The folder the listing is about, as a folder ref the bot can put in a
* message: the one it asked for, else the one the first file is in. Its
* volume is the scope the file lives in.
*/
function folderRefFor({
files,
folder,
scopes,
}: {
files: { scope?: string | null; owner_id?: number | string | null; folder?: string | null }[];
folder: string;
scopes: StorageToolContext["scopes"];
}) {
const first = files[0];
const scope = first
? scopes.find((it) => it.scope == first.scope && Number(it.owner_id) == Number(first.owner_id))
: scopes[0];
const path = folder || first?.folder || "";
const volume = scope?.scope == "workspace" ? "workspace" : "bot";
return folderRef({ volume, owner_id: scope?.owner_id || 0, path });
}
@@ -47,7 +47,9 @@ function fetchFile({
/** /**
* A file a bot named in a message (`fileRef`): the name it is saved under, * A file a bot named in a message (`fileRef`): the name it is saved under,
* linked to its bytes. The conversation's files come with the page, so most * linked to its bytes at `/media/<id>` (private: only someone who reaches
* the file's folder can load it). A folder the bot named is a `folderRef`
* instead, and its chip opens the files browser. The conversation's files come with the page, so most
* chips need no request; one that isn't among them (older than the page's * chips need no request; one that isn't among them (older than the page's
* first page) is fetched by id. A file the user doesn't reach, or one * first page) is fetched by id. A file the user doesn't reach, or one
* deleted since, shows as its id. * deleted since, shows as its id.
@@ -0,0 +1,38 @@
import Link from "@moduletrace/twui/layout/Link";
import Span from "@moduletrace/twui/layout/Span";
import { FolderOpen } from "lucide-react";
import { folderRefUrl } from "@/src/utils/folder-ref";
import { storagePathName, type StorageSpec } from "@/src/utils/storage-volumes";
type Props = {
spec: StorageSpec;
path: string;
};
/**
* A folder a bot named in a message (`folderRef`): the name it ends with,
* linked into the files browser, where the user can look at everything in
* it. Nothing is downloaded: a folder ref never serves bytes, and this chip
* never guesses from the name whether it is a file or a folder, because the
* ref itself already says it is a folder. A file the bot saved is a file
* ref (`fileRef`) and opens the file instead.
*
* Written without a literal ref in this comment on purpose: Tailwind scans
* every page and partial, reads a bracketed `name:value` as an
* arbitrary-property utility, and emits it as a real CSS property (esbuild
* then warns that `file` is not one).
*/
export default function FolderRefChip({ spec, path }: Props) {
const name = storagePathName({ volume: spec.volume, path });
return (
<Link
href={folderRefUrl({ volume: spec.volume, owner_id: spec.owner_id, path })}
title={`Open ${path || name} in Files`}
className="inline-flex h-6 max-w-full gap-1.5 items-center px-1.5 align-middle rounded-default-xs border-0 bg-sunken text-[13px] font-bold leading-4 text-ink hover:opacity-80"
>
<FolderOpen size={14} aria-hidden className="shrink-0" />
<Span className="truncate">{name}</Span>
</Link>
);
}
@@ -8,11 +8,11 @@ import {
TakeOverRefHrefPrefix, TakeOverRefHrefPrefix,
WorkspaceRefHrefPrefix, WorkspaceRefHrefPrefix,
} from "@/src/utils/link-computer-refs"; } from "@/src/utils/link-computer-refs";
import { parsePathRefHref } from "@/src/utils/path-ref"; import { parseFolderRefHref } from "@/src/utils/folder-ref";
import BotRefChip from "./bot-ref-chip"; import BotRefChip from "./bot-ref-chip";
import ComputerRefChip from "./computer-ref-chip"; import ComputerRefChip from "./computer-ref-chip";
import FileRefChip from "./file-ref-chip"; import FileRefChip from "./file-ref-chip";
import PathRefChip from "./path-ref-chip"; import FolderRefChip from "./folder-ref-chip";
import TakeOverChip from "./take-over-chip"; import TakeOverChip from "./take-over-chip";
import WorkspaceRefChip from "./workspace-ref-chip"; import WorkspaceRefChip from "./workspace-ref-chip";
@@ -20,8 +20,9 @@ import WorkspaceRefChip from "./workspace-ref-chip";
* A link in a bot message. A computer the bot named (`linkComputerRefs`) * A link in a bot message. A computer the bot named (`linkComputerRefs`)
* becomes its switch button, a take-over request a button that opens the * becomes its switch button, a take-over request a button that opens the
* computer, a bot it named a link to that bot's conversation, a workspace * computer, a bot it named a link to that bot's conversation, a workspace
* a link to that workspace, a file it saved a chip that opens it; anything * a link to that workspace, a file it saved a chip that downloads it, a
* else opens in a new tab. * folder it named a chip that opens it in the files browser; anything else
* opens in a new tab.
*/ */
export default function MarkdownLink({ href, children }: ComponentProps<"a">) { export default function MarkdownLink({ href, children }: ComponentProps<"a">) {
if (href?.startsWith(ComputerRefHrefPrefix)) { if (href?.startsWith(ComputerRefHrefPrefix)) {
@@ -63,9 +64,9 @@ export default function MarkdownLink({ href, children }: ComponentProps<"a">) {
if (file_id) return <FileRefChip file_id={file_id} />; if (file_id) return <FileRefChip file_id={file_id} />;
} }
const path_ref = parsePathRefHref({ href }); const folder_ref = parseFolderRefHref({ href });
if (path_ref) return <PathRefChip spec={path_ref.spec} path={path_ref.path} />; if (folder_ref) return <FolderRefChip spec={folder_ref.spec} path={folder_ref.path} />;
return ( return (
<Link <Link
@@ -1,36 +0,0 @@
import Link from "@moduletrace/twui/layout/Link";
import Span from "@moduletrace/twui/layout/Span";
import { FolderOpen, Paperclip } from "lucide-react";
import { pathRefUrl } from "@/src/utils/path-ref";
import { storagePathName, type StorageSpec } from "@/src/utils/storage-volumes";
type Props = {
spec: StorageSpec;
path: string;
};
/**
* A folder or file a bot named in a message (`pathRef`): the name it ends
* with, linked into the files browser, where the user can look at
* everything around it. That is how evidence a bot left on a computer is
* reached without copying it into its own files first.
*/
export default function PathRefChip({ spec, path }: Props) {
const name = storagePathName({ volume: spec.volume, path });
const is_folder = !/\.[a-z0-9]{1,8}$/i.test(name);
return (
<Link
href={pathRefUrl({ volume: spec.volume, owner_id: spec.owner_id, path })}
title={`Open ${path || name} in Files`}
className="inline-flex h-6 max-w-full gap-1.5 items-center px-1.5 align-middle rounded-default-xs border-0 bg-sunken text-[13px] font-bold leading-4 text-ink hover:opacity-80"
>
{is_folder ? (
<FolderOpen size={14} aria-hidden className="shrink-0" />
) : (
<Paperclip size={14} aria-hidden className="shrink-0" />
)}
<Span className="truncate">{name}</Span>
</Link>
);
}
@@ -65,7 +65,6 @@ export const handler: BunextAPIRouteHandler<
media_paradigms: ["bot-attachment"], media_paradigms: ["bot-attachment"],
media: { title: name }, media: { title: name },
extract_media_text_content: media_type == "document", extract_media_text_content: media_type == "document",
is_private: true,
user: { id: Number(user.id) }, user: { id: Number(user.id) },
}).catch(async (error) => { }).catch(async (error) => {
await logError({ title: `Attachment ${name} couldn't be saved`, error }).catch(() => {}); await logError({ title: `Attachment ${name} couldn't be saved`, error }).catch(() => {});
@@ -28,7 +28,11 @@ export default async function (
}; };
} }
const GET = await grabMedia({ query, user }); /**
* A non-admin only ever lists their own uploads: the filter above is
* the one that is queried, so it can't be bypassed by the request.
*/
const GET = await grabMedia({ query: final_sql_query as typeof query, user });
return GET; return GET;
} }
-1
View File
@@ -66,7 +66,6 @@ export const handler: BunextAPIRouteHandler<APIResponseObject<TaskDetail>> = asy
media_paradigms: ["task-media"], media_paradigms: ["task-media"],
media: { title: name }, media: { title: name },
extract_media_text_content: media_type == "document", extract_media_text_content: media_type == "document",
is_private: true,
user: { id: Number(user.id) }, user: { id: Number(user.id) },
}); });
-2
View File
@@ -46,7 +46,6 @@ export const handler: BunextAPIRouteHandler<APIResponseObject> = async (
reauth, reauth,
return_media_text_content, return_media_text_content,
media, media,
is_media_private,
media_name, media_name,
} = body; } = body;
@@ -105,7 +104,6 @@ export const handler: BunextAPIRouteHandler<APIResponseObject> = async (
media, media,
media_mime_type: ext, media_mime_type: ext,
user: { id: target_user_id }, user: { id: target_user_id },
is_private: is_media_private,
media_name, media_name,
}); });
+1 -2
View File
@@ -717,7 +717,7 @@ export type PublicBotFile = {
ingest_error?: string; ingest_error?: string;
created_at: number; created_at: number;
/** /**
* Where the bytes are served (`/media/private/<media id>`). * Where the bytes are served (`/media/<media id>`, private).
*/ */
url: string; url: string;
}; };
@@ -1315,7 +1315,6 @@ export type ApiReqParams<
media_paradigm?: MediaParadigmType; media_paradigm?: MediaParadigmType;
media_type?: (typeof MediaTypes)[number]["value"]; media_type?: (typeof MediaTypes)[number]["value"];
media_name?: string; media_name?: string;
is_media_private?: boolean;
is_media_primary?: boolean; is_media_primary?: boolean;
reauth?: boolean; reauth?: boolean;
+5 -1
View File
@@ -1,7 +1,11 @@
/** /**
* How a bot names one of its files in a message: `[file:12]`. The chat * How a bot names one of its files in a message: `[file:12]`. The chat
* turns it into a link to the file (`linkComputerRefs`), which a chip * turns it into a link to the file (`linkComputerRefs`), which a chip
* renders with the file's name. * renders with the file's name and opens the bytes at `/media/<id>`.
*
* A file ref is always a file. A folder is named with a `folderRef`
* (`[folder:bot:3:reports]`), which opens the files browser instead and
* never downloads anything.
*/ */
export default function fileRef({ id }: { id: number | string }) { export default function fileRef({ id }: { id: number | string }) {
return `[file:${id}]`; return `[file:${id}]`;
+77
View File
@@ -0,0 +1,77 @@
import { describe, expect, test } from "bun:test";
import folderRef, { folderRefHref, folderRefUrl, parseFolderRefHref } from "./folder-ref";
import linkComputerRefs from "./link-computer-refs";
describe("folderRef", () => {
test("writes a folder, never a file", () => {
expect(folderRef({ volume: "computer", owner_id: 3, path: "seo/coderank/raw" })).toBe(
"[folder:computer:3:seo/coderank/raw]",
);
expect(folderRef({ volume: "computer", owner_id: 3, path: "/seo/raw/ " })).toBe(
"[folder:computer:3:seo/raw]",
);
expect(folderRef({ volume: "bot", owner_id: 1 })).toBe("[folder:bot:1]");
});
});
describe("folderRefHref", () => {
test("encodes the path into the link", () => {
expect(folderRefHref({ volume: "computer", owner_id: 3, path: "seo/raw mes" })).toBe(
`#folder-${encodeURIComponent("computer:3:seo/raw mes")}`,
);
});
test("reads back what it wrote", () => {
const href = folderRefHref({ volume: "computer", owner_id: 3, path: "seo/raw mes/#1" });
expect(parseFolderRefHref({ href })).toEqual({
spec: { volume: "computers", owner_id: 3 },
path: "seo/raw mes/#1",
});
});
test("maps the ref's volume name to the browser's", () => {
expect(parseFolderRefHref({ href: folderRefHref({ volume: "bot", owner_id: 1, path: "a" }) })).toEqual({
spec: { volume: "bots", owner_id: 1 },
path: "a",
});
expect(parseFolderRefHref({ href: folderRefHref({ volume: "workspace", owner_id: 2 }) })).toEqual({
spec: { volume: "workspaces", owner_id: 2 },
path: "",
});
});
test("refuses an unknown volume, a bad owner, a file ref or nothing", () => {
expect(parseFolderRefHref({ href: "#folder-secrets:1:a" })).toBeNull();
expect(parseFolderRefHref({ href: "#folder-computer:abc:a" })).toBeNull();
expect(parseFolderRefHref({ href: "#file-12" })).toBeNull();
expect(parseFolderRefHref({ href: undefined })).toBeNull();
});
});
describe("folderRefUrl", () => {
test("opens the files browser on the folder", () => {
expect(folderRefUrl({ volume: "computer", owner_id: 3, path: "seo/raw" })).toBe(
`/admin/files?volume=computers&owner=3&path=seo%2Fraw`,
);
});
});
describe("linkComputerRefs", () => {
test("turns a folder ref into a link", () => {
const linked = linkComputerRefs({ text: "In [folder:computer:3:seo/raw]." });
expect(linked).toContain("(#folder-");
expect(parseFolderRefHref({ href: linked.match(/\((#folder-[^)]+)\)/)?.[1] })).toEqual({
spec: { volume: "computers", owner_id: 3 },
path: "seo/raw",
});
});
test("leaves a file ref to the file chip", () => {
const linked = linkComputerRefs({ text: "The report: [file:12]." });
expect(linked).toContain("(#file-12)");
expect(parseFolderRefHref({ href: linked.match(/\((#file-[^)]+)\)/)?.[1] })).toBeNull();
});
});
+106
View File
@@ -0,0 +1,106 @@
/**
* How a bot names a folder on the app's data volume in a message:
* `[folder:bot:3:reports]`, `[folder:computer:3:seo/coderank/raw]`,
* `[folder:workspace:2]`.
*
* A folder ref is always a folder: the chat turns it into a chip that opens
* the files browser (`/admin/files`) at that folder, and nothing is
* downloaded. A file the bot saved is named with a `[file:ID]` ref instead
* (`fileRef`), which is what downloads. There is no guessing from the
* name's extension: the ref says which one it is.
*/
import {
StorageBrowserPath,
storageSpecHref,
storageVolumeFromRef,
type StorageSpec,
} from "./storage-volumes";
/**
* The link target a folder ref becomes, e.g.
* `#folder-bot%3A3%3Areports`.
*/
export const FolderRefHrefPrefix = "#folder-";
type RefParams = {
/**
* The volume by the name a ref uses: `bot`, `workspace`, `computer`,
* `media`, `mcp` or `db`.
*/
volume: string;
owner_id: number | string;
path?: string | null;
};
function cleanPath({ path }: { path?: string | null }) {
return (path || "").trim().replace(/^\/+|\/+$/g, "");
}
/**
* How a bot names a folder in a message. The path is written as it is; the
* chat encodes it into the link.
*/
export default function folderRef({ volume, owner_id, path }: RefParams) {
const clean = cleanPath({ path });
return clean
? `[folder:${volume}:${owner_id}:${clean}]`
: `[folder:${volume}:${owner_id}]`;
}
/**
* The href a folder ref becomes: everything after `#folder-`, URI encoded,
* so spaces, `#` and `:` in a path survive markdown.
*/
export function folderRefHref({ volume, owner_id, path }: RefParams) {
const clean = cleanPath({ path });
return `${FolderRefHrefPrefix}${encodeURIComponent(
clean ? `${volume}:${owner_id}:${clean}` : `${volume}:${owner_id}`,
)}`;
}
/**
* Reads a folder ref's link back into a folder and a path inside it, or
* null when it names a volume that isn't one (or has no owner).
*/
export function parseFolderRefHref({
href,
}: {
href?: string | null;
}): { spec: StorageSpec; path: string } | null {
if (!href?.startsWith(FolderRefHrefPrefix)) return null;
let decoded = "";
try {
decoded = decodeURIComponent(href.slice(FolderRefHrefPrefix.length));
} catch {
return null;
}
const [name, raw_owner, ...rest] = decoded.split(":");
const volume = storageVolumeFromRef({ name });
const owner_id = Number(raw_owner);
if (!volume) return null;
if (!Number.isFinite(owner_id) || owner_id < 0) return null;
return {
spec: { volume, owner_id: Math.trunc(owner_id) },
path: rest.join(":").replace(/^\/+|\/+$/g, ""),
};
}
/**
* Where a folder ref opens: the files browser, on that folder. A volume the
* browser doesn't know lands on its first folder.
*/
export function folderRefUrl({ volume, owner_id, path }: RefParams) {
const known = storageVolumeFromRef({ name: volume });
if (!known) return StorageBrowserPath;
return storageSpecHref({ spec: { volume: known, owner_id: Number(owner_id) }, path });
}
+10 -14
View File
@@ -16,26 +16,22 @@ export default function grabDirNames(params?: Params) {
const MEDIA_RELATIVE_DIR = "media"; const MEDIA_RELATIVE_DIR = "media";
const MEDIA_DIR = path.join(DATA_DIR, MEDIA_RELATIVE_DIR); const MEDIA_DIR = path.join(DATA_DIR, MEDIA_RELATIVE_DIR);
const USER_MEDIA_PUBLIC_DIR = params?.user?.id /**
? path.join(MEDIA_DIR, String(params.user.id), "public") * A user's uploads: one directory, every file private (there is no
* public media any more).
*/
const USER_MEDIA_DIR = params?.user?.id
? path.join(MEDIA_DIR, String(params.user.id))
: undefined; : undefined;
const USER_MEDIA_PUBLIC_RELATIVE_DIR = params?.user?.id const USER_MEDIA_RELATIVE_DIR = params?.user?.id
? path.join(MEDIA_RELATIVE_DIR, String(params.user.id), "public") ? path.join(MEDIA_RELATIVE_DIR, String(params.user.id))
: undefined;
const USER_MEDIA_PRIVATE_DIR = params?.user?.id
? path.join(MEDIA_DIR, String(params.user.id), "private")
: undefined;
const USER_MEDIA_PRIVATE_RELATIVE_DIR = params?.user?.id
? path.join(MEDIA_RELATIVE_DIR, String(params.user.id), "private")
: undefined; : undefined;
return { return {
ROOT_DIR, ROOT_DIR,
MEDIA_DIR, MEDIA_DIR,
USER_MEDIA_PUBLIC_DIR, USER_MEDIA_DIR,
USER_MEDIA_PRIVATE_DIR, USER_MEDIA_RELATIVE_DIR,
DATA_DIR, DATA_DIR,
USER_MEDIA_PUBLIC_RELATIVE_DIR,
USER_MEDIA_PRIVATE_RELATIVE_DIR,
}; };
} }
+15 -11
View File
@@ -31,19 +31,23 @@ export const WorkspaceRefHrefPrefix = "#workspace-";
export const FileRefHrefPrefix = "#file-"; export const FileRefHrefPrefix = "#file-";
/** /**
* The link target a path ref (`pathRef`) becomes, e.g. * The link target a folder ref (`folderRef`) becomes, e.g.
* `#path-computer%3A3%3Aseo%2Fraw`. The chat renders it as a chip that * `#folder-computer%3A3%3Aseo%2Fraw`. The chat renders it as a chip that
* opens that folder in the files browser. * opens that folder in the files browser.
*/ */
export const PathRefHrefPrefix = "#path-"; export const FolderRefHrefPrefix = "#folder-";
/** /**
* Turns every computer ref (`[computer:3]`) and take-over ref * Turns every computer ref (`[computer:3]`) and take-over ref
* (`[take-over:3:browser]`), bot ref (`[bot:3]`), workspace ref * (`[take-over:3:browser]`), bot ref (`[bot:3]`), workspace ref
* (`[workspace:2]`), file ref (`[file:12]`) and path ref * (`[workspace:2]`), file ref (`[file:12]`) and folder ref
* (`[path:computer:3:seo/raw]`) in a message into a markdown link, so it * (`[folder:computer:3:seo/raw]`) in a message into a markdown link, so it
* survives markdown rendering as a link the chat can swap for a button or * survives markdown rendering as a link the chat can swap for a button,
* chip. * chip or download.
*
* A file ref and a folder ref are different things and are never guessed
* from a name: `[file:12]` is a saved file (its chip downloads it),
* `[folder:...]` a folder (its chip opens the files browser there).
*/ */
export default function linkComputerRefs({ text }: { text: string }) { export default function linkComputerRefs({ text }: { text: string }) {
return text return text
@@ -61,10 +65,10 @@ export default function linkComputerRefs({ text }: { text: string }) {
(_, id: string) => `[workspace ${id}](${WorkspaceRefHrefPrefix}${id})`, (_, id: string) => `[workspace ${id}](${WorkspaceRefHrefPrefix}${id})`,
) )
.replace(/\[file:(\d+)\](?!\()/g, (_, id: string) => `[file ${id}](${FileRefHrefPrefix}${id})`) .replace(/\[file:(\d+)\](?!\()/g, (_, id: string) => `[file ${id}](${FileRefHrefPrefix}${id})`)
.replace(/\[path:([a-z]+):(\d+):([^\]\n]+)\](?!\()/g, (_, volume: string, owner: string, path: string) => .replace(/\[folder:([a-z]+):(\d+):([^\]\n]+)\](?!\()/g, (_, volume: string, owner: string, path: string) =>
`[${path}](${PathRefHrefPrefix}${encodeURIComponent(`${volume}:${owner}:${path}`)})`, `[${path}](${FolderRefHrefPrefix}${encodeURIComponent(`${volume}:${owner}:${path}`)})`,
) )
.replace(/\[path:([a-z]+):(\d+)\](?!\()/g, (_, volume: string, owner: string) => .replace(/\[folder:([a-z]+):(\d+)\](?!\()/g, (_, volume: string, owner: string) =>
`[${volume} ${owner}](${PathRefHrefPrefix}${encodeURIComponent(`${volume}:${owner}`)})`, `[${volume} ${owner}](${FolderRefHrefPrefix}${encodeURIComponent(`${volume}:${owner}`)})`,
); );
} }
+1 -1
View File
@@ -43,7 +43,7 @@ describe("needsPrecachePage", () => {
expect(needsPrecachePage({ ...page, pathname: "/api/admin/bots/send-message" })).toBe(false); expect(needsPrecachePage({ ...page, pathname: "/api/admin/bots/send-message" })).toBe(false);
expect(needsPrecachePage({ ...page, pathname: "/.bunext/public/vendor/react.js" })).toBe(false); expect(needsPrecachePage({ ...page, pathname: "/.bunext/public/vendor/react.js" })).toBe(false);
expect(needsPrecachePage({ ...page, pathname: "/computers/2/vnc/vnc.html" })).toBe(false); expect(needsPrecachePage({ ...page, pathname: "/computers/2/vnc/vnc.html" })).toBe(false);
expect(needsPrecachePage({ ...page, pathname: "/media/1/private/a" })).toBe(false); expect(needsPrecachePage({ ...page, pathname: "/media/1/thumbnail" })).toBe(false);
expect(needsPrecachePage({ ...page, pathname: "/build-info.json" })).toBe(false); expect(needsPrecachePage({ ...page, pathname: "/build-info.json" })).toBe(false);
expect(needsPrecachePage({ ...page, pathname: "/sw.js" })).toBe(false); expect(needsPrecachePage({ ...page, pathname: "/sw.js" })).toBe(false);
}); });
-72
View File
@@ -1,72 +0,0 @@
import { describe, expect, test } from "bun:test";
import pathRef, { parsePathRefHref, pathRefHref, pathRefUrl } from "./path-ref";
import linkComputerRefs from "./link-computer-refs";
describe("pathRef", () => {
test("writes a folder and a file", () => {
expect(pathRef({ volume: "computer", owner_id: 3, path: "seo/coderank/raw" })).toBe(
"[path:computer:3:seo/coderank/raw]",
);
expect(pathRef({ volume: "computer", owner_id: 3, path: "/seo/raw.json " })).toBe(
"[path:computer:3:seo/raw.json]",
);
expect(pathRef({ volume: "bot", owner_id: 1 })).toBe("[path:bot:1]");
});
});
describe("pathRefHref", () => {
test("encodes the path into the link", () => {
expect(pathRefHref({ volume: "computer", owner_id: 3, path: "seo/raw mes" })).toBe(
`#path-${encodeURIComponent("computer:3:seo/raw mes")}`,
);
});
test("reads back what it wrote", () => {
const href = pathRefHref({ volume: "computer", owner_id: 3, path: "seo/raw mes/#1" });
expect(parsePathRefHref({ href })).toEqual({
spec: { volume: "computers", owner_id: 3 },
path: "seo/raw mes/#1",
});
});
test("maps the ref's volume name to the browser's", () => {
expect(parsePathRefHref({ href: pathRefHref({ volume: "bot", owner_id: 1, path: "a" }) })).toEqual({
spec: { volume: "bots", owner_id: 1 },
path: "a",
});
expect(
parsePathRefHref({ href: pathRefHref({ volume: "workspace", owner_id: 2 }) }),
).toEqual({
spec: { volume: "workspaces", owner_id: 2 },
path: "",
});
});
test("refuses an unknown volume or owner", () => {
expect(parsePathRefHref({ href: "#path-secrets:1:a" })).toBeNull();
expect(parsePathRefHref({ href: "#path-computer:abc:a" })).toBeNull();
expect(parsePathRefHref({ href: "#file-12" })).toBeNull();
expect(parsePathRefHref({ href: undefined })).toBeNull();
});
});
describe("pathRefUrl", () => {
test("opens the files browser on the folder", () => {
expect(pathRefUrl({ volume: "computer", owner_id: 3, path: "seo/raw" })).toBe(
`/admin/files?volume=computers&owner=3&path=seo%2Fraw`,
);
});
});
describe("linkComputerRefs", () => {
test("turns a path ref into a link", () => {
const linked = linkComputerRefs({ text: "In [path:computer:3:seo/raw]." });
expect(linked).toContain("(#path-");
expect(parsePathRefHref({ href: linked.match(/\((#path-[^)]+)\)/)?.[1] })).toEqual({
spec: { volume: "computers", owner_id: 3 },
path: "seo/raw",
});
});
});
-85
View File
@@ -1,85 +0,0 @@
/**
* Where a bot names a folder or file on the app's data volume in a
* message: `[path:computer:3:seo/coderank/raw]`. The chat turns it into a
* chip that opens it in the files browser (`/admin/files`), which is how a
* bot points the user at something it left on a computer (raw evidence,
* a crawl, a report it never copied) without importing it first.
*/
import { StorageBrowserPath, storageSpecHref, storageVolumeFromRef, type StorageSpec } from "./storage-volumes";
/**
* The link target a path ref becomes, e.g. `#path-computer%3A3%3Aseo%2Fraw`.
*/
export const PathRefHrefPrefix = "#path-";
type RefParams = {
/**
* The volume by the name a ref uses: `bot`, `workspace`, `computer`,
* `media`, `mcp` or `db`.
*/
volume: string;
owner_id: number | string;
path?: string | null;
};
/**
* How a bot names a folder or file in a message. The path is written as it
* is; the chat encodes it into the link.
*/
export default function pathRef({ volume, owner_id, path }: RefParams) {
const clean = (path || "").trim().replace(/^\/+|\/+$/g, "");
return clean
? `[path:${volume}:${owner_id}:${clean}]`
: `[path:${volume}:${owner_id}]`;
}
/**
* The href a path ref becomes: everything after `#path-`, URI encoded, so
* spaces, `#` and `:` in a path survive markdown.
*/
export function pathRefHref({ volume, owner_id, path }: RefParams) {
const clean = (path || "").trim().replace(/^\/+|\/+$/g, "");
return `${PathRefHrefPrefix}${encodeURIComponent(
clean ? `${volume}:${owner_id}:${clean}` : `${volume}:${owner_id}`,
)}`;
}
/**
* Reads a path ref's link back into a folder and a path inside it, or null
* when it names a volume that isn't one (or has no owner).
*/
export function parsePathRefHref({
href,
}: {
href?: string | null;
}): { spec: StorageSpec; path: string } | null {
if (!href?.startsWith(PathRefHrefPrefix)) return null;
const decoded = decodeURIComponent(href.slice(PathRefHrefPrefix.length));
const [name, raw_owner, ...rest] = decoded.split(":");
const volume = storageVolumeFromRef({ name });
const owner_id = Number(raw_owner);
if (!volume) return null;
if (!Number.isFinite(owner_id) || owner_id < 0) return null;
return {
spec: { volume, owner_id: Math.trunc(owner_id) },
path: rest.join("/").replace(/^\/+|\/+$/g, ""),
};
}
/**
* Where a path ref opens: the files browser, on that folder. A volume the
* browser doesn't know lands on its first folder.
*/
export function pathRefUrl({ volume, owner_id, path }: RefParams) {
const known = storageVolumeFromRef({ name: volume });
if (!known) return StorageBrowserPath;
return storageSpecHref({ spec: { volume: known, owner_id: Number(owner_id) }, path });
}
+2 -2
View File
@@ -4,8 +4,8 @@ import profileImageUrls from "./profile-image-urls";
describe("profileImageUrls", () => { describe("profileImageUrls", () => {
test("serves a picture privately, with a thumbnail", () => { test("serves a picture privately, with a thumbnail", () => {
expect(profileImageUrls({ media_id: 7 })).toEqual({ expect(profileImageUrls({ media_id: 7 })).toEqual({
src: "/media/private/7", src: "/media/7",
thumbnail_src: "/media/private/7/thumbnail", thumbnail_src: "/media/7/thumbnail",
}); });
}); });
+4 -3
View File
@@ -6,13 +6,14 @@ type Params = {
}; };
/** /**
* Where a bot's or workspace's picture is served (a private image, so * Where a bot's or workspace's picture is served (`/media/<id>`, private,
* only its owner can load it). Empty when it has none. * so only someone who reaches the bot or workspace can load it). Empty when
* it has none.
*/ */
export default function profileImageUrls({ media_id }: Params) { export default function profileImageUrls({ media_id }: Params) {
if (!media_id) return {}; if (!media_id) return {};
const src = `/media/private/${media_id}`; const src = `/media/${media_id}`;
return { src, thumbnail_src: `${src}/thumbnail` }; return { src, thumbnail_src: `${src}/thumbnail` };
} }
+1 -1
View File
@@ -56,7 +56,7 @@ describe("storageSpecHref", () => {
}); });
describe("volume names", () => { describe("volume names", () => {
test("maps a path ref's name to its volume", () => { test("maps a folder ref's name to its volume", () => {
expect(storageVolumeFromRef({ name: "computer" })).toBe("computers"); expect(storageVolumeFromRef({ name: "computer" })).toBe("computers");
expect(storageVolumeFromRef({ name: "bot" })).toBe("bots"); expect(storageVolumeFromRef({ name: "bot" })).toBe("bots");
expect(storageVolumeFromRef({ name: "nope" })).toBeNull(); expect(storageVolumeFromRef({ name: "nope" })).toBeNull();
+2 -2
View File
@@ -12,7 +12,7 @@ export const StorageVolumeDefs = [
value: "bots", value: "bots",
title: "Bots", title: "Bots",
/** /**
* How a path ref names it (`[path:bot:3:reports]`). * How a folder ref names it (`[folder:bot:3:reports]`).
*/ */
ref: "bot", ref: "bot",
owner_kind: "bot", owner_kind: "bot",
@@ -84,7 +84,7 @@ export function isStorageVolume(volume?: string | null): volume is StorageVolume
} }
/** /**
* The volume a path ref names (`bot`, `computer`, ...), or null. * The volume a folder ref names (`bot`, `computer`, ...), or null.
*/ */
export function storageVolumeFromRef({ name }: { name?: string | null }): StorageVolume | null { export function storageVolumeFromRef({ name }: { name?: string | null }): StorageVolume | null {
const def = StorageVolumeDefs.find((it) => it.ref == name); const def = StorageVolumeDefs.find((it) => it.ref == name);