Let bots set a picture from a file they can see
A coordinator asked to set an uploaded logo as a workspace picture could not: the picture tools take a media id, and nothing ever gave the bot one. Attachments reached the model as "[Attached image \"logo.png\"]", every file tool answers with a [file:ID] ref, and an SVG is deliberately stored as a document rather than an image. - copyMediaAsPicture copies an approved image into a new picture row (bot-image/workspace-image). The source keeps its row and bytes, so a message that attached it still loads and changing a picture can never take a file away from a conversation. - grabPictureSource resolves a picture by [file:ID] (the same scope gate every file tool uses) or by the user's own upload; parseFileRef reads the [file:12] form. - The new set_picture tool ships on every run: a bot sets its own face, the coordinator can also dress a worker. update_workspace gained picture_file_id, and setBotPicture/setProfileImage share the copy and the delete. - removeReplacedPicture deletes a replaced picture only when it really is one, never a chat upload. - uploadAndRecordMedia writes images in their own format: PNG stays PNG with its transparency, WebP stays WebP, GIF becomes PNG, everything else JPEG, and the row records the real mime type (it used to say jpeg for every image). - Pictures change live: BotUpdatedData carries image_media_id and a new server:workspace-updated pushes workspace pictures, relayed between processes. - Attachments are labelled with their media id, and BotFilesNote tells a bot that a [file:ID] can become a picture. - grabRunScopes is now the single source of a run's reachable folders, shared by the file tools, imports and the picture tools.
This commit is contained in:
@@ -107,7 +107,7 @@ Inspect live data with `bun llm-code-assistant/query-db.ts "<sql>"` - a read-onl
|
||||
- **Replies and reactions:** every bot message has a row under it (`(partials)/message-actions.tsx`, handlers from `BotMessageActionsContext`) to reply or react; the user's own messages only show the bot's reactions, read-only. Replying quotes the message above the composer (`composer-reply-preview.tsx`) and the sent one carries `bot_messages.reply_to_message_id` (not a foreign key: a deleted quote only drops it); `send-message` takes any message of the same conversation (the user's, same bot and workspace, any session, so a rollover or a moved-on thread still sends), and the conversation drops a picked quote when its session changes. Messages carry `reply_to` (`grabMessagesReplies`, also on relayed messages), shown by `message-reply-quote.tsx`, and the bot reads it as `[Replying to ...]` before the user's turn (`botMessagesToHarness`). Reactions are a fixed set (`BotMessageReactions` in `src/dict/bot-dict.ts`) in `bot_message_reactions` (one row per message, emoji and user, or bot); `api/admin/bots/react-message.ts` toggles the user's own on a bot message in their sessions (`grabUserBotMessage`, `toggleMessageReaction`), and every bot gets `react_to_message` (`src/harness/tools/bot/react-to-message-tool.ts`, chat replies and tasks: a message in the session it answers, the user's latest by default). Both push the message's fresh reactions (`grabMessagesReactions`, `mine` for the session user) to the session's user as `server:bot-reaction` (`BotMessageReactionData`), which replaces the message's own; `grabBotMessages` adds them when given `user_id`. Reactions aren't relayed between processes (production runs one).
|
||||
- **Attachments and thinking:** the composer uploads files as they're picked, pasted or dropped (`use-composer-attachments`), shows them as chips, and sends once none is still uploading; the thread shows images as thumbnails and other files as chips. Replies read the files of the last `BotReplyAttachmentMessages` messages with any (`load-attachment-parts.ts`: images and audio as parts, documents as their text up to `BotAttachmentMaxTextChars`, videos as a note); older ones are only named. Tasks don't get the files yet. While the user has the last word in a session (every user message queues a reply, so this needs no face and shows even while the bot works in another session) or the bot waits for a model (`BotWaitingStatusLine`) or a task for its computer (`BotComputerBusyStatusLine`), `BotThinkingRow` sits under the last message; a message still on its way to the server reads "<name> is getting your message…" (`BotGettingMessageStatusLine`), while a task runs it reads "<name> is working…" with the task's latest step, and a reply working through tools shows its latest step under "is thinking…" (`use-reply-progress`). A coordinator with no workers gets `CoordinatorSoloNote` instead of `CoordinatorReplyNote`, so it never claims to have handed work to a worker. Live notices come from the process that ran the job at once, and from every other process through the live relay.
|
||||
- **Voice:** the composer's mic (`(partials)/composer-voice-button.tsx`, `use-voice-recorder`, shown once the browser can record, disabled with a note while transcription has no model to run on for the bot there, `bot_voice` page prop, from `resolveHarnessCandidates`) dictates: the recording goes to `api/admin/bots/transcribe.ts` (`transcribeAudio` with the bot's models, after `grabUserConversation`) and the text lands in the input at the cursor, to edit and send; Escape cancels, `AppData.ComposerVoiceMaxMs` stops it. Each bot message has a play button (disabled with a note while speech has no model) (`speak-message-button.tsx`, one player per conversation via `SpeakMessageContext` / `use-speak-message`) that plays `GET api/admin/bots/speak-message.ts?message_id=` straight in an `<audio>` (`speakText` streamed through as the provider makes it, so it starts in a couple of seconds; a failure answers in JSON, which the hook fetches again to show; only the user's own sessions, `grabSpeakableMessage`; text from `speechText`, cut at `AppData.SpeechMaxChars`). Under Speech, every model settings form has "Hear a sample" (`settings/models/(partials)/harness-voice-preview.tsx`), which plays `AppData.VoicePreviewText` in the form's model and voice before they're saved (`GET api/admin/harness/preview-voice`, super admins and admins, that model only via `grabModelCandidate`, streamed). Neither recordings nor audio are stored. Harness errors (`HarnessError`, safe to show) come back as the buttons' titles.
|
||||
- **Pictures:** a bot or workspace can have a picture (`image_media_id`, a private `media` row, paradigm `bot-image`/`workspace-image`), set or removed through `api/admin/set-profile-image.ts` (`setProfileImage`, the old one deleted; served at `profileImageUrls`). `BotAvatar` shows a bot's picture as a rounded square, else its stone; its colour still applies. `WorkspaceAvatar` shows a workspace's, else the workspace icon. A bot is edited from the header avatar (`(partials)/bot-profile-modal.tsx`, the same `BotForm` as adding one, `api/admin/bots/update-bot.ts`; the coordinator keeps its name and stays hue-less).
|
||||
- **Pictures:** a bot or workspace can have a picture (`image_media_id`, a private `media` row, paradigm `bot-image`/`workspace-image`), set or removed through `api/admin/set-profile-image.ts` (`setProfileImage`) or, from chat, by the bots themselves through the `set_picture` tool (`src/harness/tools/bot/set-picture-tool.ts`, every run: a bot sets its own face, the coordinator any worker) and `update_workspace`'s `picture_file_id` (`setBotPicture`, `copyMediaAsPicture`; super admins and admins only, `assertCrewManager`). Both take the image by its `[file:ID]` ref (`grabPictureSource`, so a bot only sets what it could already read; chat uploads land in `uploads/...` and are labelled with their media id in `load-attachment-parts.ts`/`describeAttachment.ts`) or by an upload's media id. The bytes are **copied** into a new picture row, never moved: a chat message that attached the source keeps loading, and only a row whose paradigm is `bot-image`/`workspace-image` is deleted when the picture is replaced (`removeReplacedPicture`, `is-picture-media.ts`). A picture keeps its own format (`imageOutputFormat`, `src/utils/image-output-format.ts`: PNG stays a PNG with its transparency, GIF becomes PNG, everything else JPEG), for the full image and the thumbnail alike. Its change is pushed live (`pushBotUpdated`'s `image_media_id`, `pushWorkspaceUpdated` and `server:workspace-updated`, `relayChangedWorkspaces`), so `BotAvatar` (rounded square, else its stone, its colour still applies) and `WorkspaceAvatar` (else the workspace icon) swap it without a reload. A bot is also edited from the header avatar (`(partials)/bot-profile-modal.tsx`, the same `BotForm` as adding one, `api/admin/bots/update-bot.ts`; the coordinator keeps its name and stays hue-less).
|
||||
- **Colour:** a bot's colour is free. It lives in `bots.color` (`#rrggbb`, set from the palette tiles, the native swatch or a typed hex in `(partials)/bot-colour-picker.tsx`, `api/admin/bots/`), and the stone it's drawn on is `bots.stone` (`BotStoneShapes`), because shape no longer follows colour. `src/utils/bot-color.ts` is the only place that resolves them (`resolveBotColor`/`resolveBotStone` fall back to the old `hue` for a bot saved before this, `botColorNeedsPaperFace` flips the eyes to paper on a very dark body), and `src/utils/bot-stone-props.ts` gives every row its `color`/`stone` props. A new bot starts on the palette's next free colour (`src/utils/pick-free-color.ts`). `src/scripts/move-bot-colour-to-owners.ts` backfills an install from `hue` (safe to rerun).
|
||||
- **Avatars:** `BotStone` (`src/components/general/bot-stone.tsx`) renders the stone inline from `src/data/stone-geometry.ts`, the bot's colour as the body's SVG `fill` attribute, face (`idle|working|done|asleep|needs-you`) drawn as eyes. The coordinator is hue-less (ink body, paper eyes; inverts in dark). Never put a stone in a circle/badge or add outline/shadow. Wordmark is vector (`Wordmark`, `src/data/wordmark-paths.ts`), never live text.
|
||||
- **Copy helpers:** `src/utils/crew-status.ts` (`3 working · 1 needs you`, "Three bots are working. One needs you."), `src/utils/relative-time.ts` (relative timestamps only).
|
||||
|
||||
@@ -97,7 +97,7 @@ describe("botMessagesToHarness with a file-only reply", () => {
|
||||
).toEqual([
|
||||
{
|
||||
role: "user",
|
||||
content: `[Replying to your earlier message: "Send me the logo."]\n\n[Attached earlier: "logo.png"]`,
|
||||
content: `[Replying to your earlier message: "Send me the logo."]\n\n[Attached earlier: "logo.png", media id 9]`,
|
||||
},
|
||||
]);
|
||||
});
|
||||
@@ -130,7 +130,13 @@ describe("botMessagesToHarness with files", () => {
|
||||
botMessagesToHarness({
|
||||
messages: [{ id: 2, author: "user", text: "", attachments }],
|
||||
}),
|
||||
).toEqual([{ role: "user", content: `[Attached earlier: "shot.png"]` }]);
|
||||
).toEqual([
|
||||
{
|
||||
role: "user",
|
||||
/** Its media id rides along, so a picture can be set from it. */
|
||||
content: `[Attached earlier: "shot.png", media id 9]`,
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { HarnessContentPart, HarnessMessage } from "@/src/harness/types";
|
||||
import type { BotMessageWithAttachments } from "@/src/types";
|
||||
import describeMessageRefs, { type MessageRefNames } from "@/src/utils/describe-message-refs";
|
||||
import describeAttachment from "./describe-attachment";
|
||||
|
||||
type Params = {
|
||||
messages: Pick<
|
||||
@@ -87,7 +88,7 @@ function messageParts({
|
||||
...content,
|
||||
{
|
||||
type: "text",
|
||||
text: `[Attached earlier: ${message.attachments.map((file) => `"${file.name}"`).join(", ")}]`,
|
||||
text: `[Attached earlier: ${message.attachments.map((file) => describeAttachment({ name: file.name, id: file.id })).join(", ")}]`,
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import describeAttachment from "./describe-attachment";
|
||||
|
||||
describe("describeAttachment", () => {
|
||||
test("names the upload and its media id, so a tool can be given it", () => {
|
||||
expect(describeAttachment({ name: "summit-logo.png", id: 42 })).toBe(
|
||||
`"summit-logo.png", media id 42`,
|
||||
);
|
||||
});
|
||||
|
||||
test("takes the id as text as well as a number", () => {
|
||||
expect(describeAttachment({ name: "logo.svg", id: "7" })).toBe(`"logo.svg", media id 7`);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,22 @@
|
||||
type Params = {
|
||||
/**
|
||||
* The name the file was uploaded with.
|
||||
*/
|
||||
name: string;
|
||||
/**
|
||||
* Its `media` id.
|
||||
*/
|
||||
id: number | string;
|
||||
};
|
||||
|
||||
/**
|
||||
* How an upload is described to the model: its name and its media id, e.g.
|
||||
* `"summit-logo.png", media id 42`.
|
||||
*
|
||||
* The id matters: the picture tools (`set_picture`, `update_workspace`)
|
||||
* take an upload by media id, so a bot that can read the id can act on the
|
||||
* image the user sent instead of asking them to upload it again.
|
||||
*/
|
||||
export default function describeAttachment({ name, id }: Params) {
|
||||
return `"${name}", media id ${id}`;
|
||||
}
|
||||
@@ -6,6 +6,7 @@ import grabMessagesMedia, {
|
||||
import type { HarnessContentPart } from "@/src/harness/types";
|
||||
import type { BotMessageWithAttachments } from "@/src/types";
|
||||
import grabDirNames from "@/src/utils/grab-dir-names";
|
||||
import describeAttachment from "./describe-attachment";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
@@ -45,12 +46,18 @@ export default async function loadAttachmentParts({ messages }: Params) {
|
||||
|
||||
async function fileToParts({ file }: { file: MessageMediaRow }): Promise<HarnessContentPart[]> {
|
||||
const name = file.title || `file ${file.id}`;
|
||||
/**
|
||||
* The upload's own id, so a bot can name it back to a tool
|
||||
* (`set_picture` and `update_workspace` take one) instead of guessing
|
||||
* which file the user means.
|
||||
*/
|
||||
const label = describeAttachment({ name, id: file.id! });
|
||||
|
||||
if (file.media_type == "document") {
|
||||
const text = file.text_content?.trim();
|
||||
|
||||
if (!text) {
|
||||
return [{ type: "text", text: `[Attached file "${name}": its text couldn't be read.]` }];
|
||||
return [{ type: "text", text: `[Attached file ${label}: its text couldn't be read.]` }];
|
||||
}
|
||||
|
||||
const cut = text.length > AppData.BotAttachmentMaxTextChars;
|
||||
@@ -58,32 +65,32 @@ async function fileToParts({ file }: { file: MessageMediaRow }): Promise<Harness
|
||||
return [
|
||||
{
|
||||
type: "text",
|
||||
text: `Attached file "${name}":\n\n${text.slice(0, AppData.BotAttachmentMaxTextChars)}${cut ? `\n\n[The rest of "${name}" was cut: it's ${text.length} characters long.]` : ""}`,
|
||||
text: `Attached file ${label}:\n\n${text.slice(0, AppData.BotAttachmentMaxTextChars)}${cut ? `\n\n[The rest of "${name}" was cut: it's ${text.length} characters long.]` : ""}`,
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
if (file.media_type == "video") {
|
||||
return [{ type: "text", text: `[Attached video "${name}": videos can't be read yet.]` }];
|
||||
return [{ type: "text", text: `[Attached video ${label}: videos can't be read yet.]` }];
|
||||
}
|
||||
|
||||
const disk = Bun.file(path.join(grabDirNames().DATA_DIR, file.media_write_path || ""));
|
||||
|
||||
if (!file.media_write_path || !(await disk.exists())) {
|
||||
return [{ type: "text", text: `[Attached file "${name}" is missing on the server.]` }];
|
||||
return [{ type: "text", text: `[Attached file ${label} is missing on the server.]` }];
|
||||
}
|
||||
|
||||
const base64 = Buffer.from(await disk.arrayBuffer()).toString("base64");
|
||||
|
||||
if (file.media_type == "image") {
|
||||
return [
|
||||
{ type: "text", text: `[Attached image "${name}"]` },
|
||||
{ type: "text", text: `[Attached image ${label}]` },
|
||||
{ type: "image", url: `data:image/${file.mime_type || "jpeg"};base64,${base64}`, name },
|
||||
];
|
||||
}
|
||||
|
||||
return [
|
||||
{ type: "text", text: `[Attached audio "${name}"]` },
|
||||
{ type: "text", text: `[Attached audio ${label}]` },
|
||||
{ type: "audio", data: base64, format: file.mime_type || "mp3", name },
|
||||
];
|
||||
}
|
||||
|
||||
@@ -63,10 +63,13 @@ async function push({
|
||||
workspace_ids,
|
||||
}: Params) {
|
||||
const audience = user_id ? [user_id] : await grabBotAudience({ bot_id });
|
||||
/** Its colour and stone, so a recolour reaches every open tab. */
|
||||
/**
|
||||
* Its colour, stone and picture, so a recolour or a new face reaches
|
||||
* every open tab.
|
||||
*/
|
||||
const identity = (
|
||||
await BunSQLite.sql<BUN_SQLITE_COBBLE_BOTS>({
|
||||
sql: `SELECT color, stone FROM bots WHERE id = ? LIMIT 1`,
|
||||
sql: `SELECT color, stone, image_media_id FROM bots WHERE id = ? LIMIT 1`,
|
||||
values: [bot_id],
|
||||
})
|
||||
).payload?.[0];
|
||||
@@ -83,6 +86,7 @@ async function push({
|
||||
...(await grabBotFace({ bot_id, user_id: viewer_id })),
|
||||
color: identity?.color || null,
|
||||
stone: identity?.stone || null,
|
||||
image_media_id: identity?.image_media_id ? Number(identity.image_media_id) : null,
|
||||
...(computer_id !== undefined ? { computer_id: computer_id || null } : {}),
|
||||
...(workspace_ids !== undefined ? { workspace_ids } : {}),
|
||||
task: await grabActiveBotTask({ bot_id, user_id: viewer_id }),
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import type { BUN_SQLITE_COBBLE_WORKSPACES } from "@/db/types/db";
|
||||
import grabWorkspaceAudience from "@/src/functions/backend/db/access/grab-workspace-audience";
|
||||
import { logError } from "@/src/functions/backend/logger";
|
||||
import { sendToUser } from "@/src/websocket/(functions)/send-to-user";
|
||||
|
||||
type Params = {
|
||||
workspace_id: number | string;
|
||||
/**
|
||||
* Only these users (the connected ones, for the live relay); else
|
||||
* everyone who sees the workspace.
|
||||
*/
|
||||
users?: Set<string>;
|
||||
};
|
||||
|
||||
/**
|
||||
* Pushes `server:workspace-updated` to everyone who sees a workspace whose
|
||||
* picture changed, so the rail and its header swap it without a reload.
|
||||
* Best effort: a push that fails is logged and never fails the change that
|
||||
* caused it.
|
||||
*/
|
||||
export default async function pushWorkspaceUpdated({ workspace_id, users }: Params) {
|
||||
if (!workspace_id) return;
|
||||
|
||||
try {
|
||||
const workspace = (
|
||||
await BunSQLite.sql<BUN_SQLITE_COBBLE_WORKSPACES>({
|
||||
sql: `SELECT id, image_media_id FROM workspaces WHERE id = ? LIMIT 1`,
|
||||
values: [Number(workspace_id)],
|
||||
})
|
||||
).payload?.[0];
|
||||
|
||||
if (!workspace?.id) return;
|
||||
|
||||
const data = {
|
||||
workspace_id: Number(workspace.id),
|
||||
image_media_id: workspace.image_media_id ? Number(workspace.image_media_id) : null,
|
||||
};
|
||||
|
||||
for (const user_id of await grabWorkspaceAudience({ workspace_id: workspace.id })) {
|
||||
if (users && !users.has(String(user_id))) continue;
|
||||
|
||||
sendToUser({
|
||||
user_id,
|
||||
data: { event: "server:workspace-updated", data },
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
await logError({
|
||||
title: "Workspace update wasn't pushed",
|
||||
error,
|
||||
metadata: { workspace_id },
|
||||
}).catch(() => {});
|
||||
}
|
||||
}
|
||||
@@ -17,6 +17,7 @@ import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-comp
|
||||
import runTextLoop from "@/src/harness/text/run-text-loop";
|
||||
import grabBotChatTools from "@/src/harness/tools/bot/grab-bot-chat-tools";
|
||||
import grabBotFileTools from "@/src/harness/tools/storage/grab-bot-file-tools";
|
||||
import grabRunScopes from "@/src/functions/backend/db/bot-files/grab-run-scopes";
|
||||
import openBotExtensions from "@/src/harness/tools/extensions/open-bot-extensions";
|
||||
import openBotSkills from "@/src/harness/tools/skills/open-bot-skills";
|
||||
import botMessagesToHarness from "./bot-messages-to-harness";
|
||||
@@ -87,11 +88,18 @@ export default async function runBotReply({
|
||||
*/
|
||||
const attachments: number[] = [];
|
||||
|
||||
/**
|
||||
* The files this run reaches: its file tools and the pictures it may
|
||||
* set work from the same answer (`grabRunScopes`).
|
||||
*/
|
||||
const scopes = await grabRunScopes({ bot, user_id, workspace_id });
|
||||
|
||||
const file_tools = await grabBotFileTools({
|
||||
bot,
|
||||
user_id,
|
||||
workspace_id,
|
||||
session_id,
|
||||
scopes,
|
||||
onAttach: ({ media_id }) => attachments.push(media_id),
|
||||
});
|
||||
|
||||
@@ -101,6 +109,7 @@ export default async function runBotReply({
|
||||
computers,
|
||||
session_id,
|
||||
workspace_id,
|
||||
scopes,
|
||||
message_id: job.message_id || undefined,
|
||||
onTakeOver: ({ ref }) => take_overs.push(ref),
|
||||
});
|
||||
|
||||
@@ -24,7 +24,7 @@ import searchComputersTool from "@/src/harness/tools/computer/search-computers-t
|
||||
import grabBotFileTools from "@/src/harness/tools/storage/grab-bot-file-tools";
|
||||
import reportRunFailedTool from "@/src/harness/tools/bot/report-run-failed-tool";
|
||||
import computerRef from "@/src/utils/computer-ref";
|
||||
import grabStorageScopes from "@/src/utils/storage-access";
|
||||
import grabRunScopes from "@/src/functions/backend/db/bot-files/grab-run-scopes";
|
||||
import composeJobBrief from "./compose-job-brief";
|
||||
import grabSessionCarryOver from "./grab-session-carry-over";
|
||||
import jobSessionId from "@/src/functions/backend/db/bots-queue/job-session-id";
|
||||
@@ -108,6 +108,11 @@ export default async function runBotTask({
|
||||
...computers.filter((c) => c.id == job.computer_id),
|
||||
...computers.filter((c) => c.id != job.computer_id),
|
||||
];
|
||||
/**
|
||||
* The files this run reaches, for its file tools, an import and the
|
||||
* pictures it may set: one answer, where `grabRunScopes` decides.
|
||||
*/
|
||||
const scopes = await grabRunScopes({ bot, user_id, workspace_id });
|
||||
|
||||
const tools = [
|
||||
...(await grabBotTools({
|
||||
@@ -117,6 +122,7 @@ export default async function runBotTask({
|
||||
computers,
|
||||
session_id,
|
||||
is_coordinator: bot.kind == "coordinator",
|
||||
scopes,
|
||||
})),
|
||||
...(bot.kind == "coordinator"
|
||||
? [
|
||||
@@ -127,6 +133,7 @@ export default async function runBotTask({
|
||||
user_id,
|
||||
workspaces: coordinator_workspaces,
|
||||
workspace_id,
|
||||
scopes,
|
||||
}),
|
||||
]
|
||||
: []),
|
||||
@@ -148,11 +155,7 @@ export default async function runBotTask({
|
||||
? [
|
||||
importFileTool({
|
||||
computer_id: job.computer_id,
|
||||
scopes: grabStorageScopes({
|
||||
bot_id: bot.id,
|
||||
bot_kind: bot.kind,
|
||||
workspace_id,
|
||||
}),
|
||||
scopes,
|
||||
bot_id: bot.id,
|
||||
user_id,
|
||||
}),
|
||||
@@ -163,6 +166,7 @@ export default async function runBotTask({
|
||||
user_id,
|
||||
workspace_id,
|
||||
session_id,
|
||||
scopes,
|
||||
onAttach: ({ media_id }) => attachments.push(media_id),
|
||||
})),
|
||||
];
|
||||
|
||||
@@ -49,7 +49,8 @@ You have a folder of your own, kept with the app (not on a computer), and it sta
|
||||
|
||||
- list_files shows what's in it; read_file opens one (by its [file:ID] ref or path); write_file saves a text file, make_pdf a PDF, generate_image an image. Move and delete with move_file and delete_file when something is out of date, so the folder stays useful.
|
||||
- Text files are chunked and searchable as soon as they're saved. search_files finds the passages that answer a question, by meaning when an embedding model is set and by words otherwise. Search your files before saying you don't know something they may hold, and when the user asks about a document they gave you.
|
||||
- Uploaded files land in the same folder, so what the user sent you earlier is searchable later too.
|
||||
- Uploaded files land in the same folder, under uploads/ (uploads/2026-10/logo.png), so what the user sent you earlier is searchable later too.
|
||||
- A picture can be set from a file: set_picture makes a file your own avatar, and update_workspace sets a workspace's. Both take the [file:ID] ref, so a logo the user sent you is usually ready to go: list_files for it, then set it. An SVG or another kind of file is turned into a PNG first. Its picture is free to change and never removes your file.
|
||||
- A file and a folder are two different refs, and you never guess which one from the name:
|
||||
- A file you saved is a [file:ID] ref (from write_file, list_files, search_files or import_file). The user sees a chip that opens the file.
|
||||
- A folder is a folder ref, e.g. ${folderRef({ volume: "bot", owner_id: 5, path: "reports" })}: the user sees a chip that opens that folder in Files, and nothing is downloaded. Use the volume of the folder you mean ("bot" for your own, "workspace" for a workspace's shared one).
|
||||
|
||||
@@ -70,6 +70,7 @@ export const ReservedToolNames = [
|
||||
"report_run_failed",
|
||||
"search_computers",
|
||||
"set_bot_workspace_instructions",
|
||||
"set_picture",
|
||||
"set_workspace_computer",
|
||||
"start_task",
|
||||
"update_instructions",
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db";
|
||||
import grabUserWorkspaces from "@/src/functions/backend/db/workspaces/grab-user-workspaces";
|
||||
import grabStorageScopes, { type StorageScope } from "@/src/utils/storage-access";
|
||||
|
||||
type Params = {
|
||||
bot: Pick<BUN_SQLITE_COBBLE_BOTS, "id" | "kind">;
|
||||
/**
|
||||
* The user the run is for: the coordinator reaches their workspaces.
|
||||
*/
|
||||
user_id: number | string;
|
||||
workspace_id?: number | string | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* The directories one run may touch, and the only place the coordinator's
|
||||
* workspaces are read for it: its own folder first, then every workspace's
|
||||
* (`grabStorageScopes`).
|
||||
*
|
||||
* Every tool that takes files uses this, so a file a bot can read with
|
||||
* `list_files` is exactly a file it can set as a picture, attach to a task
|
||||
* or import into.
|
||||
*/
|
||||
export default async function grabRunScopes({ bot, user_id, workspace_id }: Params): Promise<StorageScope[]> {
|
||||
const all_workspace_ids =
|
||||
bot.kind == "coordinator"
|
||||
? (await grabUserWorkspaces({ user_id }))
|
||||
.filter((workspace) => !workspace.archived)
|
||||
.map((workspace) => Number(workspace.id))
|
||||
: [];
|
||||
|
||||
return grabStorageScopes({
|
||||
bot_id: bot.id || 0,
|
||||
bot_kind: bot.kind,
|
||||
workspace_id,
|
||||
all_workspace_ids,
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,155 @@
|
||||
import { afterAll, beforeAll, describe, expect, test } from "bun:test";
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import path from "node:path";
|
||||
import type { BUN_SQLITE_COBBLE_MEDIA } from "@/db/types/db";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
import copyMediaAsPicture, { PictureParadigms } from "./copy-media-as-picture";
|
||||
|
||||
/**
|
||||
* A data directory of its own, with one PNG in it, so a copy reads real
|
||||
* bytes and the source can be checked to survive.
|
||||
*/
|
||||
let data_dir = "";
|
||||
const source_path = "media/1/9.png";
|
||||
const png = Buffer.from(
|
||||
"iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==",
|
||||
"base64",
|
||||
);
|
||||
|
||||
beforeAll(() => {
|
||||
data_dir = mkdtempSync(path.join(tmpdir(), "cobble-picture-"));
|
||||
process.env.DATA_DIR = data_dir;
|
||||
|
||||
mkdirSync(path.dirname(path.join(data_dir, source_path)), { recursive: true });
|
||||
writeFileSync(path.join(data_dir, source_path), png);
|
||||
});
|
||||
|
||||
afterAll(() => {
|
||||
delete process.env.DATA_DIR;
|
||||
rmSync(data_dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
const media = (overrides: Partial<BUN_SQLITE_COBBLE_MEDIA> = {}) =>
|
||||
({
|
||||
id: 9,
|
||||
title: "logo.png",
|
||||
media_type: "image",
|
||||
mime_type: "image/png",
|
||||
media_write_path: source_path,
|
||||
...overrides,
|
||||
}) as BUN_SQLITE_COBBLE_MEDIA;
|
||||
|
||||
type SaveParams = Parameters<NonNullable<Parameters<typeof copyMediaAsPicture>[0]["save"]>>[0];
|
||||
|
||||
/**
|
||||
* A `save` that records what it was given and answers a new row, so the
|
||||
* copy is checked without touching sharp.
|
||||
*/
|
||||
function recorder() {
|
||||
const calls: SaveParams[] = [];
|
||||
|
||||
return {
|
||||
calls,
|
||||
save: async (params: SaveParams) => {
|
||||
calls.push(params);
|
||||
return { id: 77, ...params } as BUN_SQLITE_COBBLE_MEDIA;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe("copyMediaAsPicture", () => {
|
||||
test("names the paradigm the picture is saved under", () => {
|
||||
expect(PictureParadigms.bot).toBe("bot-image");
|
||||
expect(PictureParadigms.workspace).toBe("workspace-image");
|
||||
});
|
||||
|
||||
test("saves a workspace picture under its own paradigm and answers the new id", async () => {
|
||||
const saved = recorder();
|
||||
|
||||
const id = await copyMediaAsPicture({
|
||||
media: media(),
|
||||
user_id: 3,
|
||||
target: "workspace",
|
||||
title: "Summit Lending picture",
|
||||
save: saved.save,
|
||||
});
|
||||
|
||||
expect(id).toBe(77);
|
||||
expect(saved.calls[0]?.paradigms).toEqual(["workspace-image"]);
|
||||
expect(saved.calls[0]?.title).toBe("Summit Lending picture");
|
||||
expect(saved.calls[0]?.user_id).toBe(3);
|
||||
});
|
||||
|
||||
test("takes the image's own format, so a PNG is copied as a PNG", async () => {
|
||||
const saved = recorder();
|
||||
|
||||
await copyMediaAsPicture({ media: media(), user_id: 3, target: "bot", save: saved.save });
|
||||
|
||||
expect(saved.calls[0]?.ext).toBe("png");
|
||||
});
|
||||
|
||||
test("copies the source's bytes without taking them away", async () => {
|
||||
const saved = recorder();
|
||||
|
||||
await copyMediaAsPicture({ media: media(), user_id: 3, target: "workspace", save: saved.save });
|
||||
|
||||
expect(Buffer.from(saved.calls[0]!.data).equals(png)).toBe(true);
|
||||
expect(await Bun.file(path.join(data_dir, source_path)).exists()).toBe(true);
|
||||
});
|
||||
|
||||
test("refuses anything that isn't an image, and says how to get one", async () => {
|
||||
const saved = recorder();
|
||||
|
||||
await expect(
|
||||
copyMediaAsPicture({
|
||||
media: media({ media_type: "document", mime_type: "image/svg+xml" }),
|
||||
user_id: 3,
|
||||
target: "workspace",
|
||||
save: saved.save,
|
||||
}),
|
||||
).rejects.toThrow(/turned into a PNG first/);
|
||||
|
||||
expect(saved.calls).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("refuses a file over the picture size limit", async () => {
|
||||
const saved = recorder();
|
||||
const big_path = "media/1/big.png";
|
||||
|
||||
mkdirSync(path.dirname(path.join(data_dir, big_path)), { recursive: true });
|
||||
writeFileSync(path.join(data_dir, big_path), Buffer.alloc(AppData.ProfileImageMaxBytes + 1));
|
||||
|
||||
await expect(
|
||||
copyMediaAsPicture({
|
||||
media: media({ media_write_path: big_path }),
|
||||
user_id: 3,
|
||||
target: "workspace",
|
||||
save: saved.save,
|
||||
}),
|
||||
).rejects.toThrow(/too big/);
|
||||
|
||||
expect(saved.calls).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("refuses a row whose file is gone", async () => {
|
||||
const saved = recorder();
|
||||
|
||||
await expect(
|
||||
copyMediaAsPicture({
|
||||
media: media({ media_write_path: "media/1/missing.png" }),
|
||||
user_id: 3,
|
||||
target: "workspace",
|
||||
save: saved.save,
|
||||
}),
|
||||
).rejects.toThrow(/missing on the server/);
|
||||
});
|
||||
|
||||
test("refuses a row that is gone", async () => {
|
||||
const saved = recorder();
|
||||
|
||||
await expect(
|
||||
copyMediaAsPicture({ media: media({ id: 0 }), user_id: 3, target: "bot", save: saved.save }),
|
||||
).rejects.toThrow(/gone/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,135 @@
|
||||
import path from "node:path";
|
||||
import type { BUN_SQLITE_COBBLE_MEDIA } from "@/db/types/db";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
import { extensionOf } from "@/src/functions/backend/db/bot-files/media-type-for-file";
|
||||
import uploadAndRecordMedia from "@/src/functions/backend/db/users/media/upload-and-record-media";
|
||||
import type { MediaParadigm } from "@/src/types";
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import grabDirNames from "@/src/utils/grab-dir-names";
|
||||
|
||||
export type PictureTarget = "bot" | "workspace";
|
||||
|
||||
/**
|
||||
* The paradigm a bot's and a workspace's picture is saved under: what
|
||||
* `set-profile-image.ts` and `update-workspace.ts` both read and delete by,
|
||||
* so a picture is never mistaken for a chat upload or a bot file.
|
||||
*/
|
||||
export const PictureParadigms: Record<PictureTarget, MediaParadigm> = {
|
||||
bot: "bot-image",
|
||||
workspace: "workspace-image",
|
||||
};
|
||||
|
||||
type SaveParams = {
|
||||
data: Buffer;
|
||||
ext: string;
|
||||
paradigms: MediaParadigm[];
|
||||
title: string;
|
||||
user_id: number;
|
||||
};
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The image to copy: a `media` row the caller already checked the user
|
||||
* reaches (`grabPictureSource`).
|
||||
*/
|
||||
media: BUN_SQLITE_COBBLE_MEDIA;
|
||||
user_id: number | string;
|
||||
target: PictureTarget;
|
||||
/**
|
||||
* What the new row is called, e.g. "Summit Lending picture".
|
||||
*/
|
||||
title?: string | null;
|
||||
/**
|
||||
* Writes the copy and answers its new row. Swapped in tests, so the
|
||||
* copy is checked without sharp.
|
||||
*/
|
||||
save?: (params: SaveParams) => Promise<BUN_SQLITE_COBBLE_MEDIA | null>;
|
||||
};
|
||||
|
||||
/**
|
||||
* Copies an image the user can see into a new picture row: a bot's or a
|
||||
* workspace's picture is always its own `media` row, in its own paradigm.
|
||||
*
|
||||
* A bot is given the files it works with (`[file:ID]`) and the uploads the
|
||||
* user sent it, never a raw media id, so a picture is copied from what it
|
||||
* already reaches. The source keeps its row and its bytes: the message
|
||||
* that attached it still loads, and changing the picture later can never
|
||||
* take a file away from a conversation.
|
||||
*/
|
||||
export default async function copyMediaAsPicture({
|
||||
media,
|
||||
user_id,
|
||||
target,
|
||||
title,
|
||||
save = savePictureMedia,
|
||||
}: Params): Promise<number> {
|
||||
const id = Number(media.id);
|
||||
|
||||
if (!id) {
|
||||
throw new AppError(`That picture is gone. Pick another one.`);
|
||||
}
|
||||
|
||||
if (media.media_type != "image") {
|
||||
throw new AppError(
|
||||
`A picture has to be an image. This one is a ${media.media_type || "file"}: an SVG or a document is turned into a PNG first, then that PNG is set.`,
|
||||
);
|
||||
}
|
||||
|
||||
const bytes = await readMediaBytes({ media });
|
||||
|
||||
if (bytes.length > AppData.ProfileImageMaxBytes) {
|
||||
throw new AppError(
|
||||
`That picture is too big. Pictures can be up to ${Math.round(AppData.ProfileImageMaxBytes / 1024 / 1024)} MB.`,
|
||||
);
|
||||
}
|
||||
|
||||
const saved = await save({
|
||||
data: bytes,
|
||||
ext: extensionOf({ name: media.media_write_path, mime_type: media.mime_type }) || "png",
|
||||
paradigms: [PictureParadigms[target]],
|
||||
title: title?.trim() || `${target} picture`,
|
||||
user_id: Number(user_id),
|
||||
});
|
||||
|
||||
if (!saved?.id) {
|
||||
throw new AppError(`Couldn't save the picture. Try another image.`);
|
||||
}
|
||||
|
||||
return Number(saved.id);
|
||||
}
|
||||
|
||||
/**
|
||||
* The image's bytes, read from where `media` keeps them.
|
||||
*/
|
||||
async function readMediaBytes({ media }: { media: BUN_SQLITE_COBBLE_MEDIA }) {
|
||||
const { DATA_DIR } = grabDirNames();
|
||||
|
||||
if (!media.media_write_path) {
|
||||
throw new AppError(`Its file is missing on the server. Save it again.`);
|
||||
}
|
||||
|
||||
const file = Bun.file(path.join(DATA_DIR, media.media_write_path));
|
||||
|
||||
if (!(await file.exists())) {
|
||||
throw new AppError(`Its file is missing on the server. Save it again.`);
|
||||
}
|
||||
|
||||
return Buffer.from(await file.arrayBuffer());
|
||||
}
|
||||
|
||||
/**
|
||||
* Writes the copy as a picture: a private image, owned by the user whose
|
||||
* crew it is, under its own paradigm.
|
||||
*/
|
||||
async function savePictureMedia({ data, ext, paradigms, title, user_id }: SaveParams) {
|
||||
const res = await uploadAndRecordMedia({
|
||||
data,
|
||||
media_type: "image",
|
||||
media_mime_type: ext,
|
||||
media_paradigms: paradigms,
|
||||
media: { title },
|
||||
user: { id: user_id },
|
||||
});
|
||||
|
||||
return res.singleRes || null;
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
import type { BUN_SQLITE_COBBLE_MEDIA } from "@/db/types/db";
|
||||
import grabBotFile from "@/src/functions/backend/db/bot-files/grab-bot-file";
|
||||
import type { StorageScope } from "@/src/utils/storage-access";
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import grabOwnPictureMedia from "./grab-own-picture-media";
|
||||
|
||||
type Params = {
|
||||
user_id: number | string;
|
||||
/**
|
||||
* The directories the run may read (`grabStorageScopes`), so a file id
|
||||
* outside them is refused here rather than by the caller.
|
||||
*/
|
||||
scopes?: StorageScope[];
|
||||
/**
|
||||
* A file the run can see: its `[file:ID]` ref, which is what every file
|
||||
* tool answers with.
|
||||
*/
|
||||
file_id?: number | null;
|
||||
/**
|
||||
* An upload of the user's own, by media id.
|
||||
*/
|
||||
media_id?: number | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* The image a picture is copied from, checked to be one the user may use:
|
||||
* a file in one of the run's directories (`grabBotFile` decides that, the
|
||||
* same gate every other file tool uses) or their own upload
|
||||
* (`grabOwnPictureMedia`).
|
||||
*
|
||||
* Exactly one of `file_id` and `media_id` is needed. Null when neither is
|
||||
* given; an id that isn't reachable throws, so a tool never quietly sets
|
||||
* someone else's file as a picture.
|
||||
*/
|
||||
export default async function grabPictureSource({
|
||||
user_id,
|
||||
scopes = [],
|
||||
file_id,
|
||||
media_id,
|
||||
}: Params): Promise<BUN_SQLITE_COBBLE_MEDIA | null> {
|
||||
if (file_id) {
|
||||
const file = await grabBotFile({ scopes, file_id });
|
||||
|
||||
if (!file?.media?.id) {
|
||||
throw new AppError(
|
||||
`No file with that id is in your files. Use list_files to see what you have.`,
|
||||
);
|
||||
}
|
||||
|
||||
return file.media;
|
||||
}
|
||||
|
||||
if (media_id) {
|
||||
const media = await grabOwnPictureMedia({ media_id, user_id });
|
||||
|
||||
if (!media?.id) {
|
||||
throw new AppError(`That picture isn't one of yours. Upload it again.`);
|
||||
}
|
||||
|
||||
return media;
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import type { BUN_SQLITE_COBBLE_MEDIA_PARADIGMS } from "@/db/types/db";
|
||||
import { PictureParadigms, type PictureTarget } from "./copy-media-as-picture";
|
||||
|
||||
type Params = {
|
||||
media_id: number | string;
|
||||
/**
|
||||
* Which picture it has to be: a bot's or a workspace's.
|
||||
*/
|
||||
target: PictureTarget;
|
||||
};
|
||||
|
||||
/**
|
||||
* Whether a media row is a picture of that kind. Only one is deleted when
|
||||
* a picture is replaced: a row a chat message still attaches is left where
|
||||
* it is, since removing it would break a message the user can still scroll
|
||||
* to.
|
||||
*/
|
||||
export default async function isPictureMedia({ media_id, target }: Params) {
|
||||
const res = await BunSQLite.sql<BUN_SQLITE_COBBLE_MEDIA_PARADIGMS>({
|
||||
sql: `SELECT media_paradigm FROM media_paradigms WHERE media_id = ?`,
|
||||
values: [Number(media_id)],
|
||||
});
|
||||
|
||||
return (res.payload || []).some((row) => row.media_paradigm == PictureParadigms[target]);
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import removeReplacedPicture from "./remove-replaced-picture";
|
||||
|
||||
function spy({ is_picture = true }: { is_picture?: boolean } = {}) {
|
||||
const asked: unknown[] = [];
|
||||
const removed: unknown[] = [];
|
||||
|
||||
return {
|
||||
asked,
|
||||
removed,
|
||||
isPicture: async (params: { media_id: number | string; target: "bot" | "workspace" }) => {
|
||||
asked.push(params);
|
||||
return is_picture;
|
||||
},
|
||||
remove: async (params: unknown) => {
|
||||
removed.push(params);
|
||||
return { success: true };
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
describe("removeReplacedPicture", () => {
|
||||
test("deletes the picture a bot no longer shows", async () => {
|
||||
const deps = spy();
|
||||
|
||||
await removeReplacedPicture({
|
||||
previous: 4,
|
||||
next: 9,
|
||||
user_id: 3,
|
||||
isPicture: deps.isPicture,
|
||||
remove: deps.remove,
|
||||
});
|
||||
|
||||
expect(deps.asked).toEqual([{ media_id: 4, target: "bot" }]);
|
||||
expect(deps.removed).toEqual([{ id: 4, user_id: 3 }]);
|
||||
});
|
||||
|
||||
test("never deletes a row that isn't a picture, such as a chat upload", async () => {
|
||||
const deps = spy({ is_picture: false });
|
||||
|
||||
await removeReplacedPicture({
|
||||
previous: 4,
|
||||
next: 9,
|
||||
user_id: 3,
|
||||
isPicture: deps.isPicture,
|
||||
remove: deps.remove,
|
||||
});
|
||||
|
||||
expect(deps.removed).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("leaves a bot that had no picture alone", async () => {
|
||||
const deps = spy();
|
||||
|
||||
await removeReplacedPicture({ previous: null, next: 9, user_id: 3, isPicture: deps.isPicture, remove: deps.remove });
|
||||
|
||||
expect(deps.asked).toHaveLength(0);
|
||||
expect(deps.removed).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("keeps the picture that is still in use, e.g. naming it twice", async () => {
|
||||
const deps = spy();
|
||||
|
||||
await removeReplacedPicture({ previous: 4, next: 4, user_id: 3, isPicture: deps.isPicture, remove: deps.remove });
|
||||
await removeReplacedPicture({ previous: "4", next: 4, user_id: 3, isPicture: deps.isPicture, remove: deps.remove });
|
||||
|
||||
expect(deps.removed).toHaveLength(0);
|
||||
});
|
||||
|
||||
test("asks for the right kind of picture, a workspace's or a bot's", async () => {
|
||||
const deps = spy();
|
||||
|
||||
await removeReplacedPicture({
|
||||
previous: 4,
|
||||
next: null,
|
||||
user_id: 3,
|
||||
target: "workspace",
|
||||
isPicture: deps.isPicture,
|
||||
remove: deps.remove,
|
||||
});
|
||||
|
||||
expect(deps.asked).toEqual([{ media_id: 4, target: "workspace" }]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,48 @@
|
||||
import deleteMedia from "@/src/functions/backend/db/media/delete-media";
|
||||
import isPictureMedia from "./is-picture-media";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The picture that was shown before (`bots.image_media_id` or
|
||||
* `workspaces.image_media_id`).
|
||||
*/
|
||||
previous?: number | string | null;
|
||||
/**
|
||||
* The one written now; the same id means nothing changed.
|
||||
*/
|
||||
next?: number | string | null;
|
||||
/**
|
||||
* Whose row it is: a picture is the user's, so deleting it is theirs.
|
||||
*/
|
||||
user_id: number | string;
|
||||
target?: "bot" | "workspace";
|
||||
/**
|
||||
* Swapped in tests, so the decision is checked without a database.
|
||||
*/
|
||||
isPicture?: typeof isPictureMedia;
|
||||
remove?: typeof deleteMedia;
|
||||
};
|
||||
|
||||
/**
|
||||
* Deletes the picture a bot or a workspace no longer shows, so its bytes
|
||||
* don't stay on disk for nothing.
|
||||
*
|
||||
* Only a picture is deleted (`isPictureMedia`): a row a chat message still
|
||||
* attaches is left where it is, because removing it would break a message
|
||||
* the user can still scroll to.
|
||||
*/
|
||||
export default async function removeReplacedPicture({
|
||||
previous,
|
||||
next,
|
||||
user_id,
|
||||
target = "bot",
|
||||
isPicture = isPictureMedia,
|
||||
remove = deleteMedia,
|
||||
}: Params) {
|
||||
if (!previous) return;
|
||||
if (next != null && Number(previous) == Number(next)) return;
|
||||
|
||||
if (!(await isPicture({ media_id: previous, target }))) return;
|
||||
|
||||
await remove({ id: previous, user_id: Number(user_id) }).catch(() => {});
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import type { BUN_SQLITE_COBBLE_BOTS, BUN_SQLITE_COBBLE_MEDIA } from "@/db/types/db";
|
||||
import assertCrewManager from "@/src/functions/backend/db/access/assert-crew-manager";
|
||||
import grabUserBot from "@/src/functions/backend/db/bots/grab-user-bot";
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import copyMediaAsPicture from "./copy-media-as-picture";
|
||||
import removeReplacedPicture from "./remove-replaced-picture";
|
||||
|
||||
type Params = {
|
||||
user_id: number | string;
|
||||
bot_id: number | string;
|
||||
/**
|
||||
* The image to copy for it, or null to remove the picture it has.
|
||||
*/
|
||||
media?: BUN_SQLITE_COBBLE_MEDIA | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* Sets (or clears) a bot's picture from an image the caller already checked
|
||||
* it reaches (`grabPictureSource` hands `copyMediaAsPicture` the row).
|
||||
*
|
||||
* Only a super admin or an admin does it, the same rule as the rest of the
|
||||
* crew: a bot can dress itself, but only for a user who manages the crew.
|
||||
* The coordinator is allowed a picture, and keeps its name and hue-less
|
||||
* stone.
|
||||
*
|
||||
* Returns the new `image_media_id` (null once removed). `updated_at` moves
|
||||
* with it, so the live relay pushes the new picture to open tabs.
|
||||
*/
|
||||
export default async function setBotPicture({ user_id, bot_id, media }: Params) {
|
||||
await assertCrewManager({ user_id });
|
||||
|
||||
const bot = await grabUserBot({ bot_id, user_id });
|
||||
|
||||
if (!bot?.id) {
|
||||
throw new AppError(`Bot not found.`);
|
||||
}
|
||||
|
||||
const image_media_id = media?.id
|
||||
? await copyMediaAsPicture({
|
||||
media,
|
||||
user_id,
|
||||
target: "bot",
|
||||
title: `${bot.name || "Bot"} picture`,
|
||||
})
|
||||
: null;
|
||||
|
||||
await BunSQLite.sql({
|
||||
sql: `UPDATE bots SET image_media_id = ?, updated_at = ? WHERE id = ?`,
|
||||
values: [image_media_id, Date.now(), bot.id],
|
||||
});
|
||||
|
||||
await removeReplacedPicture({
|
||||
previous: bot.image_media_id,
|
||||
next: image_media_id,
|
||||
user_id,
|
||||
target: "bot",
|
||||
});
|
||||
|
||||
return { image_media_id, bot: bot as BUN_SQLITE_COBBLE_BOTS };
|
||||
}
|
||||
@@ -1,13 +1,14 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
import grabUserBot from "@/src/functions/backend/db/bots/grab-user-bot";
|
||||
import deleteMedia from "@/src/functions/backend/db/media/delete-media";
|
||||
import uploadAndRecordMedia from "@/src/functions/backend/db/users/media/upload-and-record-media";
|
||||
import grabUserWorkspace from "@/src/functions/backend/db/workspaces/grab-user-workspace";
|
||||
import type { ProfileImageTarget } from "@/src/schemas/profile-image-schema";
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import bufferFromBase64 from "@/src/utils/buffer-from-base-64";
|
||||
import assertCrewManager from "@/src/functions/backend/db/access/assert-crew-manager";
|
||||
import { PictureParadigms } from "./copy-media-as-picture";
|
||||
import removeReplacedPicture from "./remove-replaced-picture";
|
||||
|
||||
type Params = {
|
||||
user_id: number | string;
|
||||
@@ -19,14 +20,19 @@ type Params = {
|
||||
data_url: string | null;
|
||||
};
|
||||
|
||||
const Targets = {
|
||||
bot: { table: "bots", paradigm: "bot-image" },
|
||||
workspace: { table: "workspaces", paradigm: "workspace-image" },
|
||||
} as const;
|
||||
const Tables: Record<ProfileImageTarget, string> = {
|
||||
bot: "bots",
|
||||
workspace: "workspaces",
|
||||
};
|
||||
|
||||
/**
|
||||
* Sets (or removes) the picture of one of the user's bots or workspaces:
|
||||
* a private image, the old one deleted. Returns the new `image_media_id`.
|
||||
* a private image, the old picture deleted. Returns the new
|
||||
* `image_media_id`.
|
||||
*
|
||||
* The picture picked here is copied into its own row
|
||||
* (`PictureParadigms`), and only a picture is deleted when it is replaced
|
||||
* (`removeReplacedPicture`), never a row a chat message still attaches.
|
||||
*/
|
||||
export default async function setProfileImage({ user_id, target, id, data_url }: Params) {
|
||||
await assertCrewManager({ user_id });
|
||||
@@ -40,7 +46,7 @@ export default async function setProfileImage({ user_id, target, id, data_url }:
|
||||
throw new AppError(target == "bot" ? `Bot not found.` : `Workspace not found.`);
|
||||
}
|
||||
|
||||
const { table, paradigm } = Targets[target];
|
||||
const table = Tables[target];
|
||||
let image_media_id: number | null = null;
|
||||
|
||||
if (data_url) {
|
||||
@@ -59,8 +65,12 @@ export default async function setProfileImage({ user_id, target, id, data_url }:
|
||||
const res = await uploadAndRecordMedia({
|
||||
data: buffer,
|
||||
media_type: "image",
|
||||
/**
|
||||
* The image's own format: a transparent PNG stays a PNG
|
||||
* (`imageOutputFormat`).
|
||||
*/
|
||||
media_mime_type: ext,
|
||||
media_paradigms: [paradigm],
|
||||
media_paradigms: [PictureParadigms[target]],
|
||||
media: { title: `${record.name || target} picture` },
|
||||
user: { id: Number(user_id) },
|
||||
});
|
||||
@@ -77,9 +87,12 @@ export default async function setProfileImage({ user_id, target, id, data_url }:
|
||||
values: [image_media_id, Date.now(), record.id],
|
||||
});
|
||||
|
||||
if (record.image_media_id) {
|
||||
await deleteMedia({ id: record.image_media_id, user_id: Number(user_id) }).catch(() => {});
|
||||
}
|
||||
await removeReplacedPicture({
|
||||
previous: record.image_media_id,
|
||||
next: image_media_id,
|
||||
user_id,
|
||||
target,
|
||||
});
|
||||
|
||||
return { image_media_id };
|
||||
}
|
||||
|
||||
@@ -16,15 +16,35 @@ import grabDirNames from "@/src/utils/grab-dir-names";
|
||||
import BunSQLite, { type APIResponseObject } from "@moduletrace/bun-sqlite";
|
||||
import { existsSync, mkdirSync } from "fs";
|
||||
import path from "path";
|
||||
import sharp from "sharp";
|
||||
import sharp, { type Sharp } from "sharp";
|
||||
import canExtractText from "@/src/utils/can-extract-text";
|
||||
import capTextContent from "@/src/utils/cap-text-content";
|
||||
import fileToText from "./file-to-text";
|
||||
import MediaDataToArrayBuffer from "./media-data-to-array-buffer";
|
||||
import sniffMediaExtension from "./sniff-media-ext";
|
||||
import mediaDataToSharpInput from "./meida-data-to-sharp-input";
|
||||
import imageOutputFormat, { type ImageOutputFormat } from "@/src/utils/image-output-format";
|
||||
import deleteMedia from "../../media/delete-media";
|
||||
|
||||
/**
|
||||
* Writes a resized image in the format it came in: PNG stays PNG (with its
|
||||
* transparency), WebP stays WebP, everything else becomes JPEG.
|
||||
*/
|
||||
async function encodeImage({
|
||||
pipeline,
|
||||
format,
|
||||
quality,
|
||||
}: {
|
||||
pipeline: Sharp;
|
||||
format: ImageOutputFormat["format"];
|
||||
quality: number;
|
||||
}) {
|
||||
if (format == "png") return pipeline.png().toBuffer();
|
||||
if (format == "webp") return pipeline.webp({ quality }).toBuffer();
|
||||
|
||||
return pipeline.jpeg({ quality }).toBuffer();
|
||||
}
|
||||
|
||||
type Params = {
|
||||
media_paradigms?: MediaParadigm[];
|
||||
media_type?: MediaType;
|
||||
@@ -102,13 +122,21 @@ export default async function uploadAndRecordMedia({
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* What an image is actually written as: a picture keeps its format, so
|
||||
* a transparent PNG stays a PNG (`imageOutputFormat`). Every other file
|
||||
* keeps its own extension.
|
||||
*/
|
||||
const image_output = media_type == "image" ? imageOutputFormat({ ext: media_ext }) : null;
|
||||
const stored_ext = image_output ? image_output.ext : media_ext;
|
||||
|
||||
const media_write_path = path.join(
|
||||
MEDIA_RELATIVE_DIR,
|
||||
`${target_file_name}.${media_ext}`,
|
||||
`${target_file_name}.${stored_ext}`,
|
||||
);
|
||||
const media_thumbnail_write_path = path.join(
|
||||
MEDIA_RELATIVE_DIR,
|
||||
`${target_file_thumbnail_name}.${media_ext}`,
|
||||
`${target_file_thumbnail_name}.${stored_ext}`,
|
||||
);
|
||||
|
||||
const media_write_path_full = path.join(DATA_DIR, media_write_path);
|
||||
@@ -157,21 +185,25 @@ export default async function uploadAndRecordMedia({
|
||||
full_options = { fit: "inside", withoutEnlargement: true };
|
||||
}
|
||||
|
||||
const output = image_output || imageOutputFormat({ ext: media_ext });
|
||||
|
||||
const [full, thumb] = await Promise.all([
|
||||
sharp(buffer)
|
||||
.resize(full_width, full_height, full_options)
|
||||
.jpeg({ quality: AppData["ImageQuality"] })
|
||||
.toBuffer(),
|
||||
sharp(buffer)
|
||||
.resize(
|
||||
encodeImage({
|
||||
pipeline: sharp(buffer).resize(full_width, full_height, full_options),
|
||||
format: output.format,
|
||||
quality: AppData["ImageQuality"],
|
||||
}),
|
||||
encodeImage({
|
||||
pipeline: sharp(buffer).resize(
|
||||
thumb_width,
|
||||
is_profile_image ? thumb_width : undefined,
|
||||
is_profile_image
|
||||
? { fit: "cover" }
|
||||
: { fit: "inside", withoutEnlargement: true },
|
||||
)
|
||||
.jpeg({ quality: AppData["ImageThumbnailQuality"] })
|
||||
.toBuffer(),
|
||||
),
|
||||
format: output.format,
|
||||
quality: AppData["ImageThumbnailQuality"],
|
||||
}),
|
||||
]);
|
||||
|
||||
await Bun.write(media_write_path_full, full);
|
||||
@@ -241,7 +273,11 @@ export default async function uploadAndRecordMedia({
|
||||
media_thumbnail_write_path:
|
||||
media_type === "image" ? media_thumbnail_write_path : undefined,
|
||||
media_type: media_type as any,
|
||||
mime_type: media_type == "image" ? "jpeg" : media_ext,
|
||||
/**
|
||||
* The image's real type, not always `jpeg`: a stored PNG that the
|
||||
* row called a JPEG was served and read as one.
|
||||
*/
|
||||
mime_type: image_output ? image_output.mime_type : media_ext,
|
||||
text_content: media_text_content,
|
||||
};
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import type {
|
||||
BUN_SQLITE_COBBLE_MEDIA,
|
||||
BUN_SQLITE_COBBLE_WORKSPACES,
|
||||
BunSQLiteTables,
|
||||
} from "@/db/types/db";
|
||||
@@ -8,8 +9,9 @@ import assertPromptLength from "@/src/utils/assert-prompt-length";
|
||||
import grabUserWorkspace from "./grab-user-workspace";
|
||||
import _ from "lodash";
|
||||
import assertCrewManager from "@/src/functions/backend/db/access/assert-crew-manager";
|
||||
import deleteMedia from "@/src/functions/backend/db/media/delete-media";
|
||||
import copyMediaAsPicture from "@/src/functions/backend/db/profile-images/copy-media-as-picture";
|
||||
import grabOwnPictureMedia from "@/src/functions/backend/db/profile-images/grab-own-picture-media";
|
||||
import removeReplacedPicture from "@/src/functions/backend/db/profile-images/remove-replaced-picture";
|
||||
import resolveWorkspacePicture from "@/src/utils/resolve-workspace-picture";
|
||||
|
||||
type Params = {
|
||||
@@ -27,13 +29,20 @@ type Params = {
|
||||
* it, or undefined to leave it alone.
|
||||
*/
|
||||
image_media_id?: number | null;
|
||||
/**
|
||||
* The image to copy for the picture, for a caller that already
|
||||
* resolved one (a bot naming one of its files). Copied, so the
|
||||
* source's row and bytes stay where they are.
|
||||
*/
|
||||
picture_source?: BUN_SQLITE_COBBLE_MEDIA | null;
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* Updates one of the user's workspaces. Its `system_prompt` is shared
|
||||
* context for every bot in it, and `image_media_id` its picture: a media
|
||||
* id replaces it (the old picture is deleted), null removes it.
|
||||
* id replaces it (the old picture is deleted), null removes it, and
|
||||
* `picture_source` copies an image into a new picture row first.
|
||||
*/
|
||||
export default async function updateWorkspace({
|
||||
user_id,
|
||||
@@ -50,13 +59,33 @@ export default async function updateWorkspace({
|
||||
throw new AppError(`Workspace not found.`);
|
||||
}
|
||||
|
||||
const source = data.picture_source;
|
||||
|
||||
/**
|
||||
* A source that is already the picture changes nothing: copying it
|
||||
* again would only churn a row and delete the old one for no reason.
|
||||
*/
|
||||
const requested =
|
||||
source?.id && Number(source.id) == Number(workspace.image_media_id)
|
||||
? undefined
|
||||
: source
|
||||
? await copyMediaAsPicture({
|
||||
media: source,
|
||||
user_id,
|
||||
target: "workspace",
|
||||
title: `${workspace.name || "Workspace"} picture`,
|
||||
})
|
||||
: data.image_media_id;
|
||||
|
||||
const image_media_id = resolveWorkspacePicture({
|
||||
requested: data.image_media_id,
|
||||
requested,
|
||||
current: workspace.image_media_id,
|
||||
media:
|
||||
data.image_media_id == null
|
||||
requested == null
|
||||
? null
|
||||
: await grabOwnPictureMedia({ media_id: data.image_media_id, user_id }),
|
||||
: source
|
||||
? { id: requested, media_type: "image" }
|
||||
: await grabOwnPictureMedia({ media_id: requested, user_id }),
|
||||
});
|
||||
|
||||
const res = await BunSQLite.update<
|
||||
@@ -87,16 +116,15 @@ export default async function updateWorkspace({
|
||||
|
||||
/**
|
||||
* The picture is written, so the one it replaced isn't shown anywhere
|
||||
* any more.
|
||||
* any more. Only a picture is deleted (`removeReplacedPicture`): a row
|
||||
* a chat message still attaches stays, so the message keeps loading.
|
||||
*/
|
||||
if (
|
||||
image_media_id !== undefined &&
|
||||
workspace.image_media_id &&
|
||||
Number(workspace.image_media_id) != Number(image_media_id)
|
||||
) {
|
||||
await deleteMedia({
|
||||
id: workspace.image_media_id,
|
||||
user_id: Number(user_id),
|
||||
}).catch(() => {});
|
||||
if (image_media_id !== undefined) {
|
||||
await removeReplacedPicture({
|
||||
previous: workspace.image_media_id,
|
||||
next: image_media_id,
|
||||
user_id,
|
||||
target: "workspace",
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { BUN_SQLITE_COBBLE_BOTS, BUN_SQLITE_COBBLE_COMPUTERS } from "@/db/types/db";
|
||||
import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers";
|
||||
import grabCrewWorkers from "@/src/functions/backend/db/bots/grab-crew-workers";
|
||||
import type { StorageScope } from "@/src/utils/storage-access";
|
||||
import type { HarnessTool } from "../../types";
|
||||
import askToTakeOverTool from "../computer/ask-to-take-over-tool";
|
||||
import searchComputersTool from "../computer/search-computers-tool";
|
||||
@@ -41,6 +42,11 @@ type Params = {
|
||||
* The user message being answered.
|
||||
*/
|
||||
message_id?: number | string;
|
||||
/**
|
||||
* The files the run reaches (`grabRunScopes`), so a picture it sets is
|
||||
* copied from one it can already read.
|
||||
*/
|
||||
scopes?: StorageScope[];
|
||||
/**
|
||||
* Called when the bot asks the user to take over a computer.
|
||||
*/
|
||||
@@ -70,6 +76,7 @@ export default async function grabBotChatTools({
|
||||
session_id,
|
||||
workspace_id,
|
||||
message_id,
|
||||
scopes = [],
|
||||
onTakeOver,
|
||||
}: Params): Promise<HarnessTool[]> {
|
||||
const bot_id = bot.id || 0;
|
||||
@@ -81,6 +88,7 @@ export default async function grabBotChatTools({
|
||||
computers,
|
||||
session_id,
|
||||
is_coordinator: bot.kind == "coordinator",
|
||||
scopes,
|
||||
})),
|
||||
startTaskTool({
|
||||
user_id,
|
||||
@@ -104,7 +112,7 @@ export default async function grabBotChatTools({
|
||||
tools.push(createWorkspaceTool({ user_id, workspaces }));
|
||||
|
||||
if (workspaces.length) {
|
||||
tools.push(updateWorkspaceTool({ user_id, workspaces, workspace_id }));
|
||||
tools.push(updateWorkspaceTool({ user_id, workspaces, workspace_id, scopes }));
|
||||
}
|
||||
|
||||
tools.push(createBotTool({ user_id, workspaces, workspace_id }));
|
||||
|
||||
@@ -9,6 +9,8 @@ import updateScheduleTool from "./update-schedule-tool";
|
||||
import updateInstructionsTool from "./update-instructions-tool";
|
||||
import updateMemoryTool from "./update-memory-tool";
|
||||
import reactToMessageTool from "./react-to-message-tool";
|
||||
import setPictureTool from "./set-picture-tool";
|
||||
import type { StorageScope } from "@/src/utils/storage-access";
|
||||
|
||||
type Params = {
|
||||
user_id: number | string;
|
||||
@@ -31,6 +33,11 @@ type Params = {
|
||||
* add to any bot's or workspace's list.
|
||||
*/
|
||||
is_coordinator?: boolean;
|
||||
/**
|
||||
* The files the run reaches (`grabRunScopes`), for the picture it may
|
||||
* set from one.
|
||||
*/
|
||||
scopes?: StorageScope[];
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -45,8 +52,10 @@ export default async function grabBotTools({
|
||||
computers = [],
|
||||
session_id,
|
||||
is_coordinator,
|
||||
scopes = [],
|
||||
}: Params): Promise<HarnessTool[]> {
|
||||
return [
|
||||
setPictureTool({ user_id, bot_id, scopes, is_coordinator }),
|
||||
updateMemoryTool({ user_id, bot_id, workspace_id }),
|
||||
updateInstructionsTool({ user_id, bot_id, workspace_id }),
|
||||
createScheduleTool({ user_id, bot_id, workspace_id, computers }),
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import setPictureTool from "./set-picture-tool";
|
||||
|
||||
const signal = new AbortController().signal;
|
||||
|
||||
describe("setPictureTool", () => {
|
||||
test("a worker sets its own picture, and is told it may not dress another", async () => {
|
||||
const tool = setPictureTool({
|
||||
user_id: 1,
|
||||
bot_id: 5,
|
||||
scopes: [{ scope: "bot", owner_id: 5 }],
|
||||
});
|
||||
|
||||
await expect(
|
||||
tool.run({ args: { bot_id: 6, remove: true }, signal }),
|
||||
).rejects.toThrow(/only set your own picture/);
|
||||
});
|
||||
|
||||
test("the coordinator may name a worker", async () => {
|
||||
const tool = setPictureTool({
|
||||
user_id: 1,
|
||||
bot_id: 1,
|
||||
scopes: [{ scope: "bot", owner_id: 1 }],
|
||||
is_coordinator: true,
|
||||
});
|
||||
|
||||
/** Its own bot id is accepted as well: nothing reaches the guard. */
|
||||
await expect(
|
||||
tool.run({ args: { bot_id: 1, file_id: "[file:0]" }, signal }),
|
||||
).rejects.not.toThrow(/only set your own picture/);
|
||||
});
|
||||
|
||||
test("needs a file, a media id or remove", () => {
|
||||
const tool = setPictureTool({ user_id: 1, bot_id: 5, scopes: [] });
|
||||
|
||||
expect(tool.schema?.safeParse({}).success).toBe(false);
|
||||
expect(tool.schema?.safeParse({ remove: true }).success).toBe(true);
|
||||
expect(tool.schema?.safeParse({ file_id: "[file:12]" }).success).toBe(true);
|
||||
expect(tool.schema?.safeParse({ file_id: 12 }).success).toBe(true);
|
||||
expect(tool.schema?.safeParse({ media_id: 4 }).success).toBe(true);
|
||||
});
|
||||
|
||||
test("is named as a built-in, so a custom tool can't shadow it", () => {
|
||||
expect(setPictureTool({ user_id: 1, bot_id: 5, scopes: [] }).name).toBe("set_picture");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,115 @@
|
||||
import { z } from "zod";
|
||||
import pushBotUpdated from "@/src/bots-queue/push-bot-updated";
|
||||
import grabPictureSource from "@/src/functions/backend/db/profile-images/grab-picture-source";
|
||||
import setBotPicture from "@/src/functions/backend/db/profile-images/set-bot-picture";
|
||||
import parseFileRef from "@/src/utils/parse-file-ref";
|
||||
import type { StorageScope } from "@/src/utils/storage-access";
|
||||
import type { HarnessTool } from "../../types";
|
||||
|
||||
type Params = {
|
||||
user_id: number | string;
|
||||
/**
|
||||
* The bot running: it sets its own picture unless the coordinator names
|
||||
* another worker.
|
||||
*/
|
||||
bot_id: number | string;
|
||||
/**
|
||||
* The files this run reaches (`grabRunScopes`), so a picture is only
|
||||
* ever copied from a file the bot could already read.
|
||||
*/
|
||||
scopes: StorageScope[];
|
||||
/**
|
||||
* Whether the run is the coordinator, which may dress any worker.
|
||||
*/
|
||||
is_coordinator?: boolean;
|
||||
};
|
||||
|
||||
type Args = {
|
||||
file_id?: string | number;
|
||||
media_id?: number;
|
||||
bot_id?: number;
|
||||
remove?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* Lets a bot set its own picture from a file it can see, and the
|
||||
* coordinator set any worker's: an avatar in its place in the rail, in the
|
||||
* thread and in the crew list. Only a super admin or an admin sets one
|
||||
* (`setBotPicture` checks it), so a run for anyone else is told so.
|
||||
*/
|
||||
export default function setPictureTool({
|
||||
user_id,
|
||||
bot_id,
|
||||
scopes,
|
||||
is_coordinator,
|
||||
}: Params): HarnessTool<Args> {
|
||||
return {
|
||||
name: "set_picture",
|
||||
description: `Set a bot's picture (its avatar, shown in the rail and beside its messages) from an image in your files, or remove it. You set your own; ${
|
||||
is_coordinator ? "as the coordinator you can pass bot_id to dress a worker" : "only the coordinator sets another bot's"
|
||||
}. Give \`file_id\` with the image's [file:ID] ref, e.g. "[file:12]" (list_files shows them; what the user sends you lands in uploads/), or \`media_id\` for an image they uploaded. An SVG or another kind of file is turned into a PNG first, then that file is used. The picture has to be an image and at most 5 MB; a square one looks best.`,
|
||||
schema: z
|
||||
.object({
|
||||
file_id: z
|
||||
.union([z.string().max(60), z.number().int().positive()])
|
||||
.optional()
|
||||
.describe(`The image's [file:ID] ref, e.g. "[file:12]"`),
|
||||
media_id: z
|
||||
.number()
|
||||
.int()
|
||||
.positive()
|
||||
.optional()
|
||||
.describe(`An image the user uploaded, by media id`),
|
||||
bot_id: is_coordinator
|
||||
? z
|
||||
.number()
|
||||
.int()
|
||||
.positive()
|
||||
.optional()
|
||||
.describe(`The worker to dress; leave it out for your own picture`)
|
||||
: z.number().int().positive().optional(),
|
||||
remove: z.boolean().optional().describe(`Take the picture away`),
|
||||
})
|
||||
.refine(
|
||||
(args) =>
|
||||
Boolean(args.remove) ||
|
||||
Boolean(args.file_id) ||
|
||||
Boolean(args.media_id),
|
||||
"Give the image's file_id or media_id, or set remove.",
|
||||
),
|
||||
run: async ({ args }) => {
|
||||
const target = args.bot_id ?? Number(bot_id);
|
||||
|
||||
if (Number(target) != Number(bot_id) && !is_coordinator) {
|
||||
throw new Error(
|
||||
`You can only set your own picture. The coordinator sets another bot's.`,
|
||||
);
|
||||
}
|
||||
|
||||
const media = args.remove
|
||||
? null
|
||||
: await grabPictureSource({
|
||||
user_id,
|
||||
scopes,
|
||||
file_id: parseFileRef({ ref: args.file_id }),
|
||||
media_id: args.media_id,
|
||||
});
|
||||
|
||||
if (!args.remove && !media) {
|
||||
throw new Error(
|
||||
`Give the image's [file:ID] ref or its media_id. list_files lists the refs of the images you have.`,
|
||||
);
|
||||
}
|
||||
|
||||
const saved = await setBotPicture({ user_id, bot_id: target, media });
|
||||
|
||||
await pushBotUpdated({ bot_id: target });
|
||||
|
||||
const whose = Number(target) == Number(bot_id) ? "Your" : "That bot's";
|
||||
|
||||
return media
|
||||
? `${whose} picture is saved. Tell the user it changed; they may need to reload to see it everywhere.`
|
||||
: `${whose} picture is removed, so it shows its stone again.`;
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -24,8 +24,9 @@ type Args = {
|
||||
/**
|
||||
* Lets the coordinator change an existing crew member: its name, its
|
||||
* one-line job, and its core instructions (the start of its prompt in its
|
||||
* own chat and in every workspace). Its colour, stone and picture stay as
|
||||
* they are.
|
||||
* own chat and in every workspace). Its colour and stone stay as they are,
|
||||
* and its picture is set with `set_picture` (`set_picture_tool.ts`), which
|
||||
* also lets a bot dress itself.
|
||||
* Workspace-only instructions have their own tool
|
||||
* (`set_bot_workspace_instructions`).
|
||||
*/
|
||||
|
||||
@@ -1,7 +1,11 @@
|
||||
import { z } from "zod";
|
||||
import type { BUN_SQLITE_COBBLE_WORKSPACES } from "@/db/types/db";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
import pushWorkspaceUpdated from "@/src/bots-queue/push-workspace-updated";
|
||||
import grabPictureSource from "@/src/functions/backend/db/profile-images/grab-picture-source";
|
||||
import updateWorkspace from "@/src/functions/backend/db/workspaces/update-workspace";
|
||||
import parseFileRef from "@/src/utils/parse-file-ref";
|
||||
import type { StorageScope } from "@/src/utils/storage-access";
|
||||
import type { HarnessTool } from "../../types";
|
||||
|
||||
type Params = {
|
||||
@@ -15,6 +19,11 @@ type Params = {
|
||||
* given.
|
||||
*/
|
||||
workspace_id?: number | string | null;
|
||||
/**
|
||||
* The files this run reaches (`grabRunScopes`): a picture is copied
|
||||
* from one of them, so a bot can set the logo it was sent.
|
||||
*/
|
||||
scopes?: StorageScope[];
|
||||
};
|
||||
|
||||
type Args = {
|
||||
@@ -23,8 +32,13 @@ type Args = {
|
||||
description?: string;
|
||||
context?: string;
|
||||
/**
|
||||
* A picture for the workspace: the media id the user uploaded, or null
|
||||
* to remove it.
|
||||
* A picture for the workspace, from a file the run can see: its
|
||||
* `[file:ID]` ref.
|
||||
*/
|
||||
picture_file_id?: string | number | null;
|
||||
/**
|
||||
* A picture for the workspace: the media id of an image the user
|
||||
* uploaded, or null to remove it.
|
||||
*/
|
||||
picture_media_id?: number | null;
|
||||
};
|
||||
@@ -38,27 +52,43 @@ export default function updateWorkspaceTool({
|
||||
user_id,
|
||||
workspaces,
|
||||
workspace_id,
|
||||
scopes = [],
|
||||
}: Params): HarnessTool<Args> {
|
||||
const list = workspaces.map((w) => `${w.id}: ${w.name}`).join("; ");
|
||||
|
||||
return {
|
||||
name: "update_workspace",
|
||||
description: `Change a workspace: its name, its one-line description, its context (markdown every bot working there gets in its prompt), or its picture. Only the fields you give change; \`context\` replaces the whole text, so keep what still applies. Use it when the user asks, or to record something every bot in that workspace must know.${workspace_id ? " Leave out workspace_id for this workspace." : ""} Workspaces: ${list}.`,
|
||||
schema: z.object({
|
||||
workspace_id: z.number().int().optional(),
|
||||
name: z.string().trim().min(1).max(60).optional(),
|
||||
description: z.string().trim().max(240).optional(),
|
||||
context: z.string().trim().max(AppData.PromptMaxLength).optional(),
|
||||
picture_media_id: z
|
||||
.number()
|
||||
.int()
|
||||
.positive()
|
||||
.nullable()
|
||||
.optional()
|
||||
.describe(
|
||||
`An image the user uploaded, by media id; null removes the picture. Leave it out to keep the one it has.`,
|
||||
),
|
||||
}),
|
||||
description: `Change a workspace: its name, its one-line description, its context (markdown every bot working there gets in its prompt), or its picture. Only the fields you give change; \`context\` replaces the whole text, so keep what still applies. Use it when the user asks, or to record something every bot in that workspace must know.${workspace_id ? " Leave out workspace_id for this workspace." : ""} For the picture give \`picture_file_id\` with the image's [file:ID] ref, e.g. "[file:12]" (list_files shows them, and what the user sends you lands in uploads/), or \`picture_media_id\` for an image they uploaded; null removes it. An SVG or another kind of file is turned into a PNG first, then that file is used. Workspaces: ${list}.`,
|
||||
schema: z
|
||||
.object({
|
||||
workspace_id: z.number().int().optional(),
|
||||
name: z.string().trim().min(1).max(60).optional(),
|
||||
description: z.string().trim().max(240).optional(),
|
||||
context: z.string().trim().max(AppData.PromptMaxLength).optional(),
|
||||
picture_file_id: z
|
||||
.union([z.string().max(60), z.number().int().positive()])
|
||||
.nullable()
|
||||
.optional()
|
||||
.describe(`The picture's [file:ID] ref, e.g. "[file:12]"`),
|
||||
picture_media_id: z
|
||||
.number()
|
||||
.int()
|
||||
.positive()
|
||||
.nullable()
|
||||
.optional()
|
||||
.describe(
|
||||
`An image the user uploaded, by media id; null removes the picture. Leave it out to keep the one it has.`,
|
||||
),
|
||||
})
|
||||
.refine(
|
||||
(args) =>
|
||||
args.name !== undefined ||
|
||||
args.description !== undefined ||
|
||||
args.context !== undefined ||
|
||||
args.picture_file_id !== undefined ||
|
||||
args.picture_media_id !== undefined,
|
||||
"Give a name, a description, a context or a picture to change.",
|
||||
),
|
||||
run: async ({ args }) => {
|
||||
const target = workspaces.find((w) => w.id == (args.workspace_id || workspace_id));
|
||||
|
||||
@@ -68,16 +98,22 @@ export default function updateWorkspaceTool({
|
||||
);
|
||||
}
|
||||
|
||||
if (
|
||||
args.name === undefined &&
|
||||
args.description === undefined &&
|
||||
args.context === undefined &&
|
||||
args.picture_media_id === undefined
|
||||
) {
|
||||
throw new Error(
|
||||
`Give a name, a description, a context or a picture to change.`,
|
||||
);
|
||||
}
|
||||
const removing =
|
||||
args.picture_file_id === null || args.picture_media_id === null;
|
||||
|
||||
/**
|
||||
* A file ref is resolved against the run's directories
|
||||
* (`grabPictureSource`), so a bot can only set what it could
|
||||
* already read.
|
||||
*/
|
||||
const picture_source =
|
||||
args.picture_file_id === undefined || args.picture_file_id === null
|
||||
? null
|
||||
: await grabPictureSource({
|
||||
user_id,
|
||||
scopes,
|
||||
file_id: parseFileRef({ ref: args.picture_file_id }),
|
||||
});
|
||||
|
||||
await updateWorkspace({
|
||||
user_id,
|
||||
@@ -87,10 +123,26 @@ export default function updateWorkspaceTool({
|
||||
description: args.description,
|
||||
system_prompt: args.context,
|
||||
image_media_id: args.picture_media_id,
|
||||
picture_source,
|
||||
},
|
||||
});
|
||||
|
||||
return `${args.name || target.name} is saved. Bots there get the change from their next run.`;
|
||||
/**
|
||||
* Its picture reaches every open tab on its own, whether the
|
||||
* run is in this process or another (`pushWorkspaceUpdated`,
|
||||
* `liveRelay`).
|
||||
*/
|
||||
if (removing || picture_source) {
|
||||
await pushWorkspaceUpdated({ workspace_id: target.id });
|
||||
}
|
||||
|
||||
const picture = removing
|
||||
? ` Its picture is removed.`
|
||||
: picture_source
|
||||
? ` Its picture is set from that file.`
|
||||
: "";
|
||||
|
||||
return `${args.name || target.name} is saved. Bots there get the change from their next run.${picture}`;
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db";
|
||||
import grabUserWorkspaces from "@/src/functions/backend/db/workspaces/grab-user-workspaces";
|
||||
import grabRunScopes from "@/src/functions/backend/db/bot-files/grab-run-scopes";
|
||||
import type { StorageScope } from "@/src/utils/storage-access";
|
||||
import type { HarnessTool } from "../../types";
|
||||
import generateImageTool from "./generate-image-tool";
|
||||
import grabStorageTools from "./grab-storage-tools";
|
||||
@@ -11,6 +12,11 @@ type Params = {
|
||||
user_id: number | string;
|
||||
workspace_id?: number | string | null;
|
||||
session_id?: number | string;
|
||||
/**
|
||||
* The files the run reaches, when the caller already read them
|
||||
* (`grabRunScopes`); read here otherwise.
|
||||
*/
|
||||
scopes?: StorageScope[];
|
||||
/**
|
||||
* Called for a file the bot wrote with "attach" set, so the reply
|
||||
* carries it.
|
||||
@@ -30,19 +36,15 @@ export default async function grabBotFileTools({
|
||||
user_id,
|
||||
workspace_id,
|
||||
session_id,
|
||||
scopes,
|
||||
onAttach,
|
||||
}: Params): Promise<HarnessTool[]> {
|
||||
const all_workspace_ids =
|
||||
bot.kind == "coordinator"
|
||||
? (await grabUserWorkspaces({ user_id })).filter((w) => !w.archived).map((w) => Number(w.id))
|
||||
: [];
|
||||
|
||||
const context = storageToolContext({
|
||||
bot,
|
||||
user_id,
|
||||
workspace_id,
|
||||
session_id,
|
||||
all_workspace_ids,
|
||||
scopes: scopes || (await grabRunScopes({ bot, user_id, workspace_id })),
|
||||
onAttach,
|
||||
});
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db";
|
||||
import grabStorageScopes, { type StorageScope } from "@/src/utils/storage-access";
|
||||
import type { StorageScope } from "@/src/utils/storage-access";
|
||||
|
||||
export type StorageToolContext = {
|
||||
/**
|
||||
@@ -32,34 +32,29 @@ type Params = {
|
||||
workspace_id?: number | string | null;
|
||||
session_id?: number | string;
|
||||
/**
|
||||
* Every workspace, for the coordinator (it reaches them all).
|
||||
* The directories the run may touch (`grabRunScopes`): the bot's own
|
||||
* first, then the workspace's or, for the coordinator, every one's.
|
||||
*/
|
||||
all_workspace_ids?: number[];
|
||||
scopes: StorageScope[];
|
||||
onAttach?: StorageToolContext["onAttach"];
|
||||
};
|
||||
|
||||
/**
|
||||
* The directories one run may touch, from the bot it's for. The
|
||||
* coordinator reaches every bot's and workspace's files; a worker its own,
|
||||
* plus its workspace's shared one while it works there.
|
||||
* The context every storage tool works from: which bot it is, whose files it
|
||||
* reaches (`grabRunScopes`, which owns that decision) and where it works.
|
||||
*/
|
||||
export default function storageToolContext({
|
||||
bot,
|
||||
user_id,
|
||||
workspace_id,
|
||||
session_id,
|
||||
all_workspace_ids,
|
||||
scopes,
|
||||
onAttach,
|
||||
}: Params): StorageToolContext {
|
||||
return {
|
||||
user_id,
|
||||
bot_id: bot.id || 0,
|
||||
scopes: grabStorageScopes({
|
||||
bot_id: bot.id || 0,
|
||||
bot_kind: bot.kind,
|
||||
workspace_id,
|
||||
all_workspace_ids,
|
||||
}),
|
||||
scopes,
|
||||
session_id,
|
||||
workspace_id,
|
||||
onAttach,
|
||||
|
||||
@@ -90,6 +90,13 @@ export default function useLiveBots({ bots, open_bot_id, workspace_id = null }:
|
||||
stone: (data.stone || "") as BUN_SQLITE_COBBLE_BOTS["stone"],
|
||||
}
|
||||
: {}),
|
||||
/**
|
||||
* A picture the bot just set: its stone is
|
||||
* replaced by it without a reload.
|
||||
*/
|
||||
...(data.image_media_id !== undefined
|
||||
? { image_media_id: data.image_media_id || undefined }
|
||||
: {}),
|
||||
}
|
||||
: bot,
|
||||
);
|
||||
|
||||
@@ -4,6 +4,7 @@ import useServerEvent from "@/src/hooks/use-server-event";
|
||||
import type {
|
||||
WorkspaceCreatedData,
|
||||
WorkspaceRemovedData,
|
||||
WorkspaceUpdatedData,
|
||||
WorkspaceWithActivity,
|
||||
} from "@/src/types";
|
||||
|
||||
@@ -45,6 +46,18 @@ export default function useLiveWorkspaces({ workspaces, open_workspace_id }: Par
|
||||
setLive((prev) => prev.filter((workspace) => workspace.id != data.workspace_id)),
|
||||
});
|
||||
|
||||
useServerEvent({
|
||||
event: "server:workspace-updated",
|
||||
onData: (data: WorkspaceUpdatedData) =>
|
||||
setLive((prev) =>
|
||||
prev.map((workspace) =>
|
||||
workspace.id == data.workspace_id
|
||||
? { ...workspace, image_media_id: data.image_media_id || undefined }
|
||||
: workspace,
|
||||
),
|
||||
),
|
||||
});
|
||||
|
||||
useServerEvent({
|
||||
event: "server:bot-message",
|
||||
onData: (message: BUN_SQLITE_COBBLE_BOT_MESSAGES) => {
|
||||
|
||||
@@ -4,6 +4,7 @@ import { logError } from "@/src/functions/backend/logger";
|
||||
import grabConnectedUserIds from "@/src/websocket/(functions)/grab-connected-user-ids";
|
||||
import relayChangedBots from "./relay-changed-bots";
|
||||
import relayChangedSessions from "./relay-changed-sessions";
|
||||
import relayChangedWorkspaces from "./relay-changed-workspaces";
|
||||
import relayCreatedBots from "./relay-created-bots";
|
||||
import relayCreatedWorkspaces from "./relay-created-workspaces";
|
||||
import relayNewBotMessages from "./relay-new-bot-messages";
|
||||
@@ -74,6 +75,7 @@ export default async function liveRelay() {
|
||||
await relayCreatedBots({ since: bots_since, users });
|
||||
await relayCreatedWorkspaces({ since: bots_since, users });
|
||||
await relayChangedBots({ since: bots_since, users });
|
||||
await relayChangedWorkspaces({ since: bots_since, users });
|
||||
cursor.bots_at = now;
|
||||
|
||||
await relayChangedSessions({
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import pushWorkspaceUpdated from "@/src/bots-queue/push-workspace-updated";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* Unix ms: workspaces updated since then are pushed.
|
||||
*/
|
||||
since: number;
|
||||
users: Set<string>;
|
||||
};
|
||||
|
||||
/**
|
||||
* Pushes `server:workspace-updated` for every workspace changed since
|
||||
* `since` to the connected users who see it (`pushWorkspaceUpdated`), so a
|
||||
* picture another process set still lands here. Pushes are idempotent, so
|
||||
* overlapping windows only repeat them.
|
||||
*/
|
||||
export default async function relayChangedWorkspaces({ since, users }: Params) {
|
||||
if (!users.size) return;
|
||||
|
||||
const workspaces =
|
||||
(
|
||||
await BunSQLite.sql<{ id: number }>({
|
||||
sql: `SELECT id FROM workspaces WHERE updated_at > ? AND archived = 0`,
|
||||
values: [since],
|
||||
})
|
||||
).payload || [];
|
||||
|
||||
for (const workspace of workspaces) {
|
||||
if (!workspace.id) continue;
|
||||
|
||||
await pushWorkspaceUpdated({ workspace_id: workspace.id, users });
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,7 @@
|
||||
import userAuth from "@/src/functions/backend/auth/user-auth";
|
||||
import setProfileImage from "@/src/functions/backend/db/profile-images/set-profile-image";
|
||||
import pushBotUpdated from "@/src/bots-queue/push-bot-updated";
|
||||
import pushWorkspaceUpdated from "@/src/bots-queue/push-workspace-updated";
|
||||
import { SetProfileImageRequestSchema } from "@/src/schemas/profile-image-schema";
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import parseWithSchema from "@/src/utils/parse-with-schema";
|
||||
@@ -13,6 +15,9 @@ import type {
|
||||
/**
|
||||
* Sets or removes the picture of one of the session user's bots or
|
||||
* workspaces. Returns the new `image_media_id` (null once removed).
|
||||
*
|
||||
* The change is pushed to everyone who sees it, so a picture set in one
|
||||
* tab shows in the rail of the others without a reload.
|
||||
*/
|
||||
export const handler: BunextAPIRouteHandler<
|
||||
APIResponseObject<{ image_media_id: number | null }>
|
||||
@@ -37,10 +42,18 @@ export const handler: BunextAPIRouteHandler<
|
||||
data: body,
|
||||
});
|
||||
|
||||
const saved = await setProfileImage({ user_id: user.id, target, id, data_url });
|
||||
|
||||
if (target == "bot") {
|
||||
await pushBotUpdated({ bot_id: id });
|
||||
} else {
|
||||
await pushWorkspaceUpdated({ workspace_id: id });
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
msg: data_url ? `Picture saved.` : `Picture removed.`,
|
||||
singleRes: await setProfileImage({ user_id: user.id, target, id, data_url }),
|
||||
singleRes: saved,
|
||||
};
|
||||
} catch (error: unknown) {
|
||||
return { success: false, msg: safeErrorMessage({ error }) };
|
||||
|
||||
@@ -288,6 +288,21 @@ export type BotUpdatedData = {
|
||||
*/
|
||||
color?: string | null;
|
||||
stone?: string | null;
|
||||
/**
|
||||
* Its picture, when the push read it (a bot that set its own face): the
|
||||
* rail and the thread swap the stone for it without a reload. Null means
|
||||
* it has none.
|
||||
*/
|
||||
image_media_id?: number | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* `server:workspace-updated` data: a workspace's picture changed, so the
|
||||
* rail and its header show the new one.
|
||||
*/
|
||||
export type WorkspaceUpdatedData = {
|
||||
workspace_id: number;
|
||||
image_media_id: number | null;
|
||||
};
|
||||
|
||||
/**
|
||||
@@ -1259,6 +1274,10 @@ export const WebSocketEvents = [
|
||||
* `WorkspaceRemovedData`)
|
||||
*/
|
||||
"server:workspace-removed",
|
||||
/**
|
||||
* A workspace the user can see changed (`data`: `WorkspaceUpdatedData`)
|
||||
*/
|
||||
"server:workspace-updated",
|
||||
/**
|
||||
* A message's emoji reactions changed (`data`: `BotMessageReactionData`)
|
||||
*/
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import imageOutputFormat from "./image-output-format";
|
||||
|
||||
describe("imageOutputFormat", () => {
|
||||
test("keeps a PNG a PNG, so its transparency survives", () => {
|
||||
expect(imageOutputFormat({ ext: "png" })).toEqual({
|
||||
format: "png",
|
||||
ext: "png",
|
||||
mime_type: "image/png",
|
||||
});
|
||||
expect(imageOutputFormat({ ext: "logo.PNG" }).format).toBe("png");
|
||||
});
|
||||
|
||||
test("keeps WebP a WebP", () => {
|
||||
expect(imageOutputFormat({ ext: "webp" })).toEqual({
|
||||
format: "webp",
|
||||
ext: "webp",
|
||||
mime_type: "image/webp",
|
||||
});
|
||||
});
|
||||
|
||||
test("stores a GIF as a PNG, which keeps transparency", () => {
|
||||
expect(imageOutputFormat({ ext: "gif" })).toEqual({
|
||||
format: "png",
|
||||
ext: "png",
|
||||
mime_type: "image/png",
|
||||
});
|
||||
});
|
||||
|
||||
test("stores anything else as JPEG, as it always has", () => {
|
||||
for (const ext of ["jpeg", "jpg", "bmp", "tiff", "", null, undefined, "weird"]) {
|
||||
expect(imageOutputFormat({ ext })).toEqual({
|
||||
format: "jpeg",
|
||||
ext: "jpeg",
|
||||
mime_type: "image/jpeg",
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
test("reads the extension out of a name", () => {
|
||||
expect(imageOutputFormat({ ext: "summit-logo.png" }).format).toBe("png");
|
||||
expect(imageOutputFormat({ ext: "media/2/thing.webp" }).format).toBe("webp");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,46 @@
|
||||
type Params = {
|
||||
/**
|
||||
* The source's extension or name, e.g. `png` or `logo.png`.
|
||||
*/
|
||||
ext?: string | null;
|
||||
};
|
||||
|
||||
export type ImageOutputFormat = {
|
||||
/**
|
||||
* What sharp writes with: `png`, `webp` or `jpeg`.
|
||||
*/
|
||||
format: "png" | "webp" | "jpeg";
|
||||
/**
|
||||
* The extension the stored file gets.
|
||||
*/
|
||||
ext: string;
|
||||
/**
|
||||
* The MIME type recorded on the `media` row.
|
||||
*/
|
||||
mime_type: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* What an uploaded image is stored as.
|
||||
*
|
||||
* Pictures (a bot's or a workspace's) keep the format they came in, so a
|
||||
* transparent PNG stays a PNG with its transparency: forcing every image
|
||||
* through JPEG flattened a logo's clear corners onto black. A GIF is stored
|
||||
* as a PNG, which keeps its transparency.
|
||||
*
|
||||
* Everything else (a photo, a screenshot, an attachment) stays JPEG, which
|
||||
* is what it has always been.
|
||||
*/
|
||||
export default function imageOutputFormat({ ext }: Params = {}): ImageOutputFormat {
|
||||
const source = (ext || "")
|
||||
.split(".")
|
||||
.pop()
|
||||
?.toLowerCase()
|
||||
.replace(/[^a-z0-9]/g, "") || "";
|
||||
|
||||
if (source == "png") return { format: "png", ext: "png", mime_type: "image/png" };
|
||||
if (source == "webp") return { format: "webp", ext: "webp", mime_type: "image/webp" };
|
||||
if (source == "gif") return { format: "png", ext: "png", mime_type: "image/png" };
|
||||
|
||||
return { format: "jpeg", ext: "jpeg", mime_type: "image/jpeg" };
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import parseFileRef from "./parse-file-ref";
|
||||
|
||||
describe("parseFileRef", () => {
|
||||
test("reads the ref a bot copies out of a tool result", () => {
|
||||
expect(parseFileRef({ ref: "[file:12]" })).toBe(12);
|
||||
expect(parseFileRef({ ref: " [file: 7] " })).toBe(7);
|
||||
expect(parseFileRef({ ref: "[FILE:3]" })).toBe(3);
|
||||
});
|
||||
|
||||
test("takes a bare id as text or as a number", () => {
|
||||
expect(parseFileRef({ ref: "42" })).toBe(42);
|
||||
expect(parseFileRef({ ref: 42 })).toBe(42);
|
||||
});
|
||||
|
||||
test("refuses anything that isn't a positive whole number", () => {
|
||||
for (const ref of ["", " ", "abc", "[file:abc]", "0", "-2", 0, -2, 1.5, "[file:1.5]", null, undefined]) {
|
||||
expect(parseFileRef({ ref })).toBeNull();
|
||||
}
|
||||
});
|
||||
|
||||
test("refuses a ref that got pasted with other text around it", () => {
|
||||
expect(parseFileRef({ ref: "logo.png [file:12]" })).toBeNull();
|
||||
expect(parseFileRef({ ref: "[file:12][file:13]" })).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,31 @@
|
||||
type Params = {
|
||||
/**
|
||||
* A file ref as a bot writes it (`[file:12]`), an id as text (`"12"`),
|
||||
* or a number.
|
||||
*/
|
||||
ref?: string | number | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* The file id a `[file:ID]` ref names, so a tool can take the ref a bot
|
||||
* copied out of `list_files` instead of asking it for a bare number.
|
||||
* Null when it names nothing usable.
|
||||
*/
|
||||
export default function parseFileRef({ ref }: Params): number | null {
|
||||
if (typeof ref == "number") return Number.isInteger(ref) && ref > 0 ? ref : null;
|
||||
|
||||
if (typeof ref != "string") return null;
|
||||
|
||||
const text = ref.trim();
|
||||
|
||||
if (!text) return null;
|
||||
|
||||
const marked = text.match(/^\[file:\s*(\d+)\s*\]$/i)?.[1];
|
||||
const bare = marked || (/^\d+$/.test(text) ? text : "");
|
||||
|
||||
if (!bare) return null;
|
||||
|
||||
const id = Number(bare);
|
||||
|
||||
return Number.isInteger(id) && id > 0 ? id : null;
|
||||
}
|
||||
@@ -40,11 +40,15 @@ export default function resolveWorkspacePicture({ requested, current, media }: P
|
||||
if (current != null && Number(current) == id) return undefined;
|
||||
|
||||
if (!media || Number(media.id) != id) {
|
||||
throw new AppError(`That picture isn't one of yours. Upload it again.`);
|
||||
throw new AppError(
|
||||
`That picture isn't one you can use. Name a PNG, JPG, WebP or GIF in your files by its [file:ID] ref, or upload it in chat.`,
|
||||
);
|
||||
}
|
||||
|
||||
if (media.media_type != "image") {
|
||||
throw new AppError(`A workspace picture has to be an image.`);
|
||||
throw new AppError(
|
||||
`A workspace picture has to be an image. This one is a ${media.media_type || "file"}: an SVG or a document is turned into a PNG first, then that PNG is set.`,
|
||||
);
|
||||
}
|
||||
|
||||
return id;
|
||||
|
||||
Reference in New Issue
Block a user