Add dictation and streamed read-aloud to the chat

- A mic in the composer records, transcribes with the bot's
  transcription model (api/admin/bots/transcribe) and writes the text
  into the input at the cursor to edit before sending. Escape cancels;
  recordings aren't stored.
- Every bot message has a read-aloud button that plays
  GET api/admin/bots/speak-message, streamed from the provider as the
  audio is made, so it starts in a couple of seconds. One message plays
  at a time; only the user's own sessions can be read.
- Both buttons stay visible but disabled while the bot has no model for
  them (bot_voice page prop).
- Transcription and speech no longer pick free models automatically:
  they're not available until a model is picked, and Settings says so.
  Transcription lists speech-to-text models only, so the chat-model
  transcript path and its missing-audio check are gone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-19 21:21:04 +01:00
co-authored by Claude Opus 5.5
parent 727cbf53c8
commit 163d9a434c
37 changed files with 936 additions and 197 deletions
@@ -0,0 +1,37 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import type { BUN_SQLITE_COBBLE_BOT_MESSAGES } from "@/db/types/db";
import AppError from "@/src/utils/app-error";
import grabUserConversation from "../bot-sessions/grab-user-conversation";
type Params = {
user_id: number;
message_id: number;
};
/**
* A bot message in one of the user's own sessions, for reading aloud,
* with the conversation it's in (so a lost grant blocks it). Throws when
* it isn't theirs.
*/
export default async function grabSpeakableMessage({ user_id, message_id }: Params) {
const message = (
await BunSQLite.sql<BUN_SQLITE_COBBLE_BOT_MESSAGES>({
sql: `SELECT m.* FROM bot_messages m
JOIN bot_sessions s ON s.id = m.session_id
WHERE m.id = ? AND m.author = 'bot' AND s.user_id = ?`,
values: [message_id, user_id],
})
).payload?.[0];
if (!message?.id || !message.bot_id) {
throw new AppError(`Message not found`);
}
const conversation = await grabUserConversation({
user_id,
bot_id: message.bot_id,
workspace_id: message.workspace_id || null,
});
return { message, conversation };
}