Commit Graph
3 Commits
Author SHA1 Message Date
tbenandClaude Opus 5.5 de28d4d9f7 Let generate_image work from reference images, with an image editing model
generate_image takes reference images: files the run reaches as
[file:ID] refs, or the user's uploads by media id, up to four, each
checked like a picture source and limited to raster images.

Calls with references go to a new Image editing harness (no automatic
picks). While it has no model or is off, image generation takes them;
with its fallback on, image generation steps in after its pick fails.
Only models that take image input are tried with references, so none
drops them unseen. OpenRouter gets them on /images as input_references;
other providers as a multipart upload to /images/edits.

The Models pages show image editing's unset state as "Same as image
generation", and the models cache version is bumped so it shows at once.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-28 18:43:10 +01:00
tbenandClaude Opus 5.5 163d9a434c Add dictation and streamed read-aloud to the chat
- A mic in the composer records, transcribes with the bot's
  transcription model (api/admin/bots/transcribe) and writes the text
  into the input at the cursor to edit before sending. Escape cancels;
  recordings aren't stored.
- Every bot message has a read-aloud button that plays
  GET api/admin/bots/speak-message, streamed from the provider as the
  audio is made, so it starts in a couple of seconds. One message plays
  at a time; only the user's own sessions can be read.
- Both buttons stay visible but disabled while the bot has no model for
  them (bot_voice page prop).
- Transcription and speech no longer pick free models automatically:
  they're not available until a model is picked, and Settings says so.
  Transcription lists speech-to-text models only, so the chat-model
  transcript path and its missing-audio check are gone.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 21:21:04 +01:00
tbenandClaude Opus 5.5 6a2665dae9 Add the harness, envs, per-bot models and chat attachments
- Harness: providers, model catalog synced hourly, automatic free picks,
  cooldowns, tool loop with context trimming, MCP computer tools
- Bots queue runs replies and tasks through the harness; jobs wait
  (attempt refunded) while every model is resting
- Envs: Settings > Environment, scoped general/workspace/computer,
  written to computers and redacted from tool output
- Models: Settings > Models, per-harness provider select defaulting to
  OpenRouter, and a models modal on each bot's chat (the coordinator's
  edits the defaults)
- Chat: thinking row while a reply runs, file attachments (upload,
  paste, drop) that replies read as images, audio and document text
- Fix media uploads failing after a user's first file, and private
  files being served from the public media path

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-19 03:38:38 +01:00