Moduletrace/cobble
2
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 10 Wiki Activity

Compare commits

base: Moduletrace/cobble:v0.1.5
Branches Tags
Moduletrace/cobble:main
Moduletrace/cobble:v0.2.7 Moduletrace/cobble:v0.2.6 Moduletrace/cobble:v0.2.5 Moduletrace/cobble:v0.2.2 Moduletrace/cobble:v0.2.0 Moduletrace/cobble:v0.1.8 Moduletrace/cobble:v0.1.7 Moduletrace/cobble:v0.1.6 Moduletrace/cobble:v0.1.5 Moduletrace/cobble:v0.1.4 Moduletrace/cobble:v0.1.3 Moduletrace/cobble:v0.1.2 Moduletrace/cobble:v0.1.1 Moduletrace/cobble:v0.1.0
...
compare: Moduletrace/cobble:v0.1.8
Branches Tags
Moduletrace/cobble:main
Moduletrace/cobble:v0.2.7 Moduletrace/cobble:v0.2.6 Moduletrace/cobble:v0.2.5 Moduletrace/cobble:v0.2.2 Moduletrace/cobble:v0.2.0 Moduletrace/cobble:v0.1.8 Moduletrace/cobble:v0.1.7 Moduletrace/cobble:v0.1.6 Moduletrace/cobble:v0.1.5 Moduletrace/cobble:v0.1.4 Moduletrace/cobble:v0.1.3 Moduletrace/cobble:v0.1.2 Moduletrace/cobble:v0.1.1 Moduletrace/cobble:v0.1.0
13 Commits
v0.1.5 ... v0.1.8
Author SHA1 Message Date
tben 2e8e8de976 Release v0.1.8
CI / check (push) Canceled after 0s
Details
Release image / image (release) Canceled after 0s
Details
2026-10-23 18:00:06 +01:00
tben e75ab28041 Open a computer chip in the viewer instead of switching to it 
A computer a bot names in a reply is now a button that opens it in the
shared computer viewer modal, the bot's own computer included (sunken and
checked, as before). Switching stays in the header's computers modal, so
the chip has one job. A computer the bot can no longer use stays a plain
name.
 2026-10-23 17:58:08 +01:00
tben 4261535741 Tell every bot its own id, in every run 
The bot's id never reached its prompt: `composeBotPrompt` only took its
instructions, the appendix, its memory and its workspace, and
`grabBotPrompt` never passed `bot.id` or `bot.name` on. A bot could only
learn its id by accident, from a tool description that named it, so
"which bot am I" was unanswerable in most contexts.

- Open the prompt with who the bot is (`composeIdentity`): its name,
  whether it's the coordinator or a worker, its role, and its id, said
  to be how tools and refs point back at it. `grabBotPrompt` now passes
  the bot, which covers chat replies, tasks, routines and delegated
  tasks, since both runners build their prompt there.
- Turn the notes' example refs into functions bound to the running bot:
  `chatReplyNote`, `botFilesNote` and `coordinatorReplyNote` take its id,
  so the folder-ref example no longer points at a hardcoded `owner_id: 5`
  (another bot's folder) nor `[bot:3]`. ComputerFilesNote keeps the
  computer it's given.
- Stop `create_task` from hiding the worker's own id in its own chat.

Tests: four new `composeBotPrompt` cases pin the identity block (worker,
coordinator, unnamed, no bot at all), and a new `harness-prompts.test.ts`
checks each note's own-folder and self refs point at exactly the running
bot, for three ids.
 2026-10-23 17:40:28 +01:00
tben 0166f31cfc Release v0.1.7
CI / check (push) Canceled after 0s
Details
Release image / image (release) Canceled after 0s
Details
2026-10-23 16:50:51 +01:00
tben caa9ad87e4 Remove sticky breadcrumbs 2026-10-23 16:49:52 +01:00
tben 30a4df183c Give every admin page a breadcrumb trail 
The trail is twui's Breadcrumbs reading Home, the sections and the page
off the address, rendered as the first thing in SettingsShell's content
column, so every page that is not a conversation gets one without a page
touching it.

The last crumb is named by the page's own title rather than the address:
a record reads Scout instead of its id, /admin/logs reads Activity, and
/admin/settings/mcps reads MCPs. That replaces the breadcrumb_title prop
the four settings record pages had to pass, so settings-breadcrumbs.tsx
goes.

hasBreadcrumbTrail drops the trail on the crew of bots, home, and on
anything outside /admin; the trailing slash and query string are ignored,
so a folder in the files browser keeps the trail of the page it is on.
 2026-10-23 16:49:01 +01:00
tben ee687e1d31 Chunk Tailwind classes with tailwind-merge 
Say it once in AGENTS.md: a className is twMerge(...) with one chunk per claim, so a later chunk beats an earlier one instead of depending on CSS source order.
 2026-10-23 16:42:01 +01:00
tben 024d847b9a Let the install set how many steps a run may take 
The two limits were compiled in: a task run stopped at
AppData.HarnessMaxToolSteps (200) and a chat reply at
AppData.BotReplyMaxToolSteps (8), so an install that needed a longer task
had to rebuild.

Both are now `app_settings` rows (`harness_max_tool_steps` and
`bot_reply_max_tool_steps`), read at the two loop call sites through
grabHarnessMaxToolSteps and grabReplyMaxToolSteps, and edited by the super
admin on Settings > General beside the prompt appendix. Zero clears a
limit back to the code default, like the bot file quota.

readStepLimit keeps the saved number only when it is a whole number of at
least 1 and no more than AppData.BotStepLimitMax (1000), so a blank, a
zero or a typo falls back instead of stopping every run before its first
turn, and a huge number still ends in the wrap-up report.

runTextLoop keeps AppData.HarnessMaxToolSteps as its default, so it stays
free of database reads and its tests run as they did.
 2026-10-23 16:20:38 +01:00
tben 06586a6dc8 Release v0.1.6
CI / check (push) Canceled after 0s
Details
Release image / image (release) Canceled after 0s
Details
2026-10-23 16:05:01 +01:00
tben 55fc5ee97b Follow the thread's bottom, and offer a jump back to it 
Sending a message while scrolled up left it out of sight: the thread only
ever anchored its bottom, so nothing brought the view down. The column now
follows new messages while it sits at the bottom, stays put while it's
scrolled up, and always brings a message the user just sent into view.
While the thread sits above its bottom, "Jump to latest" floats over it.

Whether the thread is at its bottom is read from where the content ends
rather than from `scrollTop`, whose sign a reversed column reports
differently across engines. Content that grows after it lands (an image,
a wider window) is followed too, so the newest message never slips under
the composer.
 2026-10-23 16:03:37 +01:00
tben d9aea8e9c8 Record what every model call spends, and show it 
Nothing anywhere recorded what a model call cost: the harness used a
provider's usage numbers only to decide whether a turn had finished, then
dropped them. So a reply could not say what it took to make, one model
could not be compared with another on price, and there was no way to ask
what a day, a week or a month of the crew's work came to.

- `token_usages` records one row per call: user, bot, workspace, session,
  job and message, the harness, the provider and model that actually
  answered, the prompt and completion counts, and the cost when the
  provider reports one.
- `harness-usage-scope.ts` carries those ids in an AsyncLocalStorage
  scope that `run-bot-job.ts` enters once, so a run's turns, its session
  summary, its transcriptions, images and embeddings are all attributed
  without threading ids through every harness function. Nested runs merge
  rather than replace, and a null genuinely clears a field (a bot's own
  chat has no workspace).
- Work with no run behind it is charged to its owner instead:
  `run-with-fallback-scope.ts` reaches the workspace's creator else the
  bot's, and deliberately does not override a real run, so an ingest a
  bot asked for is not misattributed to the bot's owner.
- All six harnesses record through `withCandidates`'s `on_success`.
  Recording never throws: a model that answered must not lose its answer
  to a failed insert.
- A job's calls land before its message exists, so `complete-bot-job.ts`
  back-fills `message_id` once the message is inserted.
- The message's actions row shows what it cost, beside the timestamp and
  the play button, with the breakdown in its title.
- `/admin/usage` totals the install's spending for 7, 30 or 90 days: the
  total against the previous period, a day-by-day chart, a split per kind
  of call, and a table per model with the price per thousand tokens
  worked out from cost and tokens, which is what makes two models
  comparable. Crew managers only, since it is the whole install's work
  rather than one user's conversation. It sits with Routines and Activity
  in the user popover rather than under Settings, because it is a record
  of what happened, not something to configure.

Days are grouped in local time (`date(created_at / 1000, 'unixepoch',
'localtime')`), matching the local midnights the window builds and the
`YYYY-MM-DD` buckets it looks up. Grouping by the bare UTC day put a late
evening's calls under a name no bucket carried, so they vanished and the
chart's total silently disagreed with the totals card; a test pins it.

`canSeeRailItem` now decides who sees a popover or rail entry. The two
each filtered with their own inline rule and only one understood
`can_view`, so a gated page could be offered in the popover to everyone.
 2026-10-23 15:41:41 +01:00
tben 0048324d8a Let a delegated task choose the workspace it runs in 
delegate_task had no workspace argument: a task handed over from the
coordinator's own chat always ran in the worker's own chat, on that
worker's memory, files and directly assigned computers alone, even when
the work belonged to a workspace. The coordinator could not say where the
work goes, and the prompt never said the two differ.

- delegate_task takes an optional workspace_id: the worker runs the task
  in that workspace's context, with its own instructions and memory
  there, the project's files and the workspace's computers, and the
  result lands in its conversation in that workspace.
- A worker missing from that workspace is added first (pushBotCreated)
  and the reply says so.
- Handed over from inside a workspace the task always runs there; a
  different workspace_id is refused instead of ignored.
- The coordinator prompt, CoordinatorReplyNote and the tool description
  now spell the distinction out.
 2026-10-23 14:25:21 +01:00
tben f7ef218507 Take the Gitea registry out of CI and mirror it separately
CI / check (push) Canceled after 0s
Details 
The Gitea registry is on a VPN, which a GitHub runner can never reach, so
the mirror step in the release workflow could only ever fail: its last four
releases reported a step that did nothing. The workflow now builds and
pushes to GHCR alone, which is the registry the installer defaults to, and
nothing in CI mentions Gitea.

The image reaches the Gitea registry through `scripts/mirror-gitea.sh`
(`bun run mirror:gitea`) instead, run in parallel with a release and from the
VPN side: it copies from the Gitea host over ssh, where skopeo can see both
registries, so nothing has to be installed there. `skopeo copy --all` keeps
the manifest list and its signatures, and every destination tag is read back
and compared with GHCR's digest, so a partial copy fails loudly instead of
being assumed to have worked. `latest` and `major.minor` are only moved while
that release still owns them in GHCR, so mirroring an older release never
rolls them back.

Neither direction can fail the other now, which is the point: a registry that
is down, slow or full no longer takes a release with it.
 2026-10-23 12:03:23 +01:00
124 changed files with 5026 additions and 336 deletions
Expand all files Collapse all files
.github/workflows/release.yml
+19 -76
View File
@@ -1,5 +1,12 @@
name: Release image name: Release image
# Builds and pushes the multi-arch image to GHCR, the registry the installer
# defaults to.
#
# The Gitea registry is on a VPN, which a GitHub runner can never reach, so CI
# does not touch it at all: `scripts/mirror-gitea.sh` copies a release into it
# from the VPN side, in parallel and after the fact. Keeping the two apart is
# what makes a registry outage unable to fail a release.
on: on:
release: release:
types: [published] types: [published]
@@ -30,13 +37,6 @@ jobs:
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Log in to Gitea
uses: docker/login-action@v3
with:
registry: git.tben.me
username: ${{ secrets.GITEA_USERNAME }}
password: ${{ secrets.GITEA_TOKEN }}
# The version the app reports on Settings > General when the build # The version the app reports on Settings > General when the build
# isn't a published release. # isn't a published release.
- name: Read the source version - name: Read the source version
@@ -45,6 +45,11 @@ jobs:
set -euo pipefail set -euo pipefail
echo "version=$(sed -n -E 's/^[[:space:]]*"version": "([^"]+)",?$/\1/p' package.json | head -n1)" >> "$GITHUB_OUTPUT" echo "version=$(sed -n -E 's/^[[:space:]]*"version": "([^"]+)",?$/\1/p' package.json | head -n1)" >> "$GITHUB_OUTPUT"
# The same four tags for every kind of build: a release gets its own
# version plus the moving `latest` and `major.minor`, so that
# scripts/mirror-gitea.sh can copy whichever of them still belong to
# it, and a dispatch build gets the commit tag only, since it is not a
# release and must not move `latest`.
- name: Work out image tags - name: Work out image tags
id: tags id: tags
env: env:
@@ -57,32 +62,24 @@ jobs:
if [ "$EVENT" = "release" ] && [ -n "$RELEASE_TAG" ]; then if [ "$EVENT" = "release" ] && [ -n "$RELEASE_TAG" ]; then
version="$RELEASE_TAG" version="$RELEASE_TAG"
minor="$(printf '%s' "$version" | sed -E 's/^v?([0-9]+)\.([0-9]+).*/\1.\2/')" minor="$(printf '%s' "$version" | sed -E 's/^v?([0-9]+)\.([0-9]+).*/\1.\2/')"
ghcr="$(printf '%s\n' \ tags="$(printf '%s\n' \
"ghcr.io/moduletrace/cobble:latest" \ "ghcr.io/moduletrace/cobble:latest" \
"ghcr.io/moduletrace/cobble:${version}" \ "ghcr.io/moduletrace/cobble:${version}" \
"ghcr.io/moduletrace/cobble:${minor}" \ "ghcr.io/moduletrace/cobble:${minor}" \
"ghcr.io/moduletrace/cobble:${sha}")" "ghcr.io/moduletrace/cobble:${sha}")"
gitea="$(printf '%s\n' \
"git.tben.me/moduletrace/cobble:latest" \
"git.tben.me/moduletrace/cobble:${version}" \
"git.tben.me/moduletrace/cobble:${minor}" \
"git.tben.me/moduletrace/cobble:${sha}")"
else else
version="$SOURCE_VERSION" version="$SOURCE_VERSION"
ghcr="ghcr.io/moduletrace/cobble:${sha}" tags="ghcr.io/moduletrace/cobble:${sha}"
gitea="git.tben.me/moduletrace/cobble:${sha}"
fi fi
{ {
echo "version=$version" echo "version=$version"
echo "ghcr<<EOF" echo "tags<<EOF"
echo "$ghcr" echo "$tags"
echo "EOF"
echo "gitea<<EOF"
echo "$gitea"
echo "EOF" echo "EOF"
} >> "$GITHUB_OUTPUT" } >> "$GITHUB_OUTPUT"
# The registry the installer defaults to, so this one must succeed. # The only push CI makes. The installer defaults to this registry, so
# this step failing has to fail the release.
- name: Build and push to GHCR - name: Build and push to GHCR
uses: docker/build-push-action@v6 uses: docker/build-push-action@v6
with: with:
@@ -92,62 +89,8 @@ jobs:
push: true push: true
build-args: | build-args: |
COBBLE_VERSION=${{ steps.tags.outputs.version }} COBBLE_VERSION=${{ steps.tags.outputs.version }}
tags: ${{ steps.tags.outputs.ghcr }} tags: ${{ steps.tags.outputs.tags }}
provenance: false provenance: false
sbom: false sbom: false
cache-from: type=gha cache-from: type=gha
cache-to: type=gha,mode=max cache-to: type=gha,mode=max
# The mirror's registry, from the image just built: no second build,
# and Cloudflare rate limiting git.tben.me (429/1015 on blob uploads)
# can never fail a release the installer depends on. It must still be
# loud: `continue-on-error` alone leaves the check green when the
# mirror does nothing at all, which is how the 413 below went unseen
# for four releases.
- name: Mirror to Gitea
id: mirror
continue-on-error: true
env:
GITEA_TAGS: ${{ steps.tags.outputs.gitea }}
GHCR_IMAGES: ${{ steps.tags.outputs.ghcr }}
run: |
set -uo pipefail
first_ghcr="$(printf '%s\n' "$GHCR_IMAGES" | head -n1)"
args=()
while IFS= read -r tag; do
[ -n "$tag" ] || continue
args+=(--tag "${tag}")
done <<< "$GITEA_TAGS"
output="$(mktemp)"
status=0
docker buildx imagetools create "${args[@]}" "${first_ghcr}" 2>&1 | tee "$output" || status=$?
if [ "$status" -eq 0 ]; then
{
echo "### Gitea mirror"
echo
echo "Mirrored \`${first_ghcr}\` to:"
echo
printf -- '- \`%s\`\n' $GITEA_TAGS
} >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
# The usual cause: git.tben.me sits behind Cloudflare, whose
# request-body cap is 100 MB, and this image has a 243 MiB
# `bun install` layer that no route through it can carry.
reason="$(grep -oE '[0-9]{3} [A-Za-z ]{0,40}' "$output" | tail -n1)"
{
echo "### Gitea mirror failed"
echo
echo "\`${first_ghcr}\` did **not** reach the Gitea registry. The release itself is fine: the installer defaults to GHCR."
echo
echo '```'
tail -n 12 "$output"
echo '```'
echo
echo "See the [registry push limits](https://developers.cloudflare.com/workers/platform/limits/): git.tben.me is behind Cloudflare, which refuses any request body over 100 MB, and this image's \`bun install\` layer alone is about 243 MiB."
} >> "$GITHUB_STEP_SUMMARY"
echo "::warning title=Gitea mirror failed::${reason:-docker buildx imagetools create failed}; the image did not reach git.tben.me. See the job summary."
exit "$status"
AGENTS.md
+20 -6
View File
@@ -134,7 +134,7 @@ The app serves plain HTTP only; TLS is the reverse proxy's or tunnel's job, and
The container runs with `--network cobble`, `-p BIND:PORT`, `.env` at `/app/.env` read only, the host data directory at `/data`, the docker socket at `/var/run/docker.sock`, `MAIN_SERVER=1` and `PORT`, `--init`, `--cpus 1` and `--memory 2048m` (a light orchestration process: the heavy work happens in each computer's own container; overridable with `--cpus`/`--memory`), a `unless-stopped` restart policy and bounded json-file logs. `/data` must stay a bind mount: `resolveHostPath` turns paths under the app's own mounts into host paths for computer containers, and a named volume would break them. `DATA_DIR=/data` and `DB_DIR=/data/db` come from the Dockerfile; `entrypoint.sh` keeps bun-sqlite's schema record on the same volume. The container runs with `--network cobble`, `-p BIND:PORT`, `.env` at `/app/.env` read only, the host data directory at `/data`, the docker socket at `/var/run/docker.sock`, `MAIN_SERVER=1` and `PORT`, `--init`, `--cpus 1` and `--memory 2048m` (a light orchestration process: the heavy work happens in each computer's own container; overridable with `--cpus`/`--memory`), a `unless-stopped` restart policy and bounded json-file logs. `/data` must stay a bind mount: `resolveHostPath` turns paths under the app's own mounts into host paths for computer containers, and a named volume would break them. `DATA_DIR=/data` and `DB_DIR=/data/db` come from the Dockerfile; `entrypoint.sh` keeps bun-sqlite's schema record on the same volume.
Publishing is `scripts/publish-github.sh` and `scripts/publish-gitea.sh`: the shared flow (argument parsing, the release gate, tag preparation, the push, the release notes) lives in `scripts/lib/publish-common.sh` and `scripts/lib/publish-release.sh`, and only the release object differs (`gh release create` for GitHub, which is what fires the image build, and `curl` against the Gitea API with a `GITEA_TOKEN` or the token `tea login` stored). Both are marked-block scripts tested by `scripts/publish-tools.test.ts`, which lifts the blocks out with the `install.test.ts` approach. Images are built by GitHub Actions on a published release (`.github/workflows/release.yml`), for `linux/amd64` and `linux/arm64`, and pushed to both `ghcr.io/moduletrace/cobble` and `git.tben.me/moduletrace/cobble` with `latest`, version, `major.minor` and commit-sha tags. The version they are built with is passed as the `COBBLE_VERSION` build arg (the published tag, else `package.json`'s version read from `<version>`), which the image turns into `ENV COBBLE_VERSION` and `runningVersion` (`src/utils/running-version.ts`) reports on Settings > General; bump `package.json`'s `version` with a release, since a local build and a workflow-dispatch build fall back to it. `.github/workflows/ci.yml` runs `bun run typecheck` and `bun run test` on every push and pull request, and checks that `install.sh` parses. The Dockerfile pins Bun 1.4.3 to match CI. The repository is mirrored on Gitea, and the installer defaults to the GHCR image with `--image git.tben.me/moduletrace/cobble` as the Gitea alternative. Publishing is `scripts/publish-github.sh` and `scripts/publish-gitea.sh`: the shared flow (argument parsing, the release gate, tag preparation, the push, the release notes) lives in `scripts/lib/publish-common.sh` and `scripts/lib/publish-release.sh`, and only the release object differs (`gh release create` for GitHub, which is what fires the image build, and `curl` against the Gitea API with a `GITEA_TOKEN` or the token `tea login` stored). Both are marked-block scripts tested by `scripts/publish-tools.test.ts`, which lifts the blocks out with the `install.test.ts` approach. Images are built by GitHub Actions on a published release (`.github/workflows/release.yml`), for `linux/amd64` and `linux/arm64`, and pushed to `ghcr.io/moduletrace/cobble` with `latest`, version, `major.minor` and commit-sha tags. CI never touches the Gitea registry: it is on a VPN, which a runner cannot reach, so `scripts/mirror-gitea.sh` (`bun run mirror:gitea`) copies a release into `git.tben.me/moduletrace/cobble` afterwards, from the Gitea host over ssh, running skopeo there in a container and reading every destination tag's digest back against GHCR's; it only moves `latest` and `major.minor` while that release still owns them in GHCR, so mirroring an older one never rolls them back. The two run in parallel and neither can fail the other, which is the point: the installer defaults to GHCR, so a registry outage cannot fail a release. The version they are built with is passed as the `COBBLE_VERSION` build arg (the published tag, else `package.json`'s version read from `<version>`), which the image turns into `ENV COBBLE_VERSION` and `runningVersion` (`src/utils/running-version.ts`) reports on Settings > General; bump `package.json`'s `version` with a release, since a local build and a workflow-dispatch build fall back to it. `.github/workflows/ci.yml` runs `bun run typecheck` and `bun run test` on every push and pull request, and checks that `install.sh` parses. The Dockerfile pins Bun 1.4.3 to match CI. The repository is mirrored on Gitea, and the installer defaults to the GHCR image with `--image git.tben.me/moduletrace/cobble` as the Gitea alternative.
### Envs ### Envs
@@ -194,30 +194,43 @@ Every folder on the data volume, browsable read-only: a bot's own, a workspace's
- **Providers** (`HarnessProviderKinds` in `src/dict/harness-provider-dict.ts`): OpenRouter (default, created by `ensureDefaultHarnessProvider`), OpenAI, Anthropic, Gemini, DeepSeek, Groq, Mistral, xAI, Ollama (local, its models count as free), any OpenAI-compatible server. Rows in `harness_providers` (a private table: the key is encrypted with `src/utils/crypt.ts`, never sent to the client; falls back to the preset's env, e.g. `OPENROUTER_API_KEY`, via `grabEnv`). One adapter today, `src/harness/providers/openai-compatible/` (`chat`, `transcribe`, `speak`, `generateImage`, `embed` (`POST /embeddings`, batched), `listModels`), chosen in `grab-harness-adapter.ts`; a provider needing a native API gets its own adapter there (`HarnessAdapter` in `src/harness/types.ts`). - **Providers** (`HarnessProviderKinds` in `src/dict/harness-provider-dict.ts`): OpenRouter (default, created by `ensureDefaultHarnessProvider`), OpenAI, Anthropic, Gemini, DeepSeek, Groq, Mistral, xAI, Ollama (local, its models count as free), any OpenAI-compatible server. Rows in `harness_providers` (a private table: the key is encrypted with `src/utils/crypt.ts`, never sent to the client; falls back to the preset's env, e.g. `OPENROUTER_API_KEY`, via `grabEnv`). One adapter today, `src/harness/providers/openai-compatible/` (`chat`, `transcribe`, `speak`, `generateImage`, `embed` (`POST /embeddings`, batched), `listModels`), chosen in `grab-harness-adapter.ts`; a provider needing a native API gets its own adapter there (`HarnessAdapter` in `src/harness/types.ts`).
- **Catalog:** `harness_models` per provider (modalities, tools support, voices, Artificial Analysis scores, `free`, `available`). The `sync-harness-models` cron (hourly, on start) refreshes every enabled provider (`syncAllHarnessModels`); unlisted models are kept but marked unavailable. OpenRouter lists every modality with capabilities; other providers list ids, so capabilities are guessed (`guess-model-modalities`). Free on OpenRouter means `:free` or `openrouter/free`, never zero token prices. - **Catalog:** `harness_models` per provider (modalities, tools support, voices, Artificial Analysis scores, `free`, `available`). The `sync-harness-models` cron (hourly, on start) refreshes every enabled provider (`syncAllHarnessModels`); unlisted models are kept but marked unavailable. OpenRouter lists every modality with capabilities; other providers list ids, so capabilities are guessed (`guess-model-modalities`). Free on OpenRouter means `:free` or `openrouter/free`, never zero token prices.
- **Automatic picks** (`rank-auto-models`): only for the `automatic` harnesses in `Harnesses` (text, vision, image); transcription and speech have none and are not available until a model is picked (the model select reads "Not available", no fallback checkbox). Free models that can do the job (on the setting's provider when it has one; a provider without free models needs a picked model) (`model-serves-harness`: text needs tools), by vendor `AutoModelVendorPreference` (DeepSeek, Qwen, NVIDIA, Liquid), then other vendors that look competent (`AppData.HarnessAutoMin*`), then routers; within a vendor by intelligence index, then full-size before small, newest first. - **Automatic picks** (`rank-auto-models`): only for the `automatic` harnesses in `Harnesses` (text, vision, image); transcription and speech have none and are not available until a model is picked (the model select reads "Not available", no fallback checkbox). Free models that can do the job (on the setting's provider when it has one; a provider without free models needs a picked model) (`model-serves-harness`: text needs tools), by vendor `AutoModelVendorPreference` (DeepSeek, Qwen, NVIDIA, Liquid), then other vendors that look competent (`AppData.HarnessAutoMin*`), then routers; within a vendor by intelligence index, then full-size before small, newest first.
- **Calling:** `resolveHarnessCandidates` (pinned, then automatic) -> `withCandidates` tries them in order with process-wide cooldowns (`resolve/harness-cooldowns.ts`): rate limits and outages rest the model (until `Retry-After` when given), 401/402 rest the whole provider, 403/404 rest the model for an hour. Errors are `HarnessError` (`code` provider|unavailable|aborted, `retryable`), classified in `to-harness-error.ts`. Entry points: `text/generate-text.ts` (one turn), `text/run-text-loop.ts` (tool loop, sticks to one model, wraps up after `HarnessMaxToolSteps`, trims the oldest tool output before each turn to fit the model's context, `fit-transcript-to-context.ts`), `media/read-image.ts`, `media/read-media.ts` (documents locally via `fileToText`, images and audio through their harnesses), `audio/transcribe-audio.ts`, `audio/speak-text.ts` (returns the provider's audio as a stream, falling back to the next model only until one answers), `image/generate-image.ts`, `embeddings/embed-text.ts` (batches, then `fit-vector-dimensions.ts` pads or cuts each vector to `AppData.EmbeddingDimensions` and normalizes it, since a provider's width varies). A text model that can't take an image or audio part gets it read by the vision/transcription harness first (`prepare-messages-for-model.ts`). - **Calling:** `resolveHarnessCandidates` (pinned, then automatic) -> `withCandidates` tries them in order with process-wide cooldowns (`resolve/harness-cooldowns.ts`): rate limits and outages rest the model (until `Retry-After` when given), 401/402 rest the whole provider, 403/404 rest the model for an hour. Errors are `HarnessError` (`code` provider|unavailable|aborted, `retryable`), classified in `to-harness-error.ts`. Entry points: `text/generate-text.ts` (one turn), `text/run-text-loop.ts` (tool loop, sticks to one model, wraps up after `max_steps`, `AppData.HarnessMaxToolSteps` by default and the install's own saved limit for a task run, `grabHarnessMaxToolSteps` in `app-settings/grab-run-limits.ts` reading the `harness_max_tool_steps` app setting, and `grabReplyMaxToolSteps`/`bot_reply_max_tool_steps` for a chat reply in place of `AppData.BotReplyMaxToolSteps`; `src/utils/read-step-limit.ts` falls back to the code default for anything missing, below 1 or past `AppData.BotStepLimitMax`; trims the oldest tool output before each turn to fit the model's context, `fit-transcript-to-context.ts`), `media/read-image.ts`, `media/read-media.ts` (documents locally via `fileToText`, images and audio through their harnesses), `audio/transcribe-audio.ts`, `audio/speak-text.ts` (returns the provider's audio as a stream, falling back to the next model only until one answers), `image/generate-image.ts`, `embeddings/embed-text.ts` (batches, then `fit-vector-dimensions.ts` pads or cuts each vector to `AppData.EmbeddingDimensions` and normalizes it, since a provider's width varies). A text model that can't take an image or audio part gets it read by the vision/transcription harness first (`prepare-messages-for-model.ts`).
- **Per-bot models:** in a workspace, its `workspace_harness_settings` row wins (`source: "workspace"`). Otherwise a worker bot inherits the defaults per harness unless it has a `bot_harness_settings` row (same fields as `harness_settings`, unique per bot and harness). `grabHarnessSetting({ harness, bot_id })` returns the bot's row (`source: "bot"`) or the default; the coordinator always gets the default. Every entry point takes `bot_id` and passes it on (text loop -> media reading/transcription), so a bot's run uses its own models throughout. Saved by `saveBotHarnessSetting` (`inherit` deletes the row); the page data is `grabBotHarnessOverview`. A bot's chat header has a ghost model icon (`(partials)/header-icon-button.tsx`; the text model and provider show under the bot's name) that opens a modal with every harness (`(partials)/bot-models-modal.tsx`); the modal and `/admin/bots/[bot_id]/models` both use `(partials)/bot-harness-form.tsx`. On the coordinator the modal edits the defaults themselves (`api/admin/harness/save-settings`, the same settings as Settings, under Models, without the providers): super admins only, the icon is disabled for everyone else. Every one of these wraps `settings/models/(partials)/harness-settings-form.tsx`: per harness a provider select first (`harness-provider-select.tsx`: any provider, one provider, or on a bot "Same as the default"), then the model select narrowed to it. Removing a provider (`deleteHarnessProvider`) puts default and bot settings that picked it or its models back on any provider (`save` prop; "Same as the default" shows when the overview item has `default`). - **Per-bot models:** in a workspace, its `workspace_harness_settings` row wins (`source: "workspace"`). Otherwise a worker bot inherits the defaults per harness unless it has a `bot_harness_settings` row (same fields as `harness_settings`, unique per bot and harness). `grabHarnessSetting({ harness, bot_id })` returns the bot's row (`source: "bot"`) or the default; the coordinator always gets the default. Every entry point takes `bot_id` and passes it on (text loop -> media reading/transcription), so a bot's run uses its own models throughout. Saved by `saveBotHarnessSetting` (`inherit` deletes the row); the page data is `grabBotHarnessOverview`. A bot's chat header has a ghost model icon (`(partials)/header-icon-button.tsx`; the text model and provider show under the bot's name) that opens a modal with every harness (`(partials)/bot-models-modal.tsx`); the modal and `/admin/bots/[bot_id]/models` both use `(partials)/bot-harness-form.tsx`. On the coordinator the modal edits the defaults themselves (`api/admin/harness/save-settings`, the same settings as Settings, under Models, without the providers): super admins only, the icon is disabled for everyone else. Every one of these wraps `settings/models/(partials)/harness-settings-form.tsx`: per harness a provider select first (`harness-provider-select.tsx`: any provider, one provider, or on a bot "Same as the default"), then the model select narrowed to it. Removing a provider (`deleteHarnessProvider`) puts default and bot settings that picked it or its models back on any provider (`save` prop; "Same as the default" shows when the overview item has `default`).
- **Tools:** `HarnessTool` (built per run with its bot bound): own tools validate with a zod `schema`; tools described elsewhere pass a JSON schema as `parameters` and validate themselves. `run-tool-call.ts` reports bad calls and failures back to the model as `Error: ...`, and caps every tool call at `AppData.HarnessToolTimeoutMs` (aborted, its result handed back as an error the model can work around). Bot tools in `src/harness/tools/storage/`: `list_files`, `read_file`, `write_file`, `search_files`, `move_file`, `delete_file`, `ingest_file`, `copy_file`, `generate_image`, `make_pdf` (`grabBotFileTools`, every run, see Bot files and folders). Bot tools in `src/harness/tools/bot/`: `update_memory`, `update_instructions` (`grabBotTools`, every run), `start_task` and, for the coordinator with workers, `delegate_task` (`grabBotChatTools`, chat replies only; the job's `requested_by_bot_id` is the coordinator, and `relayJobToRequester` posts the worker's result or failure in its conversation, pushed as `relay` by `pushBotNotice`), `ask_user_to_take_over` (see Take-over), plus for the coordinator `create_workspace` and `update_workspace` (a new workspace with its shared context, or changing one; both also in its tasks, so a task that gathers material can write it into the workspace), `create_bot` (add a worker with its name, role and instructions, and join it to a workspace straight away) and, in its own chat, `add_bot_to_workspace` (put an existing worker in one of the user's workspaces) and `remove_bot_from_workspace` (take it out, keeping its conversation there), `assign_computer` (grant or revoke a worker's direct access, `add/remove-bot-to/from-computer`; access through a workspace stays and the result says so), `update_bot` (change a worker's name, role or core instructions, `src/functions/backend/db/bots/update-bot.ts`), `find_bot` and `find_workspace` (look the crew, or the live workspaces, up by id, name, role/description or instructions/context), `delete_bot` (a worker with its conversations, tasks, schedules and workspace places; the coordinator can't go) and `delete_workspace` (its conversations, tasks, schedules and links; its bots and computers stay). `search_computers` (`src/harness/tools/computer/search-computers-tool.ts`, chat replies and tasks) runs one read-only command on all of a bot's computers at once over the sandbox REST API (`run-computer-command.ts`), each output under its `[computer:ID]`, secrets redacted, capped by `AppData.ComputerSearch*`. - **Tools:** `HarnessTool` (built per run with its bot bound): own tools validate with a zod `schema`; tools described elsewhere pass a JSON schema as `parameters` and validate themselves. `run-tool-call.ts` reports bad calls and failures back to the model as `Error: ...`, and caps every tool call at `AppData.HarnessToolTimeoutMs` (aborted, its result handed back as an error the model can work around). Bot tools in `src/harness/tools/storage/`: `list_files`, `read_file`, `write_file`, `search_files`, `move_file`, `delete_file`, `ingest_file`, `copy_file`, `generate_image`, `make_pdf` (`grabBotFileTools`, every run, see Bot files and folders). Bot tools in `src/harness/tools/bot/`: `update_memory`, `update_instructions` (`grabBotTools`, every run), `start_task` and, for the coordinator with workers, `delegate_task` (`grabBotChatTools`, chat replies only; a task handed over in the coordinator's own chat runs in the worker's own chat unless it names a `workspace_id`, which puts it in that workspace: the worker's workspace instructions and memory, the workspace's context, files and computers apply, and it needs membership, added first via `addBotToWorkspace` when missing; handed over from inside a workspace it always runs there and a different `workspace_id` is refused; the job's `requested_by_bot_id` is the coordinator, and `relayJobToRequester` posts the worker's result or failure in its conversation, pushed as `relay` by `pushBotNotice`), `ask_user_to_take_over` (see Take-over), plus for the coordinator `create_workspace` and `update_workspace` (a new workspace with its shared context, or changing one; both also in its tasks, so a task that gathers material can write it into the workspace), `create_bot` (add a worker with its name, role and instructions, and join it to a workspace straight away) and, in its own chat, `add_bot_to_workspace` (put an existing worker in one of the user's workspaces) and `remove_bot_from_workspace` (take it out, keeping its conversation there), `assign_computer` (grant or revoke a worker's direct access, `add/remove-bot-to/from-computer`; access through a workspace stays and the result says so), `update_bot` (change a worker's name, role or core instructions, `src/functions/backend/db/bots/update-bot.ts`), `find_bot` and `find_workspace` (look the crew, or the live workspaces, up by id, name, role/description or instructions/context), `delete_bot` (a worker with its conversations, tasks, schedules and workspace places; the coordinator can't go) and `delete_workspace` (its conversations, tasks, schedules and links; its bots and computers stay). `search_computers` (`src/harness/tools/computer/search-computers-tool.ts`, chat replies and tasks) runs one read-only command on all of a bot's computers at once over the sandbox REST API (`run-computer-command.ts`), each output under its `[computer:ID]`, secrets redacted, capped by `AppData.ComputerSearch*`.
- **MCP:** `src/harness/mcp/`: `connectMcpServer` (Streamable HTTP, SSE fallback), `grabMcpTools` (each server tool becomes a `HarnessTool` with its JSON schema), `mcpResultToText` (text as is, images read by the vision harness, `isError` thrown). Computers are reached this way: `openComputerTools` (`src/harness/tools/computer/`) connects to the computer's AIO sandbox at `grabComputerMcpUrl` (the `MAIN_COMPUTER_URL` app setting for the main computer, for a dev server outside docker, e.g. `http://localhost:8170`, else `http://<hostname>:<port>/mcp`, or its IP address when the hostname can't be reached), leaves out `ComputerMcpExcludedTools`, and sets the computer's envs (see Envs). - **MCP:** `src/harness/mcp/`: `connectMcpServer` (Streamable HTTP, SSE fallback), `grabMcpTools` (each server tool becomes a `HarnessTool` with its JSON schema), `mcpResultToText` (text as is, images read by the vision harness, `isError` thrown). Computers are reached this way: `openComputerTools` (`src/harness/tools/computer/`) connects to the computer's AIO sandbox at `grabComputerMcpUrl` (the `MAIN_COMPUTER_URL` app setting for the main computer, for a dev server outside docker, e.g. `http://localhost:8170`, else `http://<hostname>:<port>/mcp`, or its IP address when the hostname can't be reached), leaves out `ComputerMcpExcludedTools`, and sets the computer's envs (see Envs).
- **MCPs and custom tools (extensions):** installs add their own MCP servers (`mcp_servers`, Settings, under MCPs) and tools (`custom_tools`, Settings, under Tools). A bot gets the ones given to it (`bot_mcp_servers`, `bot_custom_tools`) and, in a workspace, that workspace's (`workspace_mcp_servers`, `workspace_custom_tools`); `botExtensionsSql` (`src/utils/bot-extensions-sql.ts`, tested) resolves a run's, `grabBotExtensions` reads them. `all_bots` ("every bot by default") only decides what bots are given: turning it on gives it to every bot (`attachToAllBots`, the coordinator included) and every new bot gets it (`attachDefaultExtensions`, from `createBot` and `ensureCoordinatorBot`); turning it off takes it from no one, and a bot that drops one keeps it dropped (`src/scripts/attach-every-bot-extensions.ts`, safe to rerun, moved installs from the old "reaches every bot" meaning). Super admins define them. Super admins and admins (the super admin alone for the coordinator, `assertBotExtensionManager`) give and take them per bot on the bot's settings page and under its chat's composer (`composer-tools.tsx`: one "N tools" button (`composer-tools-modal.tsx`) counting everything it can use there, which opens a modal listing them (a workspace's not removable) with the add picker, so ten of them don't fill the composer; beside it `composer-computer-button.tsx` opens the bot's current computer in the viewer), and per workspace on the workspace's settings page; the settings pages share `settings/(partials)/extensions-group.tsx`, all through `useExtensions` (`src/hooks/`), `grabBotExtensionsOverview` / `grabWorkspaceExtensionsOverview` (`bot_extensions` / `workspace_extensions` page props, both on `grabExtensionsOverview`) and `api/admin/bots/` and `api/admin/workspaces/` `add-extension`, `remove-extension` (`setBotExtension`, `setWorkspaceExtension`); the MCPs and Tools pages' Access modal sets bots and workspaces in bulk (`setExtensionAccess`, an admin never touches the coordinator's or `all_bots`). Helpers in `src/functions/backend/db/extensions/`, API in `api/admin/mcps/` and `api/admin/tools/`, zod in `src/schemas/extension-schema.ts`. - **MCPs and custom tools (extensions):** installs add their own MCP servers (`mcp_servers`, Settings, under MCPs) and tools (`custom_tools`, Settings, under Tools). A bot gets the ones given to it (`bot_mcp_servers`, `bot_custom_tools`) and, in a workspace, that workspace's (`workspace_mcp_servers`, `workspace_custom_tools`); `botExtensionsSql` (`src/utils/bot-extensions-sql.ts`, tested) resolves a run's, `grabBotExtensions` reads them. `all_bots` ("every bot by default") only decides what bots are given: turning it on gives it to every bot (`attachToAllBots`, the coordinator included) and every new bot gets it (`attachDefaultExtensions`, from `createBot` and `ensureCoordinatorBot`); turning it off takes it from no one, and a bot that drops one keeps it dropped (`src/scripts/attach-every-bot-extensions.ts`, safe to rerun, moved installs from the old "reaches every bot" meaning). Super admins define them. Super admins and admins (the super admin alone for the coordinator, `assertBotExtensionManager`) give and take them per bot on the bot's settings page and under its chat's composer (`composer-tools.tsx`: one "N tools" button (`composer-tools-modal.tsx`) counting everything it can use there, which opens a modal listing them (a workspace's not removable) with the add picker, so ten of them don't fill the composer; beside it `composer-computer-button.tsx` opens the bot's current computer in the viewer), and per workspace on the workspace's settings page; the settings pages share `settings/(partials)/extensions-group.tsx`, all through `useExtensions` (`src/hooks/`), `grabBotExtensionsOverview` / `grabWorkspaceExtensionsOverview` (`bot_extensions` / `workspace_extensions` page props, both on `grabExtensionsOverview`) and `api/admin/bots/` and `api/admin/workspaces/` `add-extension`, `remove-extension` (`setBotExtension`, `setWorkspaceExtension`); the MCPs and Tools pages' Access modal sets bots and workspaces in bulk (`setExtensionAccess`, an admin never touches the coordinator's or `all_bots`). Helpers in `src/functions/backend/db/extensions/`, API in `api/admin/mcps/` and `api/admin/tools/`, zod in `src/schemas/extension-schema.ts`.
- **Skills** are the third extension kind (`skills`, Settings, under Skills; `bot_skills` / `workspace_skills`, same `all_bots` rules, `ExtensionTables.kind = "skill"`). A skill is a runbook: `name`, a unique `slug` a bot loads it by, a one-line `description` (when it applies) and the markdown `body`, capped by `AppData.SkillBodyMaxChars`. Unlike an MCP server or a tool it has no `status`/`last_error`: nothing connects or runs, and it never reports a failure (`ExtensionRow` and `ComposerToolRow` key the state line off `kind == "skill"`). A run sees only the slug and description; `openBotSkills` (`src/harness/tools/skills/`, fed by the `skills` that `grabBotExtensions` already reads) adds one `load_skill` tool that returns the body, cut at `AppData.SkillBodyShownChars` with a line saying so, and throws the valid slugs for an unknown one. Slugs are checked by `checkSkillSlug` (`src/utils/check-skill-slug.ts`, tested); `load_skill` is in `ReservedToolNames` so a custom tool can't shadow it. The settings page is `api/admin/skills/` (`save-skill`, `delete-skill` super admin only, `set-skill-access`), `grabSkillsOverview`; `deleteSkill` deletes the two link tables itself, since foreign keys are off outside the server process. `openBotExtensions` (`src/harness/tools/extensions/`) turns them into harness tools for chat replies and tasks (built-ins win a name clash; `close()` in the runners' `finally`; `note` tells the bot about a server it can't use). Templates (`fillTemplate`, `src/utils/fill-template.ts`): `{{args.x}}` and `{{env.KEY}}`, envs from `grabRunEnvs` (the bot's own beat the workspace's, which beat general; app settings never, no need to be set on computers); secret values are redacted from every result. - **Skills** are the third extension kind (`skills`, Settings, under Skills; `bot_skills` / `workspace_skills`, same `all_bots` rules, `ExtensionTables.kind = "skill"`). A skill is a runbook: `name`, a unique `slug` a bot loads it by, a one-line `description` (when it applies) and the markdown `body`, capped by `AppData.SkillBodyMaxChars`. Unlike an MCP server or a tool it has no `status`/`last_error`: nothing connects or runs, and it never reports a failure (`ExtensionRow` and `ComposerToolRow` key the state line off `kind == "skill"`). A run sees only the slug and description; `openBotSkills` (`src/harness/tools/skills/`, fed by the `skills` that `grabBotExtensions` already reads) adds one `load_skill` tool that returns the body, cut at `AppData.SkillBodyShownChars` with a line saying so, and throws the valid slugs for an unknown one. Slugs are checked by `checkSkillSlug` (`src/utils/check-skill-slug.ts`, tested); `load_skill` is in `ReservedToolNames` so a custom tool can't shadow it. The settings page is `api/admin/skills/` (`save-skill`, `delete-skill` super admin only, `set-skill-access`), `grabSkillsOverview`; `deleteSkill` deletes the two link tables itself, since foreign keys are off outside the server process. `openBotExtensions` (`src/harness/tools/extensions/`) turns them into harness tools for chat replies and tasks (built-ins win a name clash; `close()` in the runners' `finally`; `note` tells the bot about a server it can't use). Templates (`fillTemplate`, `src/utils/fill-template.ts`): `{{args.x}}` and `{{env.KEY}}`, envs from `grabRunEnvs` (the bot's own beat the workspace's, which beat general; app settings never, no need to be set on computers); secret values are redacted from every result.
- **Custom tools** (`CustomToolKinds` in `src/dict/custom-tool-dict.ts`): a name (`checkCustomToolName`: snake_case, never a built-in's, `ReservedToolNames`), a description, a JSON-schema `parameters` and a `config` per kind, built by `buildCustomTool` (`src/harness/tools/custom/`): `http` (`runHttpTool`: the app fetches, arguments URL-encoded in the URL, no body template sends the arguments as JSON), `script` (`runScriptTool`: bash/python/javascript on the run's computer, the task's or in chat the bot's current one, over the sandbox REST API in a subshell, computer envs set, arguments as JSON in `COBBLE_ARGS` and on stdin; left out without a computer), `prompt` (`runPromptTool`: one `generateText` turn with the bot's models). Super admins try one from the page (`testCustomTool`, the main computer for scripts): a saved one from its row (Try), or the form as it is, saved or not (`draft`, nothing recorded), from the Test panel in its modal (`custom-tool-test-panel.tsx`). Every run (a bot's call or a try; not bad arguments or a cancel) is saved as the tool's `status`, `last_error` and `last_checked_at` (`recordCustomToolRun`); editing resets it to `untested`. - **Custom tools** (`CustomToolKinds` in `src/dict/custom-tool-dict.ts`): a name (`checkCustomToolName`: snake_case, never a built-in's, `ReservedToolNames`), a description, a JSON-schema `parameters` and a `config` per kind, built by `buildCustomTool` (`src/harness/tools/custom/`): `http` (`runHttpTool`: the app fetches, arguments URL-encoded in the URL, no body template sends the arguments as JSON), `script` (`runScriptTool`: bash/python/javascript on the run's computer, the task's or in chat the bot's current one, over the sandbox REST API in a subshell, computer envs set, arguments as JSON in `COBBLE_ARGS` and on stdin; left out without a computer), `prompt` (`runPromptTool`: one `generateText` turn with the bot's models). Super admins try one from the page (`testCustomTool`, the main computer for scripts): a saved one from its row (Try), or the form as it is, saved or not (`draft`, nothing recorded), from the Test panel in its modal (`custom-tool-test-panel.tsx`). Every run (a bot's call or a try; not bad arguments or a cancel) is saved as the tool's `status`, `last_error` and `last_checked_at` (`recordCustomToolRun`); editing resets it to `untested`.
- **MCP servers** (`McpTransports` in `src/dict/mcp-dict.ts`): `remote` (URL plus templated headers, `connectMcpServer`) or `stdio` (a command run with the host's shell, `hostShell`, in `DATA_DIR/mcp/<id>` (`grabMcpDir`), after its `setup_command`, which `runMcpSetup` reruns whenever it changes (marker file) or on Reinstall; its env is only its own `env` map plus the shell basics, `grabMcpStdioEnv`, never the app's). Their tools are listed into `tools_cache` (`syncMcpServer`: on save, Refresh, and hourly for remote ones by the `sync-mcp-servers` cron) and named `<slug>__<tool>` (`mcpToolName`); the modal's Test connection checks the form as it is, saved or not, outside the pool (`testMcpDraft`, `testMcpServer`, `api/admin/mcps/test-mcp`, a new stdio server's setup runs in `mcp/0`); a run that can't connect, or connects again after failing, updates its `status` too (`recordMcpServerConnection`); runs build tools from the cache (`mcpHarnessTool`) and connect on the first call (`openMcpServer`). stdio servers live in a per-process pool (`src/harness/mcp/mcp-stdio-pool.ts`), stopped after `AppData.McpStdioIdleMs` idle, on save or delete. - **MCP servers** (`McpTransports` in `src/dict/mcp-dict.ts`): `remote` (URL plus templated headers, `connectMcpServer`) or `stdio` (a command run with the host's shell, `hostShell`, in `DATA_DIR/mcp/<id>` (`grabMcpDir`), after its `setup_command`, which `runMcpSetup` reruns whenever it changes (marker file) or on Reinstall; its env is only its own `env` map plus the shell basics, `grabMcpStdioEnv`, never the app's). Their tools are listed into `tools_cache` (`syncMcpServer`: on save, Refresh, and hourly for remote ones by the `sync-mcp-servers` cron) and named `<slug>__<tool>` (`mcpToolName`); the modal's Test connection checks the form as it is, saved or not, outside the pool (`testMcpDraft`, `testMcpServer`, `api/admin/mcps/test-mcp`, a new stdio server's setup runs in `mcp/0`); a run that can't connect, or connects again after failing, updates its `status` too (`recordMcpServerConnection`); runs build tools from the cache (`mcpHarnessTool`) and connect on the first call (`openMcpServer`). stdio servers live in a per-process pool (`src/harness/mcp/mcp-stdio-pool.ts`), stopped after `AppData.McpStdioIdleMs` idle, on save or delete.
- **Check before and after acting:** every reply and task gets `ActionCheckNote` (`src/data/harness-prompts.ts`): look for an existing record before creating one (update it instead), and verify every action took effect before saying it's done. The tools back it: `update_memory` and `update_instructions` read the save back (`checkSavedText`, `grabBotSavedText`) and fail if it didn't take; `create_bot` refuses a name already on the crew, `create_schedule` a twin of one of the bot's schedules (`findDuplicateCronJob`: same place and same title, or same timing and brief), and `start_task`/`delegate_task` a task with the same title still open in that session (`grabOpenBotTask`); `create_workspace` already refuses a name in use. - **Check before and after acting:** every reply and task gets `ActionCheckNote` (`src/data/harness-prompts.ts`): look for an existing record before creating one (update it instead), and verify every action took effect before saying it's done. The tools back it: `update_memory` and `update_instructions` read the save back (`checkSavedText`, `grabBotSavedText`) and fail if it didn't take; `create_bot` refuses a name already on the crew, `create_schedule` a twin of one of the bot's schedules (`findDuplicateCronJob`: same place and same title, or same timing and brief), and `start_task`/`delegate_task` a task with the same title still open in that session (`grabOpenBotTask`); `create_workspace` already refuses a name in use.
- **Replies and tasks:** `run-bot-reply.ts` answers from the last `BotReplyHistoryMessages` messages (`botMessagesToHarness`) with `ChatReplyNote`, the bot's computers (`compose-computers-note.ts`: what they are and that they're reached through `start_task`) and the chat tools, at most `BotReplyMaxToolSteps` steps, and finishes silently if the bot already had the last word (`enqueueBotReply` reuses a pending reply, so a burst of messages gets one). `run-bot-task.ts` runs the brief (`compose-job-brief.ts`) with `TaskRunNote`, the bot tools, the file tools and the job's computer tools, closing the MCP connection after. Both add `BotFilesNote` (where a bot's files live, that they persist, and to search before saying it doesn't know) and collect `attachments` for files the bot wrote with `attach`, which `completeBotJob` links to the result message. Retryable harness errors go back to the queue's backoff, the rest are `BotJobFatalError`. - **Replies and tasks:** `run-bot-reply.ts` answers from the last `BotReplyHistoryMessages` messages (`botMessagesToHarness`) with `ChatReplyNote`, the bot's computers (`compose-computers-note.ts`: what they are and that they're reached through `start_task`) and the chat tools, at most `grabReplyMaxToolSteps()` steps (the `bot_reply_max_tool_steps` app setting, else `AppData.BotReplyMaxToolSteps`), and finishes silently if the bot already had the last word (`enqueueBotReply` reuses a pending reply, so a burst of messages gets one). `run-bot-task.ts` runs the brief (`compose-job-brief.ts`) with `TaskRunNote`, the bot tools, the file tools and the job's computer tools, closing the MCP connection after. Both add `BotFilesNote` (where a bot's files live, that they persist, and to search before saying it doesn't know) and collect `attachments` for files the bot wrote with `attach`, which `completeBotJob` links to the result message. Retryable harness errors go back to the queue's backoff, the rest are `BotJobFatalError`.
- **API:** `api/admin/harness/` (`save-provider` also syncs, `sync-provider`, `delete-provider`, `save-settings`), super admin only, each returning the fresh `grabHarnessOverview()` page data. - **API:** `api/admin/harness/` (`save-provider` also syncs, `sync-provider`, `delete-provider`, `save-settings`), super admin only, each returning the fresh `grabHarnessOverview()` page data.
### Token usage
Every model call is written down in `token_usages` as it happens, so a message can show what it cost and `/admin/usage` can total the install's spending by day, by model and by provider.
- **Whose call it was:** `src/harness/usage/harness-usage-scope.ts` is an `AsyncLocalStorage` scope (`user_id`, `bot_id`, `workspace_id`, `session_id`, `job_id`, `message_id`) that a run enters once, in `run-bot-job.ts` (`runWithHarnessUsageScope`), so everything underneath - the text loop's turns, a session summary, a transcription, an image, an embedding - is attributed without threading the ids through every harness function. The store is merged, not replaced: a nested run keeps the outer ids and overrides only what it knows better, and a `null` is a real value that clears a field (a bot's own chat has no workspace). `withUsageScope` (`with-usage-scope.ts`) wraps one call the app makes on a user's behalf outside any run (the chat's transcribe and speak routes), and `runWithFallbackScope` (`run-with-fallback-scope.ts`) attributes work whose run may be missing - a file the ingest queue embeds has no run, so it is charged to its owner (`grabUsageOwnerUserId`, the workspace's creator else the bot's) - while `mode: "own"` forces the given scope (the session namer, whose job is already complete and must not pay for it). Without an owner nothing is written: a row no one can be shown is not worth a row.
- **Recording:** `record-harness-usage.ts` writes one row and never throws (a model that answered must not lose its answer to a failed insert; it logs with `logError` instead). `build-usage-call.ts` turns the candidate that really answered into the row's provider and model (`provider_id`/`provider_name` and the catalog `model_id` plus the provider's `model_slug`), and providers that report no usage still get a row, so a call is never invisible. Every entry point calls it in `withCandidates`'s `on_success`: `generate-text.ts`, `read-image.ts`, `transcribe-audio.ts`, `speak-text.ts` (billed per character, so the text's length stands in for what was spoken), `generate-image.ts`, `embed-text.ts`.
- **Messages:** a reply or task job's calls carry the job's id as they happen, since its message doesn't exist yet; `complete-bot-job.ts` then runs `UPDATE token_usages SET message_id = ? WHERE job_id = ? AND message_id IS NULL` once the message is inserted, so the thread can show what the answer cost. Calls that produce no message (naming a session, embedding a file, transcribing a dictation) keep a null `message_id` and are still counted in the totals.
- **Message usage:** `grab-message-usage.ts` sums a batch of messages' calls (one `GROUP BY message_id`) and `grab-bot-messages.ts` attaches it as `BotMessageWithAttachments.usage` (`TokenUsageSummary`: prompt, completion, total, cost, calls). `formatTokens` (`src/utils/format-tokens.ts`) writes a count as a person reads it (842 / 1.2K / 842K / 1.2M). `MessageUsageChip` (`bots/[bot_id]/(partials)/message-usage-chip.tsx`) sits in the actions row under a bot message beside its timestamp and play button, showing the total with the breakdown in its `title`, and renders nothing when a message has no calls.
- **Usage page:** `/admin/usage`, a `RailPages` entry like Routines and Activity rather than a settings sub page, since it is the install's own record rather than a setting (super admins and admins only, `canManageCrew`; the dict entry's `can_view` hides it for anyone else, and `grab-usage-server-props.ts` redirects them to the bots). Helpers in `src/functions/backend/db/usage/`: `grab-usage-overview.ts` runs `grab-usage-totals.ts` (one total and the same split per harness), `grab-usage-days.ts` -> `bucket-usage-days.ts` (one bucket per day, empty days at zero), `grab-usage-models.ts` (per provider, model slug and harness, busiest first, joined to `harness_models` for a display name), plus the previous period of the same length and a rolling week. The window comes from `grab-usage-window.ts` (`UsageDefaultDays` 30, `UsageMaxDays` 365, clamped). API `api/admin/usage` (GET only) for another period, zod in `src/schemas/usage-schema.ts`.
- **Days are local, not UTC:** `grabUsageDays` groups in SQL with `date(created_at / 1000, 'unixepoch', 'localtime')`, matching the local midnights `grabUsageWindow` builds and the `YYYY-MM-DD` `localDate` writes. Grouping by the bare UTC day put a late evening's calls in the previous bucket and silently dropped them whenever no bucket carried that name (the chart's total disagreed with the totals card); `bucket-usage-days.test.ts` pins this. Never name a local day with `toISOString().slice(0, 10)`.
- **The page:** `index.server.ts` -> `(functions)/grab-usage-server-props.ts` (crew managers only, the period in the address), `index.tsx` in `SettingsShell` with `wide`, `(sections)/usage-section.tsx` (state in the address through `history.pushState`, `use-usage-overview.ts` loading another period), `(partials)/usage-totals.tsx` (tokens, model calls, cost, previous period with a change label), `usage-day-chart.tsx` + `usage-day-bar.tsx` (bars from static `h-*` classes in `(data)/usage-bar-heights.ts`, since inline styles are not allowed), `usage-harness-breakdown.tsx` + `usage-harness-row.tsx`, and `usage-models-table.tsx` + `usage-model-row.tsx` (calls, in, out, tokens, **price per thousand tokens** worked out from cost and tokens, which is what makes one model's cost comparable with another's).
## Admin Route Hierarchy ## Admin Route Hierarchy
Live routes under `/admin` (see `AdminSections`): `/admin/bots`, `/admin/workspaces` (the index opens the first workspace), `/admin/routines`, `/admin/logs`, `/admin/help` and `/admin/settings`. `/admin` redirects to `/admin/bots`; a workspace's dashboard is `/admin/workspaces/<id>`. The routes in detail: Live routes under `/admin` (see `AdminSections`): `/admin/bots`, `/admin/workspaces` (the index opens the first workspace), `/admin/routines`, `/admin/logs`, `/admin/help` and `/admin/settings`. `/admin` redirects to `/admin/bots`; a workspace's dashboard is `/admin/workspaces/<id>`. The routes in detail:
- `/admin/bots` (live) - the Cobble chat shell, plus `[bot_id]/` - `/admin/bots` (live) - the Cobble chat shell, plus `[bot_id]/`
- `/admin/routines` (live) - every schedule the user's bots have (see Schedules), opened from "Routines" in the rail's user popover - `/admin/routines` (live) - every schedule the user's bots have (see Schedules), opened from "Routines" in the rail's user popover
- `/admin/usage` (live) - what the crew's model calls spent, per day, per kind of call and per model, for crew managers (see Token usage), opened from "Usage" in the rail's user popover
- `/admin/logs` (live) - Activity, the audit log: super admins only (others are redirected; the popover's "Activity" item shows to super admins only). Rendered in `SettingsShell` with `wide` - `/admin/logs` (live) - Activity, the audit log: super admins only (others are redirected; the popover's "Activity" item shows to super admins only). Rendered in `SettingsShell` with `wide`
- `/admin/help` (live) - familiar problems and their fixes, one `SettingsGroup` per topic (`(data)/help-topics.ts`, markdown rendered with the chat's `MessageMarkdownComponents`, the topic id as the group's anchor). First topic: signing in to Google and other strict sites with Cookie-Editor over the Desktop view (`HelpSignInTopicId` in `src/dict/help-dict.ts`; `ask_user_to_take_over` points bots to it) - `/admin/help` (live) - familiar problems and their fixes, one `SettingsGroup` per topic (`(data)/help-topics.ts`, markdown rendered with the chat's `MessageMarkdownComponents`, the topic id as the group's anchor). First topic: signing in to Google and other strict sites with Cookie-Editor over the Desktop view (`HelpSignInTopicId` in `src/dict/help-dict.ts`; `ask_user_to_take_over` points bots to it)
- Routines, Files, Activity and Help are the `RailPages` (`src/dict/rail-pages-dict.ts`), opened from the rail's user popover (which lists them, then Settings and Sign out). On them the rail shows its own aside (`(partials)/rail-pages-list.tsx`): a back link to the bots, then those pages, like settings - Routines, Files, Usage, Activity and Help are the `RailPages` (`src/dict/rail-pages-dict.ts`), opened from the rail's user popover (which lists them, then Settings and Sign out). On them the rail shows its own aside (`(partials)/rail-pages-list.tsx`): a back link to the bots, then those pages, like settings. A page carries `super_admin_only` or a `can_view` rule (Usage is `canManageCrew`), and `canSeeRailItem` (`src/utils/rbac/can-see-rail-item.ts`, tested) decides both the popover and the aside, so a page is never offered in one and hidden in the other
- `/admin/settings/backups` - super admin only (listed in `SettingsPages`), two groups. Database snapshots: create, list, restore, download and delete in `db/.backups` (`bun-sqlite backup`); "Restore from file" beside "Create backup" opens `upload-restore-modal.tsx` (upload a SQLite database file, which is saved to `.backups` then restored with `bun-sqlite import <file>`, since `restore` is interactive). Whole-data archives: `write-data-backup-archive.ts` tars all of `DATA_DIR` (database, media, computer homes, bot and workspace files, MCP installs) into `BACKUP_ARCHIVE_DIR` as `cobble-data-<epoch-ms>.tar.gz` (via `tar -czf` into a `.part` name, renamed on success so a cut-off run is never listed, with a `--transform` putting every member under one root folder named after the archive), keeping the newest `AppData.DataBackupsKept` and never touching `db/.backups`; its own "Restore from file" uploads a tar.gz (`upload-restore-data.ts`, gzip header checked first). `restore-data-backup.ts` extracts an archive to a temp dir, requires exactly one root folder holding a `db` folder (`findArchiveRoot`, since an upload is saved under a fresh timestamp and its inner name no longer matches the file name), takes a safety archive before touching anything (`write-data-backup-archive.ts`, not the locked `create-data-backup.ts`), empties `DATA_DIR` (the snapshots included) and copies the archive's in, then `restart-app.ts` restarts the process through the docker socket after the response lands (`process.exit`, the container's restart policy brings it back), because the open SQLite handle points at the replaced file. Row actions are icon-only (`RowIconButton`, `src/components/general/row-icon-button.tsx`): restore, download, delete. `BACKUP_ARCHIVE_DIR` is a `ProcessOnlyEnvKeys` start-up path (the installer mounts `$COBBLE_HOME/backups-archives` there; outside docker it falls back to `~/.cobble/backups`). Every backup and restore, of either kind, runs one at a time via `with-backup-run-lock.ts`. Helpers in `src/functions/backend/backups/`, API routes in `api/admin/backups/` - `/admin/settings/backups` - super admin only (listed in `SettingsPages`), two groups. Database snapshots: create, list, restore, download and delete in `db/.backups` (`bun-sqlite backup`); "Restore from file" beside "Create backup" opens `upload-restore-modal.tsx` (upload a SQLite database file, which is saved to `.backups` then restored with `bun-sqlite import <file>`, since `restore` is interactive). Whole-data archives: `write-data-backup-archive.ts` tars all of `DATA_DIR` (database, media, computer homes, bot and workspace files, MCP installs) into `BACKUP_ARCHIVE_DIR` as `cobble-data-<epoch-ms>.tar.gz` (via `tar -czf` into a `.part` name, renamed on success so a cut-off run is never listed, with a `--transform` putting every member under one root folder named after the archive), keeping the newest `AppData.DataBackupsKept` and never touching `db/.backups`; its own "Restore from file" uploads a tar.gz (`upload-restore-data.ts`, gzip header checked first). `restore-data-backup.ts` extracts an archive to a temp dir, requires exactly one root folder holding a `db` folder (`findArchiveRoot`, since an upload is saved under a fresh timestamp and its inner name no longer matches the file name), takes a safety archive before touching anything (`write-data-backup-archive.ts`, not the locked `create-data-backup.ts`), empties `DATA_DIR` (the snapshots included) and copies the archive's in, then `restart-app.ts` restarts the process through the docker socket after the response lands (`process.exit`, the container's restart policy brings it back), because the open SQLite handle points at the replaced file. Row actions are icon-only (`RowIconButton`, `src/components/general/row-icon-button.tsx`): restore, download, delete. `BACKUP_ARCHIVE_DIR` is a `ProcessOnlyEnvKeys` start-up path (the installer mounts `$COBBLE_HOME/backups-archives` there; outside docker it falls back to `~/.cobble/backups`). Every backup and restore, of either kind, runs one at a time via `with-backup-run-lock.ts`. Helpers in `src/functions/backend/backups/`, API routes in `api/admin/backups/`
- `/admin/settings/database` - super admin only (listed in `SettingsPages`), a read-only browser of the install's SQLite database, no writes anywhere in it. `grabDatabaseTables` reads the catalog once (every table and its columns, SQLite's own `sqlite_*` tables and sqlite-vec's `bot_file_vectors_*` shadow tables left out) and adds each table's `COUNT(*)` and its size from `dbstat` (index pages folded back onto their table). `grabTableRecords` then returns one page (`AppData.DatabaseBrowserPageSize`, 50) of the open table's records newest key first, every value already cut in SQL at `AppData.DatabaseBrowserCellMaxChars` (400) so a table of long texts never ships a megabyte of JSON, and a `truncated` flag per cell; the search matches over the concatenation of every column's text (`buildSearchClause`: every word, or the whole phrase), which is what also works on a virtual table, where `MATCH` and `LIKE` over a vector blob do not. A record's key is its single primary key column, else its rowid, else nothing (a virtual table with no primary key reads as plain cells); clicking a cell's row opens `grabTableRecord` for that key, which returns the record whole up to `AppData.DatabaseBrowserValueMaxChars` (20000), the answer to a cell cut at three lines. `clampPage` puts a page past the end on the last one. The page's state (table, search, mode, page) lives in the address through `databaseHref`, so a table or a search can be linked to; the open table's first page comes with the page props and anything else loads through the API. Helpers in `src/functions/backend/database/` (`build-table-catalog-sql`, `build-table-sizes-sql`, `build-records-sql`, `build-search-clause`, `resolve-table-key`, `build-record-row`, `quote-sql-identifier`), API `api/admin/database/records` and `record` (GET only, `grabSuperAdminUser`), zod in `src/schemas/database-schema.ts`, page under `src/pages/admin/settings/database/` - `/admin/settings/database` - super admin only (listed in `SettingsPages`), a read-only browser of the install's SQLite database, no writes anywhere in it. `grabDatabaseTables` reads the catalog once (every table and its columns, SQLite's own `sqlite_*` tables and sqlite-vec's `bot_file_vectors_*` shadow tables left out) and adds each table's `COUNT(*)` and its size from `dbstat` (index pages folded back onto their table). `grabTableRecords` then returns one page (`AppData.DatabaseBrowserPageSize`, 50) of the open table's records newest key first, every value already cut in SQL at `AppData.DatabaseBrowserCellMaxChars` (400) so a table of long texts never ships a megabyte of JSON, and a `truncated` flag per cell; the search matches over the concatenation of every column's text (`buildSearchClause`: every word, or the whole phrase), which is what also works on a virtual table, where `MATCH` and `LIKE` over a vector blob do not. A record's key is its single primary key column, else its rowid, else nothing (a virtual table with no primary key reads as plain cells); clicking a cell's row opens `grabTableRecord` for that key, which returns the record whole up to `AppData.DatabaseBrowserValueMaxChars` (20000), the answer to a cell cut at three lines. `clampPage` puts a page past the end on the last one. The page's state (table, search, mode, page) lives in the address through `databaseHref`, so a table or a search can be linked to; the open table's first page comes with the page props and anything else loads through the API. Helpers in `src/functions/backend/database/` (`build-table-catalog-sql`, `build-table-sizes-sql`, `build-records-sql`, `build-search-clause`, `resolve-table-key`, `build-record-row`, `quote-sql-identifier`), API `api/admin/database/records` and `record` (GET only, `grabSuperAdminUser`), zod in `src/schemas/database-schema.ts`, page under `src/pages/admin/settings/database/`
- `/admin/settings` (live) - redirects to `SettingsDefaultPath` (`/admin/settings/general`). Sub pages: `general/` (theme: System/Light/Dark per device via `useThemePreference`, stored in `localStorage[AppData.ThemeStorageKey]`, the same key `ThemeInitScript` and twui's `ColorSchemeSelector` read; applies instantly, no save bar; for super admins the prompt appendix (`prompt-section.tsx`, `system-prompt-append-form.tsx`) saving `api/admin/settings/system-prompt-append` into the `bot_system_prompt_append` app setting; and, for `canManageCrew`, the bot file quota: `storage-section.tsx`, `storage-quota-form.tsx` saving `api/admin/settings/storage-quota` into `app_settings`; and, for everyone, the running version: `version-section.tsx`, `version-line.tsx` showing `pageProps.running_version` from `runningVersion` (`src/utils/running-version.ts`: `COBBLE_VERSION`, the release tag the image was built with, else `package.json`'s version, in development too, with `(development)` after it and no such suffix in production), `profile/` (current user profile), `notifications/` (this device's Web Push switch, a test send, and every device the user subscribed), `change-password/` (set/change own password) `models/` (super admin only, `super_admin_only` in the dict hides it from the rail: model providers and which model each harness uses) and `environment/` (super admin only: app settings and the envs bots and computers use, per scope), each wrapped in `SettingsShell` (`(partials)/settings-shell.tsx`: 64px pane header like the conversation header, then one centered column). Content is built from `SettingsGroup` blocks (label + description left, controls right, hairline-separated, no cards) and ends with the sticky `SettingsSaveBar` (Save enabled only when dirty, Discard, one-line status). Saves don't reload the page. Keep save notices in component state: `useFormInit` clears `status` whenever the form changes. `bots/` (the crew: chat, edit with `BotForm`, models, delete a worker via `api/admin/bots/delete-bot`), `workspaces/` (open, create through the Add flow, delete), `computers/` (state and resources, live; super admins create, edit with `AddComputerForm`, retry, delete), `mcps/`, `tools/` and `skills/` (see MCPs, custom tools and skills: super admins add, edit, refresh or try, delete; admins only set Access, `extension-access-modal.tsx`; add, edit, access and try open in modals (`AddModalShell`). A skill has no status: nothing connects or runs, so `skill-row.tsx` puts its slug, who gets it (`extension-access-label.ts`) and its description where a server's or tool's status line goes), each MCP server or tool row with a status line under it (`extension-status-line.tsx`: connected or not, last run worked or failed, when, and why), those five for `canManageCrew` only (`grab-crew-settings-server-props.ts`, `can_view` in the dict) and `users/` (for `canListUsers`: super admins and admins; the dict's `can_view` gates it in the rail; add with a password (`canCreateUser`), edit (`canUpdateUser`), both with the one `UserForm` (`admin/settings/users/(partials)/user-form/`, `useFormInit`, photo included, saved in one request), revoke (adds `revoked`, `canRevokeUser`), delete (`canRevokeUser`'s rules, never yourself), pick their user types from `UserTypes` (`user-types-select.tsx`: only the ones `filterAssignableUserTypes` allows, never `UneditableUserTypes`; at least one, since login needs a type; `saveUser` checks each grant and removal with `userTypeAssignment`; the super admin's types are never shown or changed there, and they can't be revoked; the password is entered twice, `SaveUserRequestSchema` checks they match), delete with everything theirs; `api/admin/users/save-user`, `delete-user`). The lists share `SettingsList`, `SettingsListRow`, `SettingsRowAction` and `use-settings-list`. Sub pages are declared once in `src/dict/settings-pages-dict.ts`; on every `/admin/settings/*` page the rail swaps the crew for a back link plus that list. Add a sub page by adding its folder and a dict entry - `/admin/settings` (live) - redirects to `SettingsDefaultPath` (`/admin/settings/general`). Sub pages: `general/` (theme: System/Light/Dark per device via `useThemePreference`, stored in `localStorage[AppData.ThemeStorageKey]`, the same key `ThemeInitScript` and twui's `ColorSchemeSelector` read; applies instantly, no save bar; for super admins the prompt appendix (`prompt-section.tsx`, `system-prompt-append-form.tsx`) saving `api/admin/settings/system-prompt-append` into the `bot_system_prompt_append` app setting, and the run limits (`run-limits-section.tsx`, `run-limits-form.tsx`) saving `api/admin/settings/run-limits` into the `harness_max_tool_steps` and `bot_reply_max_tool_steps` app settings (`grabRunLimitsOverview`, `run_limits` page prop, 0 clearing a limit back to its code default); and, for `canManageCrew`, the bot file quota: `storage-section.tsx`, `storage-quota-form.tsx` saving `api/admin/settings/storage-quota` into `app_settings`; and, for everyone, the running version: `version-section.tsx`, `version-line.tsx` showing `pageProps.running_version` from `runningVersion` (`src/utils/running-version.ts`: `COBBLE_VERSION`, the release tag the image was built with, else `package.json`'s version, in development too, with `(development)` after it and no such suffix in production), `profile/` (current user profile), `notifications/` (this device's Web Push switch, a test send, and every device the user subscribed), `change-password/` (set/change own password) `models/` (super admin only, `super_admin_only` in the dict hides it from the rail: model providers and which model each harness uses) and `environment/` (super admin only: app settings and the envs bots and computers use, per scope), each wrapped in `SettingsShell` (`(partials)/settings-shell.tsx`: 64px pane header like the conversation header, then one centered column that opens with the breadcrumb trail, `src/components/general/admin-breadcrumbs.tsx`: twui's `Breadcrumbs` reading `Home`, the sections and the page off `pageProps.url.pathname`, the last crumb named by the page's own `title` so a record reads `Scout` rather than `3` and `/admin/logs` reads `Activity`; `hasBreadcrumbTrail` drops it only on the crew of bots, which is home, and nothing outside `/admin` gets one). Content is built from `SettingsGroup` blocks (label + description left, controls right, hairline-separated, no cards) and ends with the sticky `SettingsSaveBar` (Save enabled only when dirty, Discard, one-line status). Saves don't reload the page. Keep save notices in component state: `useFormInit` clears `status` whenever the form changes. `bots/` (the crew: chat, edit with `BotForm`, models, delete a worker via `api/admin/bots/delete-bot`), `workspaces/` (open, create through the Add flow, delete), `computers/` (state and resources, live; super admins create, edit with `AddComputerForm`, retry, delete), `mcps/`, `tools/` and `skills/` (see MCPs, custom tools and skills: super admins add, edit, refresh or try, delete; admins only set Access, `extension-access-modal.tsx`; add, edit, access and try open in modals (`AddModalShell`). A skill has no status: nothing connects or runs, so `skill-row.tsx` puts its slug, who gets it (`extension-access-label.ts`) and its description where a server's or tool's status line goes), each MCP server or tool row with a status line under it (`extension-status-line.tsx`: connected or not, last run worked or failed, when, and why), those five for `canManageCrew` only (`grab-crew-settings-server-props.ts`, `can_view` in the dict) and `users/` (for `canListUsers`: super admins and admins; the dict's `can_view` gates it in the rail; add with a password (`canCreateUser`), edit (`canUpdateUser`), both with the one `UserForm` (`admin/settings/users/(partials)/user-form/`, `useFormInit`, photo included, saved in one request), revoke (adds `revoked`, `canRevokeUser`), delete (`canRevokeUser`'s rules, never yourself), pick their user types from `UserTypes` (`user-types-select.tsx`: only the ones `filterAssignableUserTypes` allows, never `UneditableUserTypes`; at least one, since login needs a type; `saveUser` checks each grant and removal with `userTypeAssignment`; the super admin's types are never shown or changed there, and they can't be revoked; the password is entered twice, `SaveUserRequestSchema` checks they match), delete with everything theirs; `api/admin/users/save-user`, `delete-user`). The lists share `SettingsList`, `SettingsListRow`, `SettingsRowAction` and `use-settings-list`. Sub pages are declared once in `src/dict/settings-pages-dict.ts`; on every `/admin/settings/*` page the rail swaps the crew for a back link plus that list. Add a sub page by adding its folder and a dict entry
Every `/admin/*` page renders in the bots layout (`src/layouts/bots/`); there is no separate admin layout or aside. The rail header's Add icon (super admins and admins only, like the rail's "+" buttons) opens the Add flow (`(partials)/add-flow.tsx`, `AddFlowContext`): a bot, a workspace or (super admins) a computer, each form in a modal stacked over the choice, so closing it goes back there (the Bots and Workspaces labels have a "+" that opens the form directly; in a workspace's rail, "Workspace bots" has a "+" that opens `link-bots-modal.tsx` instead: the crew's workers not in it yet (`api/admin/workspaces/linkable-bots`), each added with `add-bot`, which pushes `server:bot-created`, and "New bot" stacking the bot form over it); on a workspace's pages new bots and computers join it. The rail shows the crew (`(partials)/rail-crew-list.tsx`, needs `bots` and `workspaces` in page props): the coordinator, the latest `AppData.RailRecentBots` workers (plus any open, unread or needing you, `pick-rail-bots`), "See all bots" with a few stones opening every bot in a modal, then Workspaces with a "+". On `/admin/workspaces/*` it shows the workspace's crew instead (`(partials)/rail-workspace-list.tsx`: back link, the workspace, then only the bots attached to it, "Workspace bots"; the coordinator stays in the main crew rail, and on a workspace bot's page the back link returns to the workspace page), except on `/admin/settings/*` where it shows "Back to bots" plus the settings sub pages (`(partials)/rail-settings-list.tsx`). Page heroes use the shared `AdminHero` component (`src/components/general/admin-hero.tsx`). Every `/admin/*` page renders in the bots layout (`src/layouts/bots/`); there is no separate admin layout or aside. The rail header's Add icon (super admins and admins only, like the rail's "+" buttons) opens the Add flow (`(partials)/add-flow.tsx`, `AddFlowContext`): a bot, a workspace or (super admins) a computer, each form in a modal stacked over the choice, so closing it goes back there (the Bots and Workspaces labels have a "+" that opens the form directly; in a workspace's rail, "Workspace bots" has a "+" that opens `link-bots-modal.tsx` instead: the crew's workers not in it yet (`api/admin/workspaces/linkable-bots`), each added with `add-bot`, which pushes `server:bot-created`, and "New bot" stacking the bot form over it); on a workspace's pages new bots and computers join it. The rail shows the crew (`(partials)/rail-crew-list.tsx`, needs `bots` and `workspaces` in page props): the coordinator, the latest `AppData.RailRecentBots` workers (plus any open, unread or needing you, `pick-rail-bots`), "See all bots" with a few stones opening every bot in a modal, then Workspaces with a "+". On `/admin/workspaces/*` it shows the workspace's crew instead (`(partials)/rail-workspace-list.tsx`: back link, the workspace, then only the bots attached to it, "Workspace bots"; the coordinator stays in the main crew rail, and on a workspace bot's page the back link returns to the workspace page), except on `/admin/settings/*` where it shows "Back to bots" plus the settings sub pages (`(partials)/rail-settings-list.tsx`). Page heroes use the shared `AdminHero` component (`src/components/general/admin-hero.tsx`).
@@ -269,6 +282,7 @@ Server functions use `index.server.ts` alongside `index.tsx` (e.g. `single-docum
- Shared: `Logo`, `LogoIcon`, `InnerPageHeroSection`, `AdminHero`, `LoginShell` (auth pages/forms), `TaglineSection` - Shared: `Logo`, `LogoIcon`, `InnerPageHeroSection`, `AdminHero`, `LoginShell` (auth pages/forms), `TaglineSection`
- **UNCHANGEABLE RULE - twui components always win over raw JSX elements.** Use twui components (`H1`-`H5`, `P`, `Span`, `List`, `Img`, `Row`, `Stack`, `Container`, `Section`, `Card`, `Button`, `Tag`, `Link`, etc.) before basic JSX elements (`<h1>`-`<h6>`, `<p>`, `<span>`, `<ul>`, `<li>`, `<img>`, `<a>`, `<button>`, `<div>`-as-card, etc.). Never introduce or reintroduce a raw JSX element where a twui equivalent exists. Override twui defaults via `className` rather than reaching for the raw element. - **UNCHANGEABLE RULE - twui components always win over raw JSX elements.** Use twui components (`H1`-`H5`, `P`, `Span`, `List`, `Img`, `Row`, `Stack`, `Container`, `Section`, `Card`, `Button`, `Tag`, `Link`, etc.) before basic JSX elements (`<h1>`-`<h6>`, `<p>`, `<span>`, `<ul>`, `<li>`, `<img>`, `<a>`, `<button>`, `<div>`-as-card, etc.). Never introduce or reintroduce a raw JSX element where a twui equivalent exists. Override twui defaults via `className` rather than reaching for the raw element.
- **Never create new CSS classes, CSS files, or inline `style={{...}}` hacks.** Style exclusively by composing existing component props/classNames, existing twui classNames (`twui-*`), semantic theme-token utilities (`bg-dark`, `bg-light`, `text-dark`, `text-primary`, `bg-primary`, `text-accent`, `bg-accent`, `text-secondary`, `border-slate-200`, etc.), and utility classes already used elsewhere in this codebase. Build up from what exists; extend component code only when the existing API cannot express the need. - **Never create new CSS classes, CSS files, or inline `style={{...}}` hacks.** Style exclusively by composing existing component props/classNames, existing twui classNames (`twui-*`), semantic theme-token utilities (`bg-dark`, `bg-light`, `text-dark`, `text-primary`, `bg-primary`, `text-accent`, `bg-accent`, `text-secondary`, `border-slate-200`, etc.), and utility classes already used elsewhere in this codebase. Build up from what exists; extend component code only when the existing API cannot express the need.
- **Chunk Tailwind classes with `tailwind-merge`, never one long line.** A `className` is `twMerge(...)` (`import { twMerge } from "tailwind-merge"`) with one chunk per claim: base layout and spacing first, then one string per conditional, and a caller's override last, so a later chunk always beats an earlier one instead of depending on source order in the CSS. Never concatenate conditional classes with a template literal or a bare `+`, and never leave a single 20-class string where the parts mean different things: `className={twMerge("w-full shrink-0 flex-col gap-0! items-stretch rounded-default py-1", selected && "bg-ink/5")}`.
- **Theming:** colors, radii, and fonts change only in `src/styles/main.css`: Cobble tokens as CSS vars on `:root` (light) and `.dark`, exposed in the `@theme inline` block as `paper`, `surface`, `sunken`, `ink`, `ink-secondary`, `ink-disabled`, `hairline`, `hairline-strong`, `on-ink`, `hue-<name>`, plus `shadow-popover` and `--m-fast/base/slow` motion vars (zeroed under reduced motion). Font is Lato (stand-in). Never hardcode hex colors, radii, or font stacks inside components. Hues are bot identity only, never state, and the `--color-hue-*` tokens remain for the fixed palettes that use them (the files browser's volume icons, `BotHues`' own hexes). - **Theming:** colors, radii, and fonts change only in `src/styles/main.css`: Cobble tokens as CSS vars on `:root` (light) and `.dark`, exposed in the `@theme inline` block as `paper`, `surface`, `sunken`, `ink`, `ink-secondary`, `ink-disabled`, `hairline`, `hairline-strong`, `on-ink`, `hue-<name>`, plus `shadow-popover` and `--m-fast/base/slow` motion vars (zeroed under reduced motion). Font is Lato (stand-in). Never hardcode hex colors, radii, or font stacks inside components. Hues are bot identity only, never state, and the `--color-hue-*` tokens remain for the fixed palettes that use them (the files browser's volume icons, `BotHues`' own hexes).
- **The one exception:** a bot's colour is user data, not a theme value, so it reaches `BotStone` (`src/components/general/bot-stone.tsx`) and `UnreadDot` as an SVG `fill` attribute - an arbitrary hex has no token to point at. Nothing else in the app may take a colour that way, and `resolveBotColor`/`normalizeBotColor` (`src/utils/bot-color.ts`) are the only place a colour string is validated. - **The one exception:** a bot's colour is user data, not a theme value, so it reaches `BotStone` (`src/components/general/bot-stone.tsx`) and `UnreadDot` as an SVG `fill` attribute - an arbitrary hex has no token to point at. Nothing else in the app may take a colour that way, and `resolveBotColor`/`normalizeBotColor` (`src/utils/bot-color.ts`) are the only place a colour string is validated.
- **Branding:** `docs/readme/hero-light.png`, `hero-dark.png`, `workspace-light.png` and `workspace-dark.png` are screenshots of the current app (chat and workspace dashboard, 1440px at 2x, downscaled to 1600px wide), committed and picked per theme by GitHub's `#gh-light-mode-only` / `#gh-dark-mode-only` fragment. They are not generated: when the UI changes, retake them from a seeded dev instance (`bun run seed`, `bun run seed:bots`) and keep the same names, so the README never shows an older dashboard. Never use `assets/handoff/assets/screens/*.png` for anything: that design is outdated. `public/logos/logo.svg` is the single source of truth for the logo mark; `logo-icon.tsx` renders it inline (via `BotStone`) and `logo.tsx` renders the stone beside the vector `Wordmark`. Whenever the logo changes, run `bun run gen:favicon` to regenerate `public/favicon.ico`, `public/favicon.png`, `public/apple-touch-icon.png`, and `public/logos/logo.png` (the last is used in emails). Font + favicon links live in `__root.tsx` `Head()`. - **Branding:** `docs/readme/hero-light.png`, `hero-dark.png`, `workspace-light.png` and `workspace-dark.png` are screenshots of the current app (chat and workspace dashboard, 1440px at 2x, downscaled to 1600px wide), committed and picked per theme by GitHub's `#gh-light-mode-only` / `#gh-dark-mode-only` fragment. They are not generated: when the UI changes, retake them from a seeded dev instance (`bun run seed`, `bun run seed:bots`) and keep the same names, so the README never shows an older dashboard. Never use `assets/handoff/assets/screens/*.png` for anything: that design is outdated. `public/logos/logo.svg` is the single source of truth for the logo mark; `logo-icon.tsx` renders it inline (via `BotStone`) and `logo.tsx` renders the stone beside the vector `Wordmark`. Whenever the logo changes, run `bun run gen:favicon` to regenerate `public/favicon.ico`, `public/favicon.png`, `public/apple-touch-icon.png`, and `public/logos/logo.png` (the last is used in emails). Font + favicon links live in `__root.tsx` `Head()`.
README.md
+12 -9
View File
@@ -245,15 +245,16 @@ queue and Web Push.
## Images ## Images
Every release publishes a multi-arch image to both registries: Every release publishes a multi-arch image to
`ghcr.io/moduletrace/cobble`, which is what the installer pulls. Tags are
`latest`, the release version, `major.minor` and the commit sha.
- `ghcr.io/moduletrace/cobble` The same image is copied into `git.tben.me/moduletrace/cobble` by
- `git.tben.me/moduletrace/cobble` `bun run mirror:gitea`, which runs after the build over the VPN (the Gitea
registry is not on the public internet, so GitHub Actions cannot reach it).
Tags are `latest`, the release version, `major.minor` and the commit sha. Install from Gitea with `--image git.tben.me/moduletrace/cobble`, or build
Install from Gitea with from source with `docker build -t cobble:local .` and
`--image git.tben.me/moduletrace/cobble`, or build from source with `--image cobble --version local --no-pull`.
`docker build -t cobble:local .` and `--image cobble --version local --no-pull`.
A release is published from a clean `main` with the two scripts, which tag the A release is published from a clean `main` with the two scripts, which tag the
`package.json` version, push and create the release objects: `package.json` version, push and create the release objects:
@@ -264,7 +265,9 @@ scripts/publish-gitea.sh # or: bun run publish:gitea
``` ```
Both take `--version VERSION`, `--dry-run` and `--yes`. Only the GitHub release Both take `--version VERSION`, `--dry-run` and `--yes`. Only the GitHub release
builds the image; see [RELEASING.md](RELEASING.md) for the full checklist. builds the image, and the Gitea registry is filled in afterwards, once that
build has finished and from the VPN, by `bun run mirror:gitea`; see
[RELEASING.md](RELEASING.md) for the full checklist.
## Development ## Development
RELEASING.md
+14 -6
View File
@@ -62,16 +62,24 @@ them in that order, since only the GitHub release builds images.
gh release create v0.1.0 --title v0.1.0 --notes-file notes.md --latest gh release create v0.1.0 --title v0.1.0 --notes-file notes.md --latest
7. [ ] The release workflow builds `linux/amd64` and `linux/arm64` and 7. [ ] The release workflow builds `linux/amd64` and `linux/arm64` and
pushes to both registries: pushes to `ghcr.io/moduletrace/cobble`, the registry the installer
- `ghcr.io/moduletrace/cobble` defaults to. It does not touch Gitea: that registry is on the VPN,
- `git.tben.me/moduletrace/cobble` and a GitHub runner cannot reach it.
8. [ ] Check the manifests: 8. [ ] Check the manifests:
docker buildx imagetools inspect ghcr.io/moduletrace/cobble:latest docker buildx imagetools inspect ghcr.io/moduletrace/cobble:latest
docker buildx imagetools inspect git.tben.me/moduletrace/cobble:latest
9. [ ] Confirm the GHCR package is public and pull it anonymously. 9. [ ] Confirm the GHCR package is public and pull it anonymously.
10. [ ] Publish the release to Gitea as well: 10. [ ] Once the build has finished, copy the image into the Gitea registry
from the VPN:
GITEA_TOKEN=... bun run mirror:gitea # or: scripts/mirror-gitea.sh
It copies the version and commit tags, plus `latest` and
`major.minor` while this release still owns them, and reads every
destination digest back. A failure here does not affect the release:
the installer keeps working from GHCR.
11. [ ] Publish the release to Gitea as well:
scripts/publish-gitea.sh # or: bun run publish:gitea scripts/publish-gitea.sh # or: bun run publish:gitea
@@ -79,7 +87,7 @@ them in that order, since only the GitHub release builds images.
git push gitea main git push gitea main
git push gitea refs/tags/v0.1.0 git push gitea refs/tags/v0.1.0
11. [ ] Update every install with the one-liner: 12. [ ] Update every install with the one-liner:
curl -fsSL https://url.moduletrace.com/cobble | sudo bash curl -fsSL https://url.moduletrace.com/cobble | sudo bash
db/schema.ts
+143
View File
@@ -2866,6 +2866,149 @@ const schema: BUN_SQLITE_DatabaseSchemaType = {
}, },
], ],
}, },
{
tableName: "token_usages",
tableDescription: `One model call that reported its usage: the tokens it spent, on which model and provider, for whom and for what. A bot message's total is the sum of the calls it took; the usage page sums them by model, provider and period`,
fields: [
{
fieldName: "user_id",
fieldDescription: `Whose work it was: the user whose conversation the call ran for`,
dataType: "INTEGER",
foreignKey: {
cascadeDelete: true,
destinationTableName: "users",
destinationTableColumnName: "id",
destinationTableColumnType: "INTEGER",
foreignKeyName: "token_usage_user_fk",
},
},
{
fieldName: "bot_id",
fieldDescription: `The bot it was for, when it was for one`,
dataType: "INTEGER",
foreignKey: {
cascadeDelete: true,
destinationTableName: "bots",
destinationTableColumnName: "id",
destinationTableColumnType: "INTEGER",
foreignKeyName: "token_usage_bot_fk",
},
},
{
fieldName: "workspace_id",
fieldDescription: `The workspace the call was in; null for a bot's own chat`,
dataType: "INTEGER",
foreignKey: {
cascadeDelete: true,
destinationTableName: "workspaces",
destinationTableColumnName: "id",
destinationTableColumnType: "INTEGER",
foreignKeyName: "token_usage_workspace_fk",
},
},
{
fieldName: "session_id",
fieldDescription: `The conversation it was in`,
dataType: "INTEGER",
foreignKey: {
cascadeDelete: true,
destinationTableName: "bot_sessions",
destinationTableColumnName: "id",
destinationTableColumnType: "INTEGER",
foreignKeyName: "token_usage_session_fk",
},
},
{
fieldName: "message_id",
fieldDescription: `The message it produced, so the thread can show what an answer cost. Null for calls that produce no message (naming a session, embedding a file)`,
dataType: "INTEGER",
foreignKey: {
cascadeDelete: true,
destinationTableName: "bot_messages",
destinationTableColumnName: "id",
destinationTableColumnType: "INTEGER",
foreignKeyName: "token_usage_message_fk",
},
},
{
fieldName: "job_id",
fieldDescription: `The queued job it ran for, when it ran for one`,
dataType: "INTEGER",
foreignKey: {
cascadeDelete: true,
destinationTableName: "bots_queue",
destinationTableColumnName: "id",
destinationTableColumnType: "INTEGER",
foreignKeyName: "token_usage_job_fk",
},
},
{
fieldName: "harness",
fieldDescription: `What the model was used for: a chat turn, reading media, transcribing, speaking, making an image, embedding`,
dataType: "TEXT",
options: Harnesses.map((h) => h.value),
},
{
fieldName: "provider_id",
fieldDescription: `The provider that served it. Not a foreign key: it stays readable after the provider is deleted`,
dataType: "INTEGER",
},
{
fieldName: "provider_name",
fieldDescription: `The provider's name as it was, so a deleted provider still reads`,
dataType: "TEXT",
},
{
fieldName: "model_id",
fieldDescription: `The catalog row of the model. Not a foreign key: a model the provider stops listing still counts`,
dataType: "INTEGER",
},
{
fieldName: "model_slug",
fieldDescription: `The provider's model id, as it was called`,
dataType: "TEXT",
},
{
fieldName: "prompt_tokens",
fieldDescription: `Tokens sent: the prompt, the system instructions and the tool results`,
dataType: "INTEGER",
defaultValue: 0,
},
{
fieldName: "completion_tokens",
fieldDescription: `Tokens written back: the answer, its tool calls and its reasoning`,
dataType: "INTEGER",
defaultValue: 0,
},
{
fieldName: "cost",
fieldDescription: `The provider's price in USD, when it reports one`,
dataType: "REAL",
},
],
indexes: [
{
indexName: "idx_token_usages_created_at",
indexTableFields: ["created_at"],
},
{
indexName: "idx_token_usages_message_id",
indexTableFields: ["message_id"],
},
{
indexName: "idx_token_usages_provider_model",
indexTableFields: ["provider_id", "model_slug"],
},
{
indexName: "idx_token_usages_user_created_at",
indexTableFields: ["user_id", "created_at"],
},
{
indexName: "idx_token_usages_bot_created_at",
indexTableFields: ["bot_id", "created_at"],
},
],
},
{ {
tableName: "bot_file_vectors", tableName: "bot_file_vectors",
tableDescription: `sqlite-vec vectors of a file's chunks, at \`AppData.EmbeddingDimensions\` wide whatever the model returns. A vec0 table is only recreated with \`bunx bun-sqlite schema --vector\`, so the entrypoint never passes it`, tableDescription: `sqlite-vec vectors of a file's chunks, at \`AppData.EmbeddingDimensions\` wide whatever the model returns. A vec0 table is only recreated with \`bunx bun-sqlite schema --vector\`, so the entrypoint never passes it`,
db/types/db.ts
+73 -1
View File
@@ -48,6 +48,7 @@ export const BunSQLiteTables = [
"envs", "envs",
"bot_files", "bot_files",
"bot_file_chunks", "bot_file_chunks",
"token_usages",
"bot_file_vectors", "bot_file_vectors",
"file_ingest_queue", "file_ingest_queue",
"app_settings", "app_settings",
@@ -1717,6 +1718,77 @@ export type BUN_SQLITE_COBBLE_BOT_FILE_CHUNKS = {
token_estimate?: number | ""; token_estimate?: number | "";
} }
export type BUN_SQLITE_COBBLE_TOKEN_USAGES = {
/**
* The unique identifier of the record.
*/
id?: number | "";
/**
* The time when the record was created. (Unix Timestamp)
*/
created_at?: number | "";
/**
* The time when the record was updated. (Unix Timestamp)
*/
updated_at?: number | "";
/**
* Whose work it was: the user whose conversation the call ran for
*/
user_id?: number | "";
/**
* The bot it was for, when it was for one
*/
bot_id?: number | "";
/**
* The workspace the call was in; null for a bot's own chat
*/
workspace_id?: number | "";
/**
* The conversation it was in
*/
session_id?: number | "";
/**
* The message it produced, so the thread can show what an answer cost. Null for calls that produce no message (naming a session, embedding a file)
*/
message_id?: number | "";
/**
* The queued job it ran for, when it ran for one
*/
job_id?: number | "";
/**
* What the model was used for: a chat turn, reading media, transcribing, speaking, making an image, embedding
*/
harness?: "text" | "vision" | "transcription" | "speech" | "image" | "embedding" | "";
/**
* The provider that served it. Not a foreign key: it stays readable after the provider is deleted
*/
provider_id?: number | "";
/**
* The provider's name as it was, so a deleted provider still reads
*/
provider_name?: string;
/**
* The catalog row of the model. Not a foreign key: a model the provider stops listing still counts
*/
model_id?: number | "";
/**
* The provider's model id, as it was called
*/
model_slug?: string;
/**
* Tokens sent: the prompt, the system instructions and the tool results
*/
prompt_tokens?: number | "";
/**
* Tokens written back: the answer, its tool calls and its reasoning
*/
completion_tokens?: number | "";
/**
* The provider's price in USD, when it reports one
*/
cost?: number | "";
}
export type BUN_SQLITE_COBBLE_BOT_FILE_VECTORS = { export type BUN_SQLITE_COBBLE_BOT_FILE_VECTORS = {
/** /**
* The unique identifier of the record. * The unique identifier of the record.
@@ -1800,4 +1872,4 @@ export type BUN_SQLITE_COBBLE_APP_SETTINGS = {
value?: string; value?: string;
} }
export type BUN_SQLITE_COBBLE_ALL_TYPEDEFS = BUN_SQLITE_COBBLE_USERS & BUN_SQLITE_COBBLE_USER_TYPES & BUN_SQLITE_COBBLE_SSO_LOGIN_CODES & BUN_SQLITE_COBBLE_USER_PASSWORDS & BUN_SQLITE_COBBLE_MEDIA & BUN_SQLITE_COBBLE_MEDIA_PARADIGMS & BUN_SQLITE_COBBLE_BOTS & BUN_SQLITE_COBBLE_BOT_SESSIONS & BUN_SQLITE_COBBLE_BOT_MESSAGES & BUN_SQLITE_COBBLE_BOT_MESSAGE_REACTIONS & BUN_SQLITE_COBBLE_BOT_TASKS & BUN_SQLITE_COBBLE_BOT_TASK_STEPS & BUN_SQLITE_COBBLE_TASKS & BUN_SQLITE_COBBLE_TASK_MEDIA & BUN_SQLITE_COBBLE_TASK_NOTES & BUN_SQLITE_COBBLE_BOTS_QUEUE & BUN_SQLITE_COBBLE_BOTS_QUEUE_UPDATES & BUN_SQLITE_COBBLE_CRON_JOBS & BUN_SQLITE_COBBLE_WORKSPACES & BUN_SQLITE_COBBLE_WORKSPACE_BOTS & BUN_SQLITE_COBBLE_WORKSPACE_USERS & BUN_SQLITE_COBBLE_BOT_USERS & BUN_SQLITE_COBBLE_BOT_USER_MEMORIES & BUN_SQLITE_COBBLE_BOT_USER_FACES & BUN_SQLITE_COBBLE_BOT_MESSAGE_MEDIA & BUN_SQLITE_COBBLE_CONTAINERS & BUN_SQLITE_COBBLE_COMPUTERS & BUN_SQLITE_COBBLE_BOT_COMPUTERS & BUN_SQLITE_COBBLE_WORKSPACE_COMPUTERS & BUN_SQLITE_COBBLE_HARNESS_PROVIDERS & BUN_SQLITE_COBBLE_HARNESS_MODELS & BUN_SQLITE_COBBLE_HARNESS_SETTINGS & BUN_SQLITE_COBBLE_BOT_HARNESS_SETTINGS & BUN_SQLITE_COBBLE_WORKSPACE_HARNESS_SETTINGS & BUN_SQLITE_COBBLE_MCP_SERVERS & BUN_SQLITE_COBBLE_CUSTOM_TOOLS & BUN_SQLITE_COBBLE_SKILLS & BUN_SQLITE_COBBLE_BOT_MCP_SERVERS & BUN_SQLITE_COBBLE_WORKSPACE_MCP_SERVERS & BUN_SQLITE_COBBLE_BOT_CUSTOM_TOOLS & BUN_SQLITE_COBBLE_WORKSPACE_CUSTOM_TOOLS & BUN_SQLITE_COBBLE_BOT_SKILLS & BUN_SQLITE_COBBLE_WORKSPACE_SKILLS & BUN_SQLITE_COBBLE_LOGS & BUN_SQLITE_COBBLE_USER_PRESENCE & BUN_SQLITE_COBBLE_PUSH_SUBSCRIPTIONS & BUN_SQLITE_COBBLE_ENVS & BUN_SQLITE_COBBLE_BOT_FILES & BUN_SQLITE_COBBLE_BOT_FILE_CHUNKS & BUN_SQLITE_COBBLE_BOT_FILE_VECTORS & BUN_SQLITE_COBBLE_FILE_INGEST_QUEUE & BUN_SQLITE_COBBLE_APP_SETTINGS export type BUN_SQLITE_COBBLE_ALL_TYPEDEFS = BUN_SQLITE_COBBLE_USERS & BUN_SQLITE_COBBLE_USER_TYPES & BUN_SQLITE_COBBLE_SSO_LOGIN_CODES & BUN_SQLITE_COBBLE_USER_PASSWORDS & BUN_SQLITE_COBBLE_MEDIA & BUN_SQLITE_COBBLE_MEDIA_PARADIGMS & BUN_SQLITE_COBBLE_BOTS & BUN_SQLITE_COBBLE_BOT_SESSIONS & BUN_SQLITE_COBBLE_BOT_MESSAGES & BUN_SQLITE_COBBLE_BOT_MESSAGE_REACTIONS & BUN_SQLITE_COBBLE_BOT_TASKS & BUN_SQLITE_COBBLE_BOT_TASK_STEPS & BUN_SQLITE_COBBLE_TASKS & BUN_SQLITE_COBBLE_TASK_MEDIA & BUN_SQLITE_COBBLE_TASK_NOTES & BUN_SQLITE_COBBLE_BOTS_QUEUE & BUN_SQLITE_COBBLE_BOTS_QUEUE_UPDATES & BUN_SQLITE_COBBLE_CRON_JOBS & BUN_SQLITE_COBBLE_WORKSPACES & BUN_SQLITE_COBBLE_WORKSPACE_BOTS & BUN_SQLITE_COBBLE_WORKSPACE_USERS & BUN_SQLITE_COBBLE_BOT_USERS & BUN_SQLITE_COBBLE_BOT_USER_MEMORIES & BUN_SQLITE_COBBLE_BOT_USER_FACES & BUN_SQLITE_COBBLE_BOT_MESSAGE_MEDIA & BUN_SQLITE_COBBLE_CONTAINERS & BUN_SQLITE_COBBLE_COMPUTERS & BUN_SQLITE_COBBLE_BOT_COMPUTERS & BUN_SQLITE_COBBLE_WORKSPACE_COMPUTERS & BUN_SQLITE_COBBLE_HARNESS_PROVIDERS & BUN_SQLITE_COBBLE_HARNESS_MODELS & BUN_SQLITE_COBBLE_HARNESS_SETTINGS & BUN_SQLITE_COBBLE_BOT_HARNESS_SETTINGS & BUN_SQLITE_COBBLE_WORKSPACE_HARNESS_SETTINGS & BUN_SQLITE_COBBLE_MCP_SERVERS & BUN_SQLITE_COBBLE_CUSTOM_TOOLS & BUN_SQLITE_COBBLE_SKILLS & BUN_SQLITE_COBBLE_BOT_MCP_SERVERS & BUN_SQLITE_COBBLE_WORKSPACE_MCP_SERVERS & BUN_SQLITE_COBBLE_BOT_CUSTOM_TOOLS & BUN_SQLITE_COBBLE_WORKSPACE_CUSTOM_TOOLS & BUN_SQLITE_COBBLE_BOT_SKILLS & BUN_SQLITE_COBBLE_WORKSPACE_SKILLS & BUN_SQLITE_COBBLE_LOGS & BUN_SQLITE_COBBLE_USER_PRESENCE & BUN_SQLITE_COBBLE_PUSH_SUBSCRIPTIONS & BUN_SQLITE_COBBLE_ENVS & BUN_SQLITE_COBBLE_BOT_FILES & BUN_SQLITE_COBBLE_BOT_FILE_CHUNKS & BUN_SQLITE_COBBLE_TOKEN_USAGES & BUN_SQLITE_COBBLE_BOT_FILE_VECTORS & BUN_SQLITE_COBBLE_FILE_INGEST_QUEUE & BUN_SQLITE_COBBLE_APP_SETTINGS
package.json
+3 -2
View File
@@ -1,6 +1,6 @@
{ {
"name": "cobble", "name": "cobble",
"version": "0.1.5", "version": "0.1.8",
"module": "index.ts", "module": "index.ts",
"type": "module", "type": "module",
"private": true, "private": true,
@@ -19,7 +19,8 @@
"test": "bun test", "test": "bun test",
"check:bundle": "bun src/scripts/check-bundle-size.ts", "check:bundle": "bun src/scripts/check-bundle-size.ts",
"publish:github": "bash scripts/publish-github.sh", "publish:github": "bash scripts/publish-github.sh",
"publish:gitea": "bash scripts/publish-gitea.sh" "publish:gitea": "bash scripts/publish-gitea.sh",
"mirror:gitea": "bash scripts/mirror-gitea.sh"
}, },
"devDependencies": { "devDependencies": {
"@next/mdx": "^16.3.0", "@next/mdx": "^16.3.0",
scripts/mirror-gitea.sh
Executable
+318
View File
@@ -0,0 +1,318 @@
#!/usr/bin/env bash
# Copies a published Cobble image from GHCR into the Gitea registry.
#
# scripts/mirror-gitea.sh
#
# The Gitea registry is on the VPN and GitHub Actions runners are on the
# public internet, so CI can never reach it. The release workflow therefore
# builds and pushes to GHCR only, and this script mirrors the result
# separately: a registry that is unreachable, slow or full can never fail a
# release the installer depends on.
#
# It copies from the Gitea host itself, because that host reaches GHCR over
# the public internet and its own registry over the docker network it shares
# with it. Nothing has to be installed on that host: skopeo runs there in a
# container.
#
# `skopeo copy --all` keeps the manifest list and its signatures, so every
# architecture comes across and the digest is unchanged. Each destination tag
# is then read back and compared, so a partial copy is reported as a failure
# instead of being assumed to have worked.
#
# Run it after scripts/publish-github.sh: that release is what builds the
# image, so there is nothing to copy until it has finished.
set -Eeuo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=lib/publish-common.sh
. "$SCRIPT_DIR/lib/publish-common.sh"
# shellcheck source=lib/publish-release.sh
. "$SCRIPT_DIR/lib/publish-release.sh"
SOURCE_REPO="${MIRROR_SOURCE_REPO:-moduletrace/cobble}"
SOURCE_REGISTRY_HOST="${MIRROR_SOURCE_REGISTRY_HOST:-ghcr.io}"
HOST="${MIRROR_HOST:-10.7.0.4}"
HOST_USER="${MIRROR_HOST_USER:-root}"
REGISTRY_HOST="${MIRROR_REGISTRY_HOST:-gitea:3000}"
REGISTRY_REPO="${MIRROR_REGISTRY_REPO:-moduletrace/cobble}"
# The name a person types, which is not the one the copy uses: the host
# reaches its registry by docker network name, while an install reaches it
# through the proxy in front of it.
REGISTRY_PUBLIC_HOST="${MIRROR_REGISTRY_PUBLIC_HOST:-git.tben.me}"
NETWORK="${MIRROR_NETWORK:-cloudflare}"
SKOPEO_IMAGE="${MIRROR_SKOPEO_IMAGE:-quay.io/skopeo/stable:latest}"
MIRROR_USER="${MIRROR_USER:-cobble}"
CONNECT_TIMEOUT="${MIRROR_CONNECT_TIMEOUT:-10}"
usage() {
cat <<EOF
Copies a published Cobble image from GHCR into the Gitea registry.
Usage: scripts/mirror-gitea.sh [options]
Options:
--version VERSION Version to mirror, with or without the leading v.
Defaults to package.json's version.
--yes, -y Do not prompt.
--dry-run Print what would run, copy nothing.
-h, --help Show this help.
The version is copied as vX.Y.Z and sha-<commit>. It is copied as latest and
major.minor too, but only while it is still the newest release in GHCR, so
mirroring an older release never rolls those two tags backwards. Every
destination is read back and its digest compared with GHCR's.
Run it after scripts/publish-github.sh, and after the image build for that
release has finished.
Environment:
GITEA_TOKEN Token for the registry (required).
MIRROR_HOST The Gitea host to copy from over the VPN.
Default: 10.7.0.4
MIRROR_HOST_USER Its ssh user. Default: root
MIRROR_REGISTRY_HOST The registry as that host sees it, by its docker
network name. Default: gitea:3000
MIRROR_REGISTRY_PUBLIC_HOST
The registry's public name, for the pull command
it prints. Default: git.tben.me
MIRROR_SOURCE_REGISTRY_HOST
Where the release workflow pushed.
Default: ghcr.io
MIRROR_NETWORK The docker network the registry is on, so the
copy container can resolve it.
Default: cloudflare
MIRROR_USER The registry user for basic auth. Default: cobble
MIRROR_SKOPEO_IMAGE The skopeo image the host runs.
Default: quay.io/skopeo/stable:latest
DRY_RUN=1, ASSUME_YES=1 Same as --dry-run and --yes.
EOF
}
parse_publish_args "$@"
if [ "$HELP" = "1" ]; then
usage
exit 0
fi
reject_unknown_args "$0"
# MARK:mirror-tags
# The moving tag a release owns: major.minor, read from the version the same
# way the release workflow reads it, so the two never disagree.
mirror_minor_tag() {
printf '%s' "${1#v}" | sed -E 's/^([0-9]+)\.([0-9]+).*/\1.\2/'
}
# The commit tag for a release: the first 12 characters of the commit the
# tag points at, which is `sha-${GITHUB_SHA::12}` in the workflow.
mirror_sha_tag() {
local sha
sha="$(git rev-parse -q --verify "refs/tags/$1^{commit}" 2>/dev/null || true)"
if [ -z "$sha" ]; then
return 1
fi
printf 'sha-%s' "${sha:0:12}"
}
# The tags this release owns outright, which no newer release claims.
mirror_fixed_tags() {
local sha_tag
printf '%s\n' "$1"
if sha_tag="$(mirror_sha_tag "$1")"; then
printf '%s\n' "$sha_tag"
fi
}
# The tags a newer release takes over.
mirror_moving_tags() {
printf 'latest\n%s\n' "$(mirror_minor_tag "$1")"
}
# END:mirror-tags
# MARK:mirror-guard
# Whether this release may still write the moving tags. It may while GHCR's
# `latest` is its own image: once a newer release has moved `latest`, copying
# an older version into the Gitea registry must not drag it back.
mirror_owns_moving_tags() {
if [ -z "$2" ]; then
# Nothing to compare against, so this release is taken as the newest.
return 0
fi
[ "$1" = "$2" ]
}
# END:mirror-guard
# MARK:mirror-digest
MANIFEST_ACCEPT="application/vnd.oci.image.index.v1+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.docker.distribution.manifest.v2+json"
# Reads a tag's digest from GHCR over the public internet. The token is
# anonymous: the image is public, and a registry answers a bare manifest
# request with 401 until it is given one.
github_registry_digest() {
local tag="$1"
local token
token="$(
curl -sSf "https://$SOURCE_REGISTRY_HOST/token?scope=repository:$SOURCE_REPO:pull" 2>/dev/null |
sed -n 's/.*"token":"\([^"]*\)".*/\1/p'
)" || return 1
if [ -z "$token" ]; then
return 1
fi
curl -sSf -o /dev/null -D- \
-H "Authorization: Bearer $token" \
-H "Accept: $MANIFEST_ACCEPT" \
"https://$SOURCE_REGISTRY_HOST/v2/$SOURCE_REPO/manifests/$tag" 2>/dev/null |
tr -d '\r' | sed -n 's/^docker-content-digest: //Ip' | head -n1
}
# END:mirror-digest
# MARK:mirror-remote
# Runs a bash script on the Gitea host, which is the one place that reaches
# both registries. The script arrives on stdin, so nothing is installed there
# and no quoting has to survive a round trip through ssh.
mirror_on_host() {
ssh -o BatchMode=yes -o ConnectTimeout="$CONNECT_TIMEOUT" "$HOST_USER@$HOST" \
"MIRROR_NETWORK='$NETWORK' MIRROR_SKOPEO_IMAGE='$SKOPEO_IMAGE' bash -s"
}
# Reads a destination tag's digest back from the registry. Empty means the
# tag is not there, which a caller reports differently from a mismatch.
mirror_digest_on_host() {
mirror_on_host <<REMOTE
set -uo pipefail
docker run --rm --network "\$MIRROR_NETWORK" "\$MIRROR_SKOPEO_IMAGE" \\
inspect --tls-verify=false --creds '$MIRROR_USER:$GITEA_TOKEN' \\
--format '{{.Digest}}' 'docker://$REGISTRY_HOST/$REGISTRY_REPO:$1' 2>/dev/null | tail -n1
REMOTE
}
# Copies one tag, leaving skopeo's output in MIRROR_COPY_LOG so a failure can
# be shown without streaming its progress into the report.
mirror_copy_on_host() {
MIRROR_COPY_LOG="$(mktemp -t cobble-mirror.XXXXXX)"
mirror_on_host <<REMOTE >"$MIRROR_COPY_LOG" 2>&1
set -uo pipefail
docker run --rm --network "\$MIRROR_NETWORK" "\$MIRROR_SKOPEO_IMAGE" \\
copy --all --retry-times 5 \\
--src-tls-verify=true --dest-tls-verify=false \\
--dest-creds '$MIRROR_USER:$GITEA_TOKEN' \\
'docker://$SOURCE_REGISTRY_HOST/$SOURCE_REPO:$1' \\
'docker://$REGISTRY_HOST/$REGISTRY_REPO:$2'
REMOTE
}
# END:mirror-remote
# MARK:mirror-main
require_repo_root
step "Resolving the version"
if [ -z "$VERSION_RAW" ]; then
VERSION_RAW="$(source_version)"
if [ -z "$VERSION_RAW" ]; then
fail "No version in package.json and none passed with --version."
fi
detail "Read from package.json: $VERSION_RAW"
fi
validate_version "$VERSION_RAW"
TAG="$(normalize_version "$VERSION_RAW")"
ok "Mirroring $TAG."
require_cmd git
require_cmd ssh "Install an ssh client."
require_cmd curl "Install curl."
if [ -z "${GITEA_TOKEN:-}" ]; then
fail "No Gitea token. Set GITEA_TOKEN to a token whose user can push packages to the $REGISTRY_REPO owner."
fi
step "Checking the released image in $SOURCE_REGISTRY_HOST"
SOURCE_DIGEST="$(github_registry_digest "$TAG")" ||
fail "No $TAG in $SOURCE_REGISTRY_HOST/$SOURCE_REPO. Publish the GitHub release first: scripts/publish-github.sh builds the image."
if [ -z "$SOURCE_DIGEST" ]; then
fail "No $TAG in $SOURCE_REGISTRY_HOST/$SOURCE_REPO. Publish the GitHub release first: scripts/publish-github.sh builds the image."
fi
ok "$TAG is $SOURCE_DIGEST."
LATEST_DIGEST="$(github_registry_digest latest || true)"
TAGS_TO_COPY="$(mirror_fixed_tags "$TAG")"
MOVING_SKIPPED=""
if mirror_owns_moving_tags "$SOURCE_DIGEST" "$LATEST_DIGEST"; then
TAGS_TO_COPY="$(
printf '%s\n%s\n' "$TAGS_TO_COPY" "$(mirror_moving_tags "$TAG")" |
awk 'NF && !seen[$0]++'
)"
else
MOVING_SKIPPED="$(mirror_moving_tags "$TAG" | tr '\n' ' ')"
fi
step "Mirroring $TAG into the Gitea registry"
printf ' %shost%s %s\n' "$C_DIM" "$C_RESET" "$HOST_USER@$HOST"
printf ' %sregistry%s %s/%s\n' "$C_DIM" "$C_RESET" "$REGISTRY_HOST" "$REGISTRY_REPO"
printf ' %ssource%s %s/%s\n' "$C_DIM" "$C_RESET" "$SOURCE_REGISTRY_HOST" "$SOURCE_REPO"
printf ' %stags%s %s\n' "$C_DIM" "$C_RESET" "$(printf '%s' "$TAGS_TO_COPY" | tr '\n' ' ')"
if [ -n "$MOVING_SKIPPED" ]; then
warn "$SOURCE_REGISTRY_HOST/$SOURCE_REPO:latest belongs to a newer release, so ${MOVING_SKIPPED% } is left where it is."
fi
step "Checking the Gitea host"
if is_dry_run; then
detail "[dry run] ssh -o BatchMode=yes $HOST_USER@$HOST 'bash -s'"
remote_hostname="$HOST"
else
remote_hostname="$(
mirror_on_host <<'REMOTE'
hostname
REMOTE
)" || fail "Could not reach $HOST_USER@$HOST over ssh. The Gitea host is on the VPN: connect to it first."
ok "Reached $remote_hostname."
fi
confirm "Mirror $TAG into $REGISTRY_HOST/$REGISTRY_REPO?"
failed=0
for dest_tag in $TAGS_TO_COPY; do
step "Copying $TAG to $dest_tag"
if is_dry_run; then
detail "[dry run] skopeo copy --all $SOURCE_REGISTRY_HOST/$SOURCE_REPO:$TAG -> $REGISTRY_HOST/$REGISTRY_REPO:$dest_tag"
continue
fi
if ! mirror_copy_on_host "$TAG" "$dest_tag"; then
sed -n '$!{/^$/d};p' "${MIRROR_COPY_LOG:-/dev/null}" 2>/dev/null | tail -n 8 >&2
warn "The copy to $dest_tag failed."
failed=1
continue
fi
# The digest is what makes the two registries the same image: a copy that
# stored a different manifest list would read back differently here.
copied_digest="$(mirror_digest_on_host "$dest_tag")"
if [ "$copied_digest" != "$SOURCE_DIGEST" ]; then
warn "$dest_tag reads back as ${copied_digest:-nothing}, not $SOURCE_DIGEST."
failed=1
continue
fi
ok "$dest_tag is $SOURCE_DIGEST."
done
if [ -n "${MIRROR_COPY_LOG:-}" ] && [ -f "$MIRROR_COPY_LOG" ]; then
rm -f "$MIRROR_COPY_LOG"
fi
step "Mirrored $TAG"
printf ' %sregistry%s %s/%s\n' "$C_DIM" "$C_RESET" "$REGISTRY_HOST" "$REGISTRY_REPO"
printf ' %stags%s %s\n' "$C_DIM" "$C_RESET" "$(printf '%s' "$TAGS_TO_COPY" | tr '\n' ' ')"
printf ' %sdigest%s %s\n' "$C_DIM" "$C_RESET" "$SOURCE_DIGEST"
if [ "$DRY_RUN" = "1" ]; then
printf '\n%sDry run: copied nothing.%s\n' "$C_YELLOW" "$C_RESET"
fi
if [ "$failed" = "1" ]; then
fail "The Gitea registry does not match $SOURCE_REGISTRY_HOST/$SOURCE_REPO. The release itself is unaffected: the installer defaults to GHCR."
fi
printf '\nAn install that can reach the Gitea registry can pull it with:\n'
printf ' curl -fsSL https://url.moduletrace.com/cobble | sudo bash -s -- --image %s/%s:%s\n' \
"$REGISTRY_PUBLIC_HOST" "$REGISTRY_REPO" "$TAG"
# END:mirror-main
scripts/publish-gitea.sh
+6 -6
View File
@@ -8,9 +8,10 @@
# creates the Gitea release over the API. # creates the Gitea release over the API.
# #
# Gitea has no release workflow, so nothing is built here: the image is built # Gitea has no release workflow, so nothing is built here: the image is built
# once by .github/workflows/release.yml and mirrored to # once by .github/workflows/release.yml and copied into the Gitea registry by
# git.tben.me/moduletrace/cobble. This script only publishes the source and # scripts/mirror-gitea.sh, run from the VPN side, since a GitHub runner can
# the release object. # never reach it. This script only publishes the source and the release
# object.
# #
# The token comes from GITEA_TOKEN, or from the Gitea section of ~/.config/tea # The token comes from GITEA_TOKEN, or from the Gitea section of ~/.config/tea
# if the tea CLI is installed. It needs the `write:repository` scope on the # if the tea CLI is installed. It needs the `write:repository` scope on the
@@ -221,6 +222,5 @@ else
fi fi
print_release_summary "$REMOTE" "$BRANCH" "https://${REMOTE_HOST}/${REPO_SLUG}/releases/tag/$TAG" print_release_summary "$REMOTE" "$BRANCH" "https://${REMOTE_HOST}/${REPO_SLUG}/releases/tag/$TAG"
printf '\nThe image is built by the GitHub release; mirror it into the Gitea registry with:\n' printf '\nThe image is built by the GitHub release, which cannot reach this registry.\nOnce that build has finished, copy it in over the VPN with:\n'
printf ' docker buildx imagetools create --tag git.tben.me/moduletrace/cobble:latest \\\n' printf ' bun run mirror:gitea\n'
printf ' ghcr.io/moduletrace/cobble:latest\n'
scripts/publish-github.sh
+4 -3
View File
@@ -7,11 +7,12 @@
# installer syntax), tags HEAD with the package.json version, pushes main and # installer syntax), tags HEAD with the package.json version, pushes main and
# the tag to the `origin` remote, then creates the GitHub release. Publishing # the tag to the `origin` remote, then creates the GitHub release. Publishing
# that release is what fires .github/workflows/release.yml, which builds the # that release is what fires .github/workflows/release.yml, which builds the
# multi-arch image and pushes it to GHCR (and mirrors it to Gitea). # multi-arch image and pushes it to GHCR, the registry the installer defaults
# to. It never pushes to Gitea: that registry is on the VPN, out of a
# runner's reach, so scripts/mirror-gitea.sh copies the image separately.
# #
# Run scripts/publish-gitea.sh afterwards for the source mirror's own release # Run scripts/publish-gitea.sh afterwards for the source mirror's own release
# object. There is no existing publish script in the repo: this and its Gitea # object, and scripts/mirror-gitea.sh once the image build has finished.
# twin are it.
set -Eeuo pipefail set -Eeuo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
scripts/publish-tools.test.ts
+160 -1
View File
@@ -4,6 +4,7 @@ import path from "path";
const ROOT = path.join(import.meta.dir, ".."); const ROOT = path.join(import.meta.dir, "..");
const COMMON = path.join(ROOT, "scripts/lib/publish-common.sh"); const COMMON = path.join(ROOT, "scripts/lib/publish-common.sh");
const RELEASE = path.join(ROOT, "scripts/lib/publish-release.sh"); const RELEASE = path.join(ROOT, "scripts/lib/publish-release.sh");
const MIRROR = path.join(ROOT, "scripts/mirror-gitea.sh");
const SCRIPTS = [ const SCRIPTS = [
path.join(ROOT, "scripts/publish-github.sh"), path.join(ROOT, "scripts/publish-github.sh"),
path.join(ROOT, "scripts/publish-gitea.sh"), path.join(ROOT, "scripts/publish-gitea.sh"),
@@ -99,13 +100,21 @@ async function pathWithoutJq(): Promise<string> {
} }
describe("publish scripts syntax", () => { describe("publish scripts syntax", () => {
for (const script of [...SCRIPTS, COMMON, RELEASE]) { for (const script of [...SCRIPTS, MIRROR, COMMON, RELEASE]) {
test(`${path.relative(ROOT, script)} parses`, async () => { test(`${path.relative(ROOT, script)} parses`, async () => {
const proc = Bun.spawn({ cmd: ["bash", "-n", script] }); const proc = Bun.spawn({ cmd: ["bash", "-n", script] });
expect(await proc.exited).toBe(0); expect(await proc.exited).toBe(0);
}); });
} }
test("the mirror script sources the shared libraries", async () => {
const source = await Bun.file(MIRROR).text();
expect(source).toContain('SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"');
expect(source).toContain('. "$SCRIPT_DIR/lib/publish-common.sh"');
expect(source).toContain('. "$SCRIPT_DIR/lib/publish-release.sh"');
});
test("both publish scripts source the shared library", async () => { test("both publish scripts source the shared library", async () => {
for (const script of SCRIPTS) { for (const script of SCRIPTS) {
const source = await Bun.file(script).text(); const source = await Bun.file(script).text();
@@ -141,6 +150,156 @@ describe("publish scripts remote wiring", () => {
}); });
}); });
describe("mirror script wiring", () => {
test("CI never pushes to the Gitea registry", async () => {
const workflow = await Bun.file(
path.join(ROOT, ".github/workflows/release.yml"),
).text();
expect(workflow).not.toContain("git.tben.me");
expect(workflow).not.toContain("GITEA_TOKEN");
expect(workflow).not.toContain("imagetools create");
expect(workflow).toContain("Build and push to GHCR");
});
test("the mirror script copies every tag the workflow publishes", async () => {
const [workflow, mirror] = await Promise.all([
Bun.file(path.join(ROOT, ".github/workflows/release.yml")).text(),
Bun.file(MIRROR).text(),
]);
// The workflow's own four tags, so a mirror tag can never be invented
// here and go missing there.
for (const tag of ["latest", "${version}", "${minor}", "${sha}"]) {
expect(workflow).toContain(tag);
}
expect(mirror).toContain("mirror_minor_tag");
expect(mirror).toContain("mirror_sha_tag");
expect(mirror).toContain("mirror_fixed_tags");
expect(mirror).toContain("mirror_moving_tags");
});
test("the mirror script asks before copying and verifies each digest", async () => {
const source = await Bun.file(MIRROR).text();
expect(source).toContain("confirm \"Mirror $TAG into");
expect(source).toContain("mirror_digest_on_host");
expect(source).toContain("skopeo");
expect(source).toContain("--all");
});
test("the pull command it prints names the public host, not the docker one", async () => {
const source = await Bun.file(MIRROR).text();
// The copy reaches the registry by docker network name, which is not
// resolvable for an install; printing that would hand out a broken
// command.
expect(source).toContain("MIRROR_REGISTRY_PUBLIC_HOST");
expect(source).toContain('"$REGISTRY_PUBLIC_HOST" "$REGISTRY_REPO" "$TAG"');
expect(source).not.toContain('"${REGISTRY_HOST%%:*}" "$REGISTRY_REPO" "$TAG"');
});
test("without a token the mirror script refuses instead of guessing", async () => {
const proc = Bun.spawn({
cmd: ["bash", MIRROR, "--yes"],
cwd: ROOT,
env: { ...process.env, GITEA_TOKEN: "" },
stdout: "pipe",
stderr: "pipe",
});
const [stderr, code] = await Promise.all([
new Response(proc.stderr).text(),
proc.exited,
]);
expect(code).toBe(1);
expect(stderr).toContain("No Gitea token");
});
});
describe("mirror lib tag handling", () => {
test("major.minor is read exactly as the release workflow reads it", async () => {
const block = await blockCaller({ file: MIRROR, name: "mirror-tags" });
for (const [version, expected] of [
["v0.1.5", "0.1"],
["v1.2.3", "1.2"],
["v10.20.30", "10.20"],
["v0.1.5-rc.1", "0.1"],
] as const) {
const out = await runBash({
script: [block, `mirror_minor_tag '${version}'`].join("\n"),
});
expect(out).toBe(expected);
}
});
test("fixed tags are the version and its commit, moving ones the rest", async () => {
const block = await blockCaller({ file: MIRROR, name: "mirror-tags" });
const fixed = await runBash({
script: [block, "mirror_fixed_tags v0.1.5"].join("\n"),
});
expect(fixed.split("\n")[0]).toBe("v0.1.5");
const moving = await runBash({
script: [block, "mirror_moving_tags v0.1.5"].join("\n"),
});
expect(moving.split("\n")).toEqual(["latest", "0.1"]);
});
test("a tag with no commit on disk still mirrors its version", async () => {
const block = await blockCaller({ file: MIRROR, name: "mirror-tags" });
const out = await runBash({
script: [block, "mirror_fixed_tags v99.99.99"].join("\n"),
});
expect(out).toBe("v99.99.99");
});
});
describe("mirror lib moving tag guard", () => {
test("a release may move latest while it owns it", async () => {
const block = await blockCaller({ file: MIRROR, name: "mirror-guard" });
const out = await runBash({
script: [
block,
'mirror_owns_moving_tags sha256:aa sha256:aa && printf owns',
].join("\n"),
});
expect(out).toBe("owns");
});
test("a newer release's latest is never rolled back", async () => {
const block = await blockCaller({ file: MIRROR, name: "mirror-guard" });
const out = await runBash({
script: [
block,
'if mirror_owns_moving_tags sha256:aa sha256:bb; then printf owns; else printf refuses; fi',
].join("\n"),
});
expect(out).toBe("refuses");
});
test("with no latest to compare, the release is taken as newest", async () => {
const block = await blockCaller({ file: MIRROR, name: "mirror-guard" });
const out = await runBash({
script: [
block,
'mirror_owns_moving_tags sha256:aa "" && printf owns',
].join("\n"),
});
expect(out).toBe("owns");
});
});
describe("publish lib version handling", () => { describe("publish lib version handling", () => {
test("normalizes a version with and without the v", async () => { test("normalizes a version with and without the v", async () => {
const block = await blockCaller({ name: "version" }); const block = await blockCaller({ name: "version" });
src/bots-queue/name-bot-session.ts
+26 -9
View File
@@ -4,6 +4,7 @@ import { AppData } from "@/src/data/app-data";
import { SessionNamePrompt } from "@/src/data/harness-prompts"; import { SessionNamePrompt } from "@/src/data/harness-prompts";
import grabBotMessages from "@/src/functions/backend/db/bots/grab-bot-messages"; import grabBotMessages from "@/src/functions/backend/db/bots/grab-bot-messages";
import generateText from "@/src/harness/text/generate-text"; import generateText from "@/src/harness/text/generate-text";
import { runWithHarnessUsageScope } from "@/src/harness/usage/harness-usage-scope";
import parseSessionName from "./parse-session-name"; import parseSessionName from "./parse-session-name";
type Params = { type Params = {
@@ -30,7 +31,7 @@ export default async function nameBotSession({
}: Params): Promise<boolean> { }: Params): Promise<boolean> {
const session = ( const session = (
await BunSQLite.sql<BUN_SQLITE_COBBLE_BOT_SESSIONS>({ await BunSQLite.sql<BUN_SQLITE_COBBLE_BOT_SESSIONS>({
sql: `SELECT id, title, summary FROM bot_sessions WHERE id = ? LIMIT 1`, sql: `SELECT id, user_id, title, summary FROM bot_sessions WHERE id = ? LIMIT 1`,
values: [session_id], values: [session_id],
}) })
).payload?.[0]; ).payload?.[0];
@@ -57,14 +58,30 @@ export default async function nameBotSession({
) )
.join("\n\n"); .join("\n\n");
const result = await generateText({ /**
messages: [ * The naming turn is a model call too, so it is recorded for this
{ role: "system", content: SessionNamePrompt }, * conversation. It runs after the job is complete, on purpose, so it
{ role: "user", content: transcript }, * has no job and no message of its own to charge it to.
], */
bot_id, const result = await runWithHarnessUsageScope({
workspace_id, scope: {
signal, user_id: session.user_id || undefined,
bot_id,
workspace_id: workspace_id || null,
session_id: session.id,
job_id: null,
message_id: null,
},
task: () =>
generateText({
messages: [
{ role: "system", content: SessionNamePrompt },
{ role: "user", content: transcript },
],
bot_id,
workspace_id,
signal,
}),
}); });
const name = parseSessionName({ text: result.text }); const name = parseSessionName({ text: result.text });
src/bots-queue/run-bot-job.ts
+20 -3
View File
@@ -1,6 +1,7 @@
import grabBotById from "@/src/functions/backend/db/bots/grab-bot-by-id"; import grabBotById from "@/src/functions/backend/db/bots/grab-bot-by-id";
import grabJobUserId from "@/src/functions/backend/db/bots-queue/grab-job-user-id"; import grabJobUserId from "@/src/functions/backend/db/bots-queue/grab-job-user-id";
import HarnessError from "@/src/harness/harness-error"; import HarnessError from "@/src/harness/harness-error";
import { runWithHarnessUsageScope } from "@/src/harness/usage/harness-usage-scope";
import BotJobFatalError from "./bot-job-fatal-error"; import BotJobFatalError from "./bot-job-fatal-error";
import runBotReply from "./run-bot-reply"; import runBotReply from "./run-bot-reply";
import runBotTask from "./run-bot-task"; import runBotTask from "./run-bot-task";
@@ -33,9 +34,25 @@ const runBotJob: BotJobRunner = async (context) => {
const owned = { ...bot, id: bot.id }; const owned = { ...bot, id: bot.id };
try { try {
return job.kind == "reply" /**
? await runBotReply({ ...context, bot: owned, user_id }) * Everything the job's models do is recorded for this user, this
: await runBotTask({ ...context, bot: owned, user_id }); * bot, this conversation and this job. A reply's text loop adds
* the message it is producing to the one-turn scope it opens.
*/
return await runWithHarnessUsageScope({
scope: {
user_id,
bot_id: bot.id,
workspace_id: job.workspace_id || null,
session_id: job.session_id || null,
job_id: job.id || null,
message_id: null,
},
task: () =>
job.kind == "reply"
? runBotReply({ ...context, bot: owned, user_id })
: runBotTask({ ...context, bot: owned, user_id }),
});
} catch (error: unknown) { } catch (error: unknown) {
if ( if (
error instanceof HarnessError && error instanceof HarnessError &&
src/bots-queue/run-bot-reply.ts
+8 -7
View File
@@ -2,15 +2,16 @@ import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db";
import { AppData } from "@/src/data/app-data"; import { AppData } from "@/src/data/app-data";
import { import {
ActionCheckNote, ActionCheckNote,
BotFilesNote, botFilesNote,
ChatReplyNote, chatReplyNote,
ComputerFilesNote, ComputerFilesNote,
CoordinatorReplyNote, coordinatorReplyNote,
CoordinatorSoloNote, CoordinatorSoloNote,
TasksNote, TasksNote,
TaskReminderNote, TaskReminderNote,
WorkspaceLearningNote, WorkspaceLearningNote,
} from "@/src/data/harness-prompts"; } from "@/src/data/harness-prompts";
import { grabReplyMaxToolSteps } from "@/src/functions/backend/db/app-settings/grab-run-limits";
import grabBotMessages from "@/src/functions/backend/db/bots/grab-bot-messages"; import grabBotMessages from "@/src/functions/backend/db/bots/grab-bot-messages";
import grabBotPrompt from "@/src/functions/backend/db/bots/grab-bot-prompt"; import grabBotPrompt from "@/src/functions/backend/db/bots/grab-bot-prompt";
import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers"; import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers";
@@ -137,7 +138,7 @@ export default async function runBotReply({
bot.kind != "coordinator" bot.kind != "coordinator"
? "" ? ""
: tools.some((tool) => tool.name == "delegate_task") : tools.some((tool) => tool.name == "delegate_task")
? CoordinatorReplyNote ? coordinatorReplyNote({ bot_id: bot.id })
: CoordinatorSoloNote; : CoordinatorSoloNote;
try { try {
@@ -145,8 +146,8 @@ export default async function runBotReply({
system: [ system: [
prompt, prompt,
carry_over, carry_over,
ChatReplyNote, chatReplyNote({ bot_id: bot.id }),
BotFilesNote, botFilesNote({ bot_id: bot.id }),
ComputerFilesNote({ computer_id: computers[0]?.id || 0, can_import: false }), ComputerFilesNote({ computer_id: computers[0]?.id || 0, can_import: false }),
TasksNote, TasksNote,
TaskReminderNote, TaskReminderNote,
@@ -173,7 +174,7 @@ export default async function runBotReply({
}), }),
}), }),
tools: [...tools, ...file_tools, ...extensions.tools, ...skills.tools], tools: [...tools, ...file_tools, ...extensions.tools, ...skills.tools],
max_steps: AppData.BotReplyMaxToolSteps, max_steps: await grabReplyMaxToolSteps(),
bot_id: bot.id, bot_id: bot.id,
workspace_id, workspace_id,
signal, signal,
src/bots-queue/run-bot-task.ts
+4 -2
View File
@@ -1,13 +1,14 @@
import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db"; import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db";
import { import {
ActionCheckNote, ActionCheckNote,
BotFilesNote, botFilesNote,
ComputerFilesNote, ComputerFilesNote,
ScheduledRunNote, ScheduledRunNote,
TasksNote, TasksNote,
TaskRunNote, TaskRunNote,
WorkspaceLearningNote, WorkspaceLearningNote,
} from "@/src/data/harness-prompts"; } from "@/src/data/harness-prompts";
import { grabHarnessMaxToolSteps } from "@/src/functions/backend/db/app-settings/grab-run-limits";
import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers"; import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers";
import grabBotPrompt from "@/src/functions/backend/db/bots/grab-bot-prompt"; import grabBotPrompt from "@/src/functions/backend/db/bots/grab-bot-prompt";
import grabUserWorkspaces from "@/src/functions/backend/db/workspaces/grab-user-workspaces"; import grabUserWorkspaces from "@/src/functions/backend/db/workspaces/grab-user-workspaces";
@@ -190,7 +191,7 @@ export default async function runBotTask({
carry_over, carry_over,
TaskRunNote, TaskRunNote,
scheduled ? ScheduledRunNote : "", scheduled ? ScheduledRunNote : "",
BotFilesNote, botFilesNote({ bot_id: bot.id }),
ComputerFilesNote({ computer_id: job.computer_id || 0 }), ComputerFilesNote({ computer_id: job.computer_id || 0 }),
TasksNote, TasksNote,
ActionCheckNote, ActionCheckNote,
@@ -227,6 +228,7 @@ export default async function runBotTask({
}, },
], ],
tools: [...tools, ...extensions.tools, ...skills.tools], tools: [...tools, ...extensions.tools, ...skills.tools],
max_steps: await grabHarnessMaxToolSteps(),
bot_id: bot.id, bot_id: bot.id,
workspace_id, workspace_id,
signal, signal,
src/components/general/admin-breadcrumbs.tsx
+59
View File
@@ -0,0 +1,59 @@
import Breadcrumbs from "@moduletrace/twui/elements/Breadcrumbs";
import Stack from "@moduletrace/twui/layout/Stack";
import { ChevronRight } from "lucide-react";
import { AdminHomePath } from "@/src/dict/admin-sections-dict";
import hasBreadcrumbTrail from "@/src/utils/has-breadcrumb-trail";
import Divider from "@moduletrace/twui/layout/Divider";
import { twMerge } from "tailwind-merge";
type Props = {
/**
* The address the page is at, e.g. `/admin/settings/bots/3`.
*/
pathname?: string | null;
/**
* The page's own name, for the last crumb: what the address can't say,
* since its last segment is an id (`Scout`, not `3`) or a slug that
* differs from the name the page shows (`MCPs`, not `Mcps`).
*/
title?: string;
};
/**
* The trail above a page's content: `Home`, then every section the address
* passes through, then the page itself, named by `title`.
*
* It is `twui`'s `Breadcrumbs`, which reads the trail off the path, so the
* sections come from the address and only the page's own name is given
* here; `hasBreadcrumbTrail` says which pages get one at all.
*/
export default function AdminBreadcrumbs({ pathname, title }: Props) {
if (!hasBreadcrumbTrail({ pathname })) return null;
return (
<Stack className={twMerge("w-full shrink-0 pt-5 pb-1")}>
<Breadcrumbs
pageUrl={pathname || AdminHomePath}
currentTitle={title}
excludeRegexMatch={/^admin$/}
linkProps={{
className:
"border-0 text-[13px] leading-[18px] text-ink-secondary no-underline hover:text-ink hover:opacity-100 dark:text-ink-secondary",
}}
currentLinkProps={{
title,
className:
"border-0 text-[13px] leading-[18px] text-ink no-underline pointer-events-none dark:text-ink",
}}
divider={
<ChevronRight
size={12}
aria-hidden
className="shrink-0 text-ink-secondary"
/>
}
/>
<Divider />
</Stack>
);
}
src/data/app-data.ts
+23 -1
View File
@@ -246,7 +246,10 @@ export const AppData = {
HarnessCooldownMs: 60 * 1000, HarnessCooldownMs: 60 * 1000,
HarnessMaxCooldownMs: 24 * 60 * 60 * 1000, HarnessMaxCooldownMs: 24 * 60 * 60 * 1000,
/** /**
* Model turns in one text run before it must stop and report. * Model turns in one text run before it must stop and report. The
* install may save its own limit (the `harness_max_tool_steps` setting,
* Settings > General), which wins over this one; a task run reads it
* through `grabHarnessMaxToolSteps`.
*/ */
HarnessMaxToolSteps: 200, HarnessMaxToolSteps: 200,
/** /**
@@ -328,6 +331,16 @@ export const AppData = {
* How many older messages each scroll to the top of the thread loads. * How many older messages each scroll to the top of the thread loads.
*/ */
ConversationOlderMessages: 30, ConversationOlderMessages: 30,
/**
* Days of usage the usage page totals by default, and the most it will
* total at once.
*/
UsageDefaultDays: 30,
UsageMaxDays: 365,
/**
* Models listed per page on the usage page, the busiest first.
*/
UsageModelPageSize: 20,
/** /**
* How much of a bot's last message the rail's lists carry, as plain * How much of a bot's last message the rail's lists carry, as plain
* text (`messagePreview`): a row shows one line of it. * text (`messagePreview`): a row shows one line of it.
@@ -379,8 +392,17 @@ export const AppData = {
WorkspaceActivityItems: 6, WorkspaceActivityItems: 6,
/** /**
* Tool turns a chat reply may take. Long work goes to a task instead. * Tool turns a chat reply may take. Long work goes to a task instead.
* The install may save its own limit (the `bot_reply_max_tool_steps`
* setting, Settings > General), which wins over this one; a reply
* reads it through `grabReplyMaxToolSteps`.
*/ */
BotReplyMaxToolSteps: 8, BotReplyMaxToolSteps: 8,
/**
* The most steps a saved run limit may ask for, for a task run or a
* chat reply. A typo in the settings page can't turn one run into an
* hour of tool calls, and a huge limit still ends with a report.
*/
BotStepLimitMax: 1000,
/** /**
* Chat attachments: per file, per message, and how much of a * Chat attachments: per file, per message, and how much of a
* document's text a reply reads. * document's text a reply reads.
src/data/coordinator-system-prompt.ts
+3
View File
@@ -27,6 +27,9 @@ Permission to draft is not permission to send, permission to analyze is not perm
## 2. Delegate well ## 2. Delegate well
- Pick by role, instructions and tools, not by name. - Pick by role, instructions and tools, not by name.
- Say where the work runs, and mean it. \`delegate_task\` takes an optional \`workspace_id\`, and that one argument decides the whole context the worker has: its own chat, or a workspace. A task in the worker's own chat has only that worker's own memory, files and directly assigned computers. A task in a workspace has the workspace's shared context, the worker's instructions and memory for that workspace, the project's files, and the workspace's computers as well as the worker's. The result also lands in a different conversation: the worker's own chat, or its conversation in that workspace beside the rest of that project. Choose by where the work belongs, not by which is easier to type.
- Handed over from a conversation inside a workspace, the task always runs in that workspace and \`workspace_id\` is not accepted: say so, don't pretend you moved it.
- Delegating into a workspace the worker isn't in adds it to that workspace as a side effect. Say that in your reply when it happens; don't add members silently for a one-off.
- Give a full brief: objective, inputs, deliverable, acceptance criteria, constraints. - Give a full brief: objective, inputs, deliverable, acceptance criteria, constraints.
- Check \`list_tasks\` first. Never give two bots the same job, and don't restart work an open task already covers. - Check \`list_tasks\` first. Never give two bots the same job, and don't restart work an open task already covers.
- Track what comes back, judge it against the request, and integrate it. Don't pass a worker's unchecked claim on as fact. - Track what comes back, judge it against the request, and integrate it. Don't pass a worker's unchecked claim on as fact.
src/data/harness-prompts.test.ts
+57
View File
@@ -0,0 +1,57 @@
import { describe, expect, test } from "bun:test";
import {
botFilesNote,
chatReplyNote,
ComputerFilesNote,
coordinatorReplyNote,
TasksNote,
} from "./harness-prompts";
/**
* Every note a bot reads names it by its own id, so a bot copying an
* example never points at another bot's folder, conversation or task.
*/
const bot_ids = [1, 5, 42] as const;
/**
* The bot ids a note's own-folder examples point at, in order.
*/
function folderOwners(note: string) {
return [...note.matchAll(/\[folder:bot:(\d+):/g)].map((match) => match[1]);
}
describe("harness prompts", () => {
for (const bot_id of bot_ids) {
test(`chat reply note names bot ${bot_id}'s own folder`, () => {
const owners = folderOwners(chatReplyNote({ bot_id }));
expect(owners).toEqual([String(bot_id)]);
});
test(`files note names bot ${bot_id}'s own folder`, () => {
const owners = folderOwners(botFilesNote({ bot_id }));
expect(owners).toEqual([String(bot_id)]);
});
test(`coordinator note names itself as bot ${bot_id}`, () => {
const note = coordinatorReplyNote({ bot_id });
expect(note).toContain(`[bot:${bot_id}]`);
expect(note).not.toContain("[bot:3]");
});
}
test("a note that already names the running bot is unchanged by its id", () => {
expect(botFilesNote({ bot_id: 5 })).toContain("[folder:bot:5:reports]");
});
test("the computer files note keeps naming the computer it was given", () => {
expect(ComputerFilesNote({ computer_id: 8 })).toContain("[folder:computer:8:seo/coderank/raw]");
});
test("notes with nothing bot-specific carry no ref example", () => {
expect(TasksNote).not.toContain("[folder:bot:");
expect(TasksNote).not.toContain("[bot:");
});
});
src/data/harness-prompts.ts
+20 -7
View File
@@ -19,9 +19,19 @@ export const ReadImagePrompt = `Describe this image for someone who can't see it
export const MediaReadNote = `Some attachments in this conversation were read for you by another model; you see its reading in [brackets] instead of the file.`; export const MediaReadNote = `Some attachments in this conversation were read for you by another model; you see its reading in [brackets] instead of the file.`;
/** /**
* Added to a bot's prompt when it answers the user in chat. * The folder-ref example a note shows, always over the running bot's own
* folder: a literal owner id in the prompt teaches the bot to name another
* bot's folder as if it were its own.
*/ */
export const ChatReplyNote = `# Replying in chat function ownFolderRef({ bot_id }: { bot_id: number | string }) {
return folderRef({ volume: "bot", owner_id: bot_id, path: "reports" });
}
/**
* Added to a bot's prompt when it answers the user in chat. `bot_id` is the
* bot itself: the examples it reads are its own refs.
*/
export const chatReplyNote = ({ bot_id }: { bot_id: number | string }) => `# Replying in chat
You are chatting with the user. Answer their latest messages directly and briefly, in markdown. You are chatting with the user. Answer their latest messages directly and briefly, in markdown.
@@ -33,7 +43,7 @@ You are chatting with the user. Answer their latest messages directly and briefl
- To add a worker to the crew, use create_bot once the user has asked for it or agreed, and add_bot_to_workspace to place it. A bot exists only when create_bot made it: never describe one as if it did, and never write it into a workspace's context. - To add a worker to the crew, use create_bot once the user has asked for it or agreed, and add_bot_to_workspace to place it. A bot exists only when create_bot made it: never describe one as if it did, and never write it into a workspace's context.
- Never claim you did something you didn't do. - Never claim you did something you didn't do.
- The user can mention a bot as [bot:ID] and a workspace as [workspace:ID]; a [Mentions: ...] line after their message says who and what those are. Write the same refs when you name them back. - The user can mention a bot as [bot:ID] and a workspace as [workspace:ID]; a [Mentions: ...] line after their message says who and what those are. Write the same refs when you name them back.
- Documents, images and reports you make are files: save them with write_file, make_pdf or generate_image, and send one to the user with it ("attach"). When you name a file, write its [file:ID] ref, e.g. [file:12], so the user can open it. When you name a folder, write a folder ref instead, e.g. ${folderRef({ volume: "bot", owner_id: 5, path: "reports" })}: that opens the folder in Files and downloads nothing. - Documents, images and reports you make are files: save them with write_file, make_pdf or generate_image, and send one to the user with it ("attach"). When you name a file, write its [file:ID] ref, e.g. [file:12], so the user can open it. When you name a folder, write a folder ref instead, e.g. ${ownFolderRef({ bot_id })}: that opens the folder in Files and downloads nothing.
## Keep the work moving ## Keep the work moving
@@ -41,9 +51,10 @@ Be proactive. Follow where the conversation is heading and, once you've answered
/** /**
* Added to every run, chat replies and tasks alike: where its files live, * Added to every run, chat replies and tasks alike: where its files live,
* that they persist, and how to search them. * that they persist, and how to search them. `bot_id` is the bot itself,
* so the folder-ref example it reads is its own folder.
*/ */
export const BotFilesNote = `# Your files export const botFilesNote = ({ bot_id }: { bot_id: number | string }) => `# Your files
You have a folder of your own, kept with the app (not on a computer), and it stays between runs and conversations. You have a folder of your own, kept with the app (not on a computer), and it stays between runs and conversations.
@@ -53,7 +64,7 @@ You have a folder of your own, kept with the app (not on a computer), and it sta
- A picture can be set from a file: set_picture makes a file your own avatar, and update_workspace sets a workspace's. Both take the [file:ID] ref, so a logo the user sent you is usually ready to go: list_files for it, then set it. An SVG or another kind of file is turned into a PNG first. Its picture is free to change and never removes your file. - A picture can be set from a file: set_picture makes a file your own avatar, and update_workspace sets a workspace's. Both take the [file:ID] ref, so a logo the user sent you is usually ready to go: list_files for it, then set it. An SVG or another kind of file is turned into a PNG first. Its picture is free to change and never removes your file.
- A file and a folder are two different refs, and you never guess which one from the name: - A file and a folder are two different refs, and you never guess which one from the name:
- A file you saved is a [file:ID] ref (from write_file, list_files, search_files or import_file). The user sees a chip that opens the file. - A file you saved is a [file:ID] ref (from write_file, list_files, search_files or import_file). The user sees a chip that opens the file.
- A folder is a folder ref, e.g. ${folderRef({ volume: "bot", owner_id: 5, path: "reports" })}: the user sees a chip that opens that folder in Files, and nothing is downloaded. Use the volume of the folder you mean ("bot" for your own, "workspace" for a workspace's shared one). - A folder is a folder ref, e.g. ${ownFolderRef({ bot_id })}: the user sees a chip that opens that folder in Files, and nothing is downloaded. Use the volume of the folder you mean ("bot" for your own, "workspace" for a workspace's shared one).
- Never write a path as plain text and expect the user to find it; never write a [file:ID] ref for something that is a folder. - Never write a path as plain text and expect the user to find it; never write a [file:ID] ref for something that is a folder.
- A file you only described is not saved. Save it first, then say it's saved. - A file you only described is not saved. Save it first, then say it's saved.
- Save what the user will want before the long tail of a run. A step limit ends a run where it stands: write the report first with write_file (or copy it off the computer with import_file), then carry on verifying. - Save what the user will want before the long tail of a run. A step limit ends a run where it stands: write the report first with write_file (or copy it off the computer with import_file), then carry on verifying.
@@ -114,8 +125,10 @@ Every action counts here: saving memory or instructions, creating or changing a
/** /**
* Added to the coordinator's chat prompt: it hands work to the crew. * Added to the coordinator's chat prompt: it hands work to the crew.
* `bot_id` is the coordinator itself, so the ref example it copies is its
* own rather than a worker's.
*/ */
export const CoordinatorReplyNote = `As the coordinator, hand work that fits a crew member's role to them with delegate_task. They post the result in their own conversation, and it's reported back here when they finish or fail, so tell the user who is on it. When a report comes back, pass on what matters to the user, follow up on anything left, and propose what comes next: the next piece of work, which bot would take it, or a question that settles the direction. Once the user agrees, you can also grow the crew with create_bot, put a worker in a workspace with add_bot_to_workspace, and create or change a workspace with create_workspace and update_workspace. When you name a crew member, write its ref instead of its name, e.g. ${botRef({ id: 3 })}: the user sees it as the bot's name, linked to its conversation.`; export const coordinatorReplyNote = ({ bot_id }: { bot_id: number | string }) => `As the coordinator, hand work that fits a crew member's role to them with delegate_task. They post the result in their own conversation, and it's reported back here when they finish or fail, so tell the user who is on it and where: a task with no workspace_id runs in that bot's own chat, on its own memory, files and directly assigned computers; one with workspace_id runs in the workspace, on the workspace's context, files and computers as well as the bot's, and lands in that project's conversation. Never swap one for the other without saying so. When a report comes back, pass on what matters to the user, follow up on anything left, and propose what comes next: the next piece of work, which bot would take it, or a question that settles the direction. Once the user agrees, you can also grow the crew with create_bot, put a worker in a workspace with add_bot_to_workspace, and create or change a workspace with create_workspace and update_workspace. When you name a crew member, write its ref instead of its name, e.g. ${botRef({ id: bot_id })} for yourself: the user sees it as the bot's name, linked to its conversation.`;
/** /**
* Added to the coordinator's chat prompt instead while it has no workers, * Added to the coordinator's chat prompt instead while it has no workers,
src/dict/admin-sections-dict.ts
+1
View File
@@ -7,6 +7,7 @@ export const AdminSections = [
"/admin/workspaces", "/admin/workspaces",
"/admin/routines", "/admin/routines",
"/admin/files", "/admin/files",
"/admin/usage",
"/admin/logs", "/admin/logs",
"/admin/help", "/admin/help",
"/admin/settings", "/admin/settings",
src/dict/bot-dict.ts
+6
View File
@@ -110,6 +110,12 @@ export const CoordinatorBotName = "Coordinator";
*/ */
export const BotSystemPromptAppendSettingKey = "bot_system_prompt_append"; export const BotSystemPromptAppendSettingKey = "bot_system_prompt_append";
/**
* The `app_settings` key holding the install's own limit on the tool turns
* one chat reply may take. Unset means `AppData.BotReplyMaxToolSteps`.
*/
export const BotReplyMaxToolStepsSettingKey = "bot_reply_max_tool_steps";
/** /**
* A bot's face is its status. There are no badges or spinners. * A bot's face is its status. There are no badges or spinners.
*/ */
src/dict/harness-dict.ts
+6
View File
@@ -53,6 +53,12 @@ export const Harnesses = [
export type HarnessKind = (typeof Harnesses)[number]["value"]; export type HarnessKind = (typeof Harnesses)[number]["value"];
/**
* The `app_settings` key holding the install's own limit on model turns in
* one run (a task). Unset means `AppData.HarnessMaxToolSteps`.
*/
export const HarnessMaxToolStepsSettingKey = "harness_max_tool_steps";
/** /**
* Vendors preferred for automatic picks, best first. Free models from any * Vendors preferred for automatic picks, best first. Free models from any
* other vendor come after these, and only if they look competent. * other vendor come after these, and only if they look competent.
src/dict/rail-pages-dict.ts
+9 -1
View File
@@ -1,4 +1,6 @@
import { Activity, CircleHelp, Clock, FolderOpen, type LucideIcon } from "lucide-react"; import { Activity, BarChart3, CircleHelp, Clock, FolderOpen, type LucideIcon } from "lucide-react";
import type { BUN_SQLITE_COBBLE_USER_TYPES } from "@/db/types/db";
import canManageCrew from "@/src/utils/rbac/can-manage-crew";
import { HelpPath } from "./help-dict"; import { HelpPath } from "./help-dict";
export type RailPage = { export type RailPage = {
@@ -9,11 +11,16 @@ export type RailPage = {
* Shown and served to super admins only. * Shown and served to super admins only.
*/ */
super_admin_only?: boolean; super_admin_only?: boolean;
/**
* Who sees it, when that's an RBAC rule rather than super admins only.
*/
can_view?: (params: { user_types?: Pick<BUN_SQLITE_COBBLE_USER_TYPES, "user_type">[] | null }) => boolean;
}; };
export const RoutinesPath = "/admin/routines"; export const RoutinesPath = "/admin/routines";
export const FilesPath = "/admin/files"; export const FilesPath = "/admin/files";
export const ActivityPath = "/admin/logs"; export const ActivityPath = "/admin/logs";
export const UsagePath = "/admin/usage";
/** /**
* Pages opened from the rail's user popover that have their own rail: a * Pages opened from the rail's user popover that have their own rail: a
@@ -22,6 +29,7 @@ export const ActivityPath = "/admin/logs";
export const RailPages: RailPage[] = [ export const RailPages: RailPage[] = [
{ title: "Routines", url: RoutinesPath, Icon: Clock }, { title: "Routines", url: RoutinesPath, Icon: Clock },
{ title: "Files", url: FilesPath, Icon: FolderOpen }, { title: "Files", url: FilesPath, Icon: FolderOpen },
{ title: "Usage", url: UsagePath, Icon: BarChart3, can_view: canManageCrew },
{ title: "Activity", url: ActivityPath, Icon: Activity, super_admin_only: true }, { title: "Activity", url: ActivityPath, Icon: Activity, super_admin_only: true },
{ title: "Help", url: HelpPath, Icon: CircleHelp }, { title: "Help", url: HelpPath, Icon: CircleHelp },
]; ];
src/functions/backend/db/app-settings/grab-run-limits.ts
+75
View File
@@ -0,0 +1,75 @@
import { AppData } from "@/src/data/app-data";
import { BotReplyMaxToolStepsSettingKey } from "@/src/dict/bot-dict";
import { HarnessMaxToolStepsSettingKey } from "@/src/dict/harness-dict";
import readStepLimit from "@/src/utils/read-step-limit";
import grabAppSetting from "./grab-app-setting";
/**
* Model turns a task run may take: the install's own limit
* (`harness_max_tool_steps`, Settings > General), else
* `AppData.HarnessMaxToolSteps`. `runTextLoop`'s default is the code's one,
* so a task run passes this in.
*/
export async function grabHarnessMaxToolSteps() {
return readStepLimit({
saved: await grabAppSetting({ key: HarnessMaxToolStepsSettingKey }),
fallback: AppData.HarnessMaxToolSteps,
max: AppData.BotStepLimitMax,
});
}
/**
* Model turns a chat reply may take: the install's own limit
* (`bot_reply_max_tool_steps`, Settings > General), else
* `AppData.BotReplyMaxToolSteps`.
*/
export async function grabReplyMaxToolSteps() {
return readStepLimit({
saved: await grabAppSetting({ key: BotReplyMaxToolStepsSettingKey }),
fallback: AppData.BotReplyMaxToolSteps,
max: AppData.BotStepLimitMax,
});
}
/**
* Both limits with their defaults, for the settings page.
*/
export default async function grabRunLimitsOverview() {
const [task_steps, reply_steps] = await Promise.all([
grabHarnessMaxToolSteps(),
grabReplyMaxToolSteps(),
]);
const overview: RunLimitsOverview = {
task_steps,
reply_steps,
task_default: AppData.HarnessMaxToolSteps,
reply_default: AppData.BotReplyMaxToolSteps,
max: AppData.BotStepLimitMax,
};
return overview;
}
export type RunLimitsOverview = {
/**
* The limit a task run uses right now.
*/
task_steps: number;
/**
* The limit a chat reply uses right now.
*/
reply_steps: number;
/**
* What a task run uses when the install saved no limit.
*/
task_default: number;
/**
* What a chat reply uses when the install saved no limit.
*/
reply_default: number;
/**
* The most the settings page accepts.
*/
max: number;
};
src/functions/backend/db/bot-files/ingest-bot-file.ts
+79 -10
View File
@@ -7,6 +7,10 @@ import type {
import { AppData } from "@/src/data/app-data"; import { AppData } from "@/src/data/app-data";
import embedText from "@/src/harness/embeddings/embed-text"; import embedText from "@/src/harness/embeddings/embed-text";
import HarnessError from "@/src/harness/harness-error"; import HarnessError from "@/src/harness/harness-error";
import { grabHarnessUsageScope } from "@/src/harness/usage/harness-usage-scope";
import type { HarnessUsageScope } from "@/src/harness/usage/types";
import runWithFallbackScope from "@/src/harness/usage/run-with-fallback-scope";
import grabUsageOwnerUserId from "@/src/functions/backend/db/usage/grab-usage-owner";
import chunkText from "./chunk-text"; import chunkText from "./chunk-text";
import extractBotFileText from "./extract-bot-file-text"; import extractBotFileText from "./extract-bot-file-text";
@@ -72,13 +76,27 @@ export default async function ingestBotFile({
return { status: "skipped", chunks: 0 }; return { status: "skipped", chunks: 0 };
} }
/**
* A bot's own run already carries its user. The ingest queue has no run
* behind it, so everything this file costs - reading an image or an
* audio file, embedding its chunks - is entered under the owner's scope
* instead: it spends real tokens and must show on the usage page.
*/
const owner_scope: HarnessUsageScope | null = grabHarnessUsageScope()?.user_id
? null
: await resolveOwnerScope({ file });
try { try {
const text = await extractBotFileText({ const text = await runWithFallbackScope({
write_path: media?.media_write_path, scope: owner_scope,
text_content: media?.text_content, task: () =>
name: file.name, extractBotFileText({
mime_type: file.mime_type, write_path: media?.media_write_path,
read_media, text_content: media?.text_content,
name: file.name,
mime_type: file.mime_type,
read_media,
}),
}); });
await clearChunks({ file_id: file.id }); await clearChunks({ file_id: file.id });
@@ -107,11 +125,11 @@ export default async function ingestBotFile({
*/ */
for (let start = 0; start < chunks.length; start += AppData.EmbeddingBatchSize) { for (let start = 0; start < chunks.length; start += AppData.EmbeddingBatchSize) {
const batch = chunks.slice(start, start + AppData.EmbeddingBatchSize); const batch = chunks.slice(start, start + AppData.EmbeddingBatchSize);
const { vectors } = await embedText({ const { vectors } = await runEmbed({
input: batch, scope: owner_scope,
bot_id: file.bot_id!,
workspace_id: file.scope == "workspace" ? file.owner_id : null,
signal, signal,
file,
batch,
}); });
for (let i = 0; i < vectors.length; i++) { for (let i = 0; i < vectors.length; i++) {
@@ -252,3 +270,54 @@ async function markIngested({
values: [chunks > 0 ? 1 : 0, now, file_id], values: [chunks > 0 ? 1 : 0, now, file_id],
}).catch(() => {}); }).catch(() => {});
} }
/**
* One batch of a file's chunks, embedded under whatever attribution owns
* it: the run that asked for the ingest, else the file's owner.
*/
async function runEmbed({
scope,
signal,
file,
batch,
}: {
scope: HarnessUsageScope | null;
signal?: AbortSignal;
file: BUN_SQLITE_COBBLE_BOT_FILES;
batch: string[];
}) {
return await runWithFallbackScope({
scope,
task: () =>
embedText({
input: batch,
bot_id: file.bot_id!,
workspace_id: file.scope == "workspace" ? file.owner_id : null,
signal,
}),
});
}
/**
* The attribution a file the queue ingests reports under: its workspace's
* creator when it shares one, else its bot's.
*/
async function resolveOwnerScope({
file,
}: {
file: BUN_SQLITE_COBBLE_BOT_FILES;
}): Promise<HarnessUsageScope | null> {
const workspace_id = file.scope == "workspace" ? file.owner_id : null;
const user_id = await grabUsageOwnerUserId({ bot_id: file.bot_id, workspace_id });
if (!user_id) return null;
return {
user_id,
bot_id: file.scope == "bot" ? file.owner_id : file.bot_id,
workspace_id,
session_id: null,
message_id: null,
job_id: null,
};
}
src/functions/backend/db/bots-queue/complete-bot-job.ts
+12
View File
@@ -89,6 +89,18 @@ export default async function completeBotJob({
media_ids: attachments, media_ids: attachments,
}); });
if (job.id && message?.id) {
/**
* The job's model calls were written down as they happened, before
* its message existed. Now that it does, they belong to it, so the
* thread can show what the answer cost.
*/
await BunSQLite.sql({
sql: `UPDATE token_usages SET message_id = ? WHERE job_id = ? AND message_id IS NULL`,
values: [message.id, job.id],
}).catch(() => {});
}
const user_id = await grabJobUserId({ job }); const user_id = await grabJobUserId({ job });
const bot = !user_id const bot = !user_id
src/functions/backend/db/bots/grab-bot-messages.ts
+6 -1
View File
@@ -1,6 +1,7 @@
import type { BUN_SQLITE_COBBLE_BOT_MESSAGES } from "@/db/types/db"; import type { BUN_SQLITE_COBBLE_BOT_MESSAGES } from "@/db/types/db";
import type { BotMessageWithAttachments } from "@/src/types"; import type { BotMessageWithAttachments } from "@/src/types";
import BunSQLite from "@moduletrace/bun-sqlite"; import BunSQLite from "@moduletrace/bun-sqlite";
import grabMessageUsage from "../usage/grab-message-usage";
import grabMessagesMedia from "./grab-messages-media"; import grabMessagesMedia from "./grab-messages-media";
import grabMessagesReactions from "./grab-messages-reactions"; import grabMessagesReactions from "./grab-messages-reactions";
import grabMessagesReplies from "./grab-messages-replies"; import grabMessagesReplies from "./grab-messages-replies";
@@ -53,12 +54,13 @@ export default async function grabBotMessages({
const messages = res.payload || []; const messages = res.payload || [];
const message_ids = messages.map((message) => Number(message.id)); const message_ids = messages.map((message) => Number(message.id));
const [media, replies, reactions] = await Promise.all([ const [media, replies, reactions, usage] = await Promise.all([
grabMessagesMedia({ message_ids }), grabMessagesMedia({ message_ids }),
grabMessagesReplies({ message_ids }), grabMessagesReplies({ message_ids }),
user_id user_id
? grabMessagesReactions({ message_ids, user_id }) ? grabMessagesReactions({ message_ids, user_id })
: Promise.resolve(new Map()), : Promise.resolve(new Map()),
grabMessageUsage({ message_ids }),
]); ]);
/** /**
@@ -74,6 +76,8 @@ export default async function grabBotMessages({
const files = media.get(id); const files = media.get(id);
const reply_to = replies.get(id); const reply_to = replies.get(id);
const spent = usage.get(id);
return { return {
...message, ...message,
...(files ...(files
@@ -88,6 +92,7 @@ export default async function grabBotMessages({
: {}), : {}),
...(reply_to ? { reply_to } : {}), ...(reply_to ? { reply_to } : {}),
...(user_id ? { reactions: reactions.get(id) || [] } : {}), ...(user_id ? { reactions: reactions.get(id) || [] } : {}),
...(spent ? { usage: spent } : {}),
}; };
}); });
} }
src/functions/backend/db/bots/grab-bot-prompt.ts
+5 -4
View File
@@ -19,10 +19,10 @@ type Params = {
}; };
/** /**
* The full prompt for one of the user's bots (`composeBotPrompt`): its * The full prompt for one of the user's bots (`composeBotPrompt`): who it
* core instructions, the install-wide appendix, its memory, what it * is (its name and id), its core instructions, the install-wide appendix,
* remembers about this user, and in a workspace that workspace's context * its memory, what it remembers about this user, and in a workspace that
* and what the bot has for it. * workspace's context and what the bot has for it.
*/ */
export default async function grabBotPrompt({ bot_id, user_id, workspace_id }: Params) { export default async function grabBotPrompt({ bot_id, user_id, workspace_id }: Params) {
const bot = await grabUserBot({ bot_id, user_id }); const bot = await grabUserBot({ bot_id, user_id });
@@ -40,6 +40,7 @@ export default async function grabBotPrompt({ bot_id, user_id, workspace_id }: P
} }
return composeBotPrompt({ return composeBotPrompt({
bot: { id: bot.id, name: bot.name, kind: bot.kind, role: bot.role },
system_prompt: bot.system_prompt, system_prompt: bot.system_prompt,
appendix: await grabSystemPromptAppend(), appendix: await grabSystemPromptAppend(),
memory: bot.memory, memory: bot.memory,
src/functions/backend/db/usage/bucket-usage-days.test.ts
+101
View File
@@ -0,0 +1,101 @@
import { describe, expect, test } from "bun:test";
import bucketUsageDays, { localDate } from "./bucket-usage-days";
import { DayMs, startOfDay } from "./grab-usage-window";
function row(local_day: string, tokens: number) {
return {
local_day,
calls: 1,
prompt_tokens: tokens,
completion_tokens: 0,
cost: null,
};
}
const now = new Date(2026, 9, 23, 15, 0, 0).getTime();
const end = now;
const start = startOfDay({ at: now }) - 2 * DayMs;
describe("localDate", () => {
test("names the local calendar day, not the UTC one", () => {
/**
* Half an hour past local midnight is still the previous UTC day
* for anyone east of Greenwich, and the bug the chart had: the row
* was counted under one name and looked for under another.
*/
const at = new Date(2026, 9, 23, 0, 30, 0).getTime();
expect(localDate({ at })).toBe("2026-10-23");
});
test("pads a single-digit month and day", () => {
expect(localDate({ at: new Date(2026, 0, 5, 12, 0, 0).getTime() })).toBe("2026-01-05");
});
});
describe("bucketUsageDays", () => {
test("one bucket per day of the window, oldest first", () => {
const days = bucketUsageDays({ rows: [], start, end });
expect(days.length).toBe(3);
expect(days.map((day) => day.label)).toEqual(
[2, 1, 0].map((back) => localDate({ at: startOfDay({ at: now }) - back * DayMs })),
);
});
test("every row lands in its day, and yesterday's late call does not vanish", () => {
const today = localDate({ at: startOfDay({ at: now }) });
const yesterday = localDate({ at: startOfDay({ at: now }) - DayMs });
const days = bucketUsageDays({
rows: [row(today, 1500), row(yesterday, 400)],
start,
end,
});
const by_label = new Map(days.map((day) => [day.label, day.total_tokens]));
expect(by_label.get(today)).toBe(1500);
expect(by_label.get(yesterday)).toBe(400);
expect(days.reduce((total, day) => total + day.total_tokens, 0)).toBe(1900);
});
test("a day with nothing in it is still a bucket, at zero", () => {
const days = bucketUsageDays({
rows: [row(localDate({ at: startOfDay({ at: now }) }), 100)],
start,
end,
});
expect(days.filter((day) => day.total_tokens === 0).length).toBe(2);
expect(days.every((day) => day.calls === 0 || day.calls === 1)).toBe(true);
});
test("a bucket's midnight is that day's local midnight", () => {
const days = bucketUsageDays({ rows: [], start, end });
for (const day of days) {
expect(day.at).toBe(startOfDay({ at: day.at }));
expect(localDate({ at: day.at })).toBe(day.label);
}
});
test("cost and calls are summed, not counted twice", () => {
const label = localDate({ at: startOfDay({ at: now }) });
const days = bucketUsageDays({
rows: [
{ ...row(label, 100), calls: 3, cost: 0.25 },
{ ...row(label, 50), calls: 2, cost: 0.5 },
],
start,
end,
});
const bucket = days.find((day) => day.label === label);
expect(bucket?.total_tokens).toBe(150);
expect(bucket?.cost).toBe(0.75);
expect(days.reduce((total, day) => total + day.calls, 0)).toBe(5);
});
});
src/functions/backend/db/usage/bucket-usage-days.ts
+115
View File
@@ -0,0 +1,115 @@
import { DayMs, startOfDay } from "./grab-usage-window";
import { toSummary } from "./grab-usage-totals";
import type { TokenUsageSummary } from "@/src/harness/usage/types";
type Row = {
/**
* The day the call was made, `YYYY-MM-DD`, already in local time.
*/
local_day: string;
calls: number | null;
prompt_tokens: number | null;
completion_tokens: number | null;
cost: number | null;
};
type Params = {
rows: Row[];
start: number;
end: number;
};
export type UsageDay = {
/**
* The day's local midnight (epoch ms).
*/
at: number;
/**
* `YYYY-MM-DD`, in local time.
*/
label: string;
} & TokenUsageSummary;
/**
* The rows SQL grouped by local day, laid onto one bucket per day of the
* window, oldest first. Days with nothing in them are still there, at
* zero, so the bars keep their spacing, and every row lands in a bucket:
* a day outside the window would be a bug in the query's own bounds.
*/
export default function bucketUsageDays({ rows, start, end }: Params): UsageDay[] {
const by_day = new Map<string, TokenUsageSummary>();
for (const row of rows) {
/**
* Accumulated, not replaced: the query groups by day, so a repeated
* day means someone asked for two sets of rows, and the later one
* must add to the earlier rather than hide it.
*/
const label = String(row.local_day);
const summary = toSummary({ row });
by_day.set(label, mergeSummaries({ into: by_day.get(label), summary }));
}
const days: UsageDay[] = [];
const last = startOfDay({ at: end });
for (let at = startOfDay({ at: start }); at <= last; at += DayMs) {
const label = localDate({ at });
days.push({
at,
label,
...(by_day.get(label) || emptySummary()),
});
}
return days;
}
/**
* The day a local midnight falls on, in local time.
*
* Never `toISOString`: an epoch millisecond read in UTC is a different
* calendar day for anyone east or west of Greenwich, and a call counted
* under one day would then look for a bucket named after another and
* disappear from the chart (which is exactly what it did before).
*/
export function localDate({ at }: { at: number }) {
const date = new Date(at);
return [
date.getFullYear(),
String(date.getMonth() + 1).padStart(2, "0"),
String(date.getDate()).padStart(2, "0"),
].join("-");
}
function emptySummary(): TokenUsageSummary {
return { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0, calls: 0 };
}
/**
* One summary added onto another. A day that reported no cost still costs
* nothing; only a real cost is carried, so an unpriced day doesn't turn a
* priced one into zero.
*/
function mergeSummaries({
into,
summary,
}: {
into?: TokenUsageSummary;
summary: TokenUsageSummary;
}): TokenUsageSummary {
const prompt_tokens = (into?.prompt_tokens || 0) + summary.prompt_tokens;
const completion_tokens = (into?.completion_tokens || 0) + summary.completion_tokens;
const cost = (into?.cost || 0) + (summary.cost || 0);
return {
prompt_tokens,
completion_tokens,
total_tokens: prompt_tokens + completion_tokens,
...(cost > 0 ? { cost } : {}),
calls: (into?.calls || 0) + summary.calls,
};
}
src/functions/backend/db/usage/grab-message-usage.ts
+56
View File
@@ -0,0 +1,56 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import type { TokenUsageSummary } from "@/src/harness/usage/types";
type Params = {
/**
* The messages whose calls are summed. A message with no calls is
* absent from the answer.
*/
message_ids: number[];
};
/**
* What each of these messages cost, summed over every model call it took
* (chat turns, media readings, image generations). This is what a
* message's token line shows.
*/
export default async function grabMessageUsage({ message_ids }: Params) {
const totals = new Map<number, TokenUsageSummary>();
const ids = [...new Set(message_ids.map(Number).filter((id) => id > 0))];
if (!ids.length) return totals;
const res = await BunSQLite.sql<{
message_id: number;
prompt_tokens: number | null;
completion_tokens: number | null;
cost: number | null;
calls: number;
}>({
sql: `SELECT message_id,
SUM(prompt_tokens) AS prompt_tokens,
SUM(completion_tokens) AS completion_tokens,
SUM(cost) AS cost,
COUNT(*) AS calls
FROM token_usages
WHERE message_id IN (${ids.map(() => "?").join(", ")})
GROUP BY message_id`,
values: ids,
});
for (const row of res.payload || []) {
const prompt_tokens = Number(row.prompt_tokens) || 0;
const completion_tokens = Number(row.completion_tokens) || 0;
const cost = Number(row.cost);
totals.set(Number(row.message_id), {
prompt_tokens,
completion_tokens,
total_tokens: prompt_tokens + completion_tokens,
...(Number.isFinite(cost) && cost > 0 ? { cost } : {}),
calls: Number(row.calls) || 0,
});
}
return totals;
}
src/functions/backend/db/usage/grab-usage-days.ts
+42
View File
@@ -0,0 +1,42 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import bucketUsageDays, { type UsageDay } from "./bucket-usage-days";
export type { UsageDay };
type Params = {
start: number;
end: number;
};
/**
* One bucket per day in the window, oldest first, each the tokens spent
* that day. Days with nothing in them are still there, at zero, so the
* bars keep their spacing.
*
* SQLite has no timezone support, so the day is taken with its `localtime`
* modifier, which reads the same server timezone the page's own day
* boundaries come from. Grouping by the bare UTC day instead put a late
* evening's calls in the previous day's bucket, and dropped them whenever
* no bucket carried that name.
*/
export default async function grabUsageDays({ start, end }: Params): Promise<UsageDay[]> {
const res = await BunSQLite.sql<{
local_day: string;
calls: number | null;
prompt_tokens: number | null;
completion_tokens: number | null;
cost: number | null;
}>({
sql: `SELECT date(created_at / 1000, 'unixepoch', 'localtime') AS local_day,
COUNT(*) AS calls,
SUM(prompt_tokens) AS prompt_tokens,
SUM(completion_tokens) AS completion_tokens,
SUM(cost) AS cost
FROM token_usages
WHERE created_at >= ? AND created_at < ?
GROUP BY local_day`,
values: [start, end],
});
return bucketUsageDays({ rows: res.payload || [], start, end });
}
src/functions/backend/db/usage/grab-usage-models.ts
+75
View File
@@ -0,0 +1,75 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import { toSummary } from "./grab-usage-totals";
import type { TokenUsageSummary } from "@/src/harness/usage/types";
type Params = {
start: number;
end: number;
/**
* How many the page shows, busiest first.
*/
limit: number;
};
export type UsageModelRow = {
/**
* The provider's own id for the model, as the call recorded it
* (`deepseek/deepseek-chat`). This is what a person compares, not the
* catalog row's numeric id.
*/
model_slug: string;
/**
* The catalog's display name, when the model is still listed. A model
* a provider stopped listing keeps its slug and loses its name.
*/
model_name: string | null;
provider_name: string;
provider_id: number | null;
harness: string;
} & TokenUsageSummary;
/**
* Every model the period ran, busiest first (by tokens), with what it
* spent. This is how one works out what a model costs: its tokens against
* its calls, for the window asked for.
*/
export default async function grabUsageModels({ start, end, limit }: Params): Promise<UsageModelRow[]> {
const res = await BunSQLite.sql<{
model_slug: string | null;
model_name: string | null;
provider_name: string | null;
provider_id: number | null;
harness: string | null;
calls: number | null;
prompt_tokens: number | null;
completion_tokens: number | null;
cost: number | null;
}>({
sql: `SELECT u.model_slug,
m.name AS model_name,
u.provider_name,
u.provider_id,
u.harness,
COUNT(*) AS calls,
SUM(u.prompt_tokens) AS prompt_tokens,
SUM(u.completion_tokens) AS completion_tokens,
SUM(u.cost) AS cost
FROM token_usages u
LEFT JOIN harness_models m
ON m.provider_id = u.provider_id AND m.slug = u.model_slug
WHERE u.created_at >= ? AND u.created_at < ?
GROUP BY u.provider_name, u.model_slug, u.harness
ORDER BY SUM(u.prompt_tokens) + SUM(u.completion_tokens) DESC
LIMIT ?`,
values: [start, end, limit],
});
return (res.payload || []).map((row) => ({
model_slug: row.model_slug || "unknown",
model_name: row.model_name || null,
provider_name: row.provider_name || "unknown",
provider_id: row.provider_id == null ? null : Number(row.provider_id),
harness: String(row.harness || "text"),
...toSummary({ row }),
}));
}
src/functions/backend/db/usage/grab-usage-overview.ts
+53
View File
@@ -0,0 +1,53 @@
import { AppData } from "@/src/data/app-data";
import type { UsageOverview } from "@/src/types";
import grabUsageDays from "./grab-usage-days";
import grabUsageModels from "./grab-usage-models";
import grabUsageTotals from "./grab-usage-totals";
import grabUsageWindow, { DayMs } from "./grab-usage-window";
type Params = {
/**
* How many days back to total, today included.
*/
days?: number | string | null;
now?: number;
};
/**
* Everything the usage page shows for one window: the totals, the same day
* by day, per harness and per model, plus the day before the window and a
* rolling week, so the page can say whether work is going up or down.
*/
export default async function grabUsageOverview({
days,
now = Date.now(),
}: Params): Promise<UsageOverview> {
const window = grabUsageWindow({ days, now });
const previous_start = window.start - window.days * DayMs;
const [totals, days_by_day, models, previous, week] = await Promise.all([
grabUsageTotals({ start: window.start, end: window.end }),
grabUsageDays({ start: window.start, end: window.end }),
grabUsageModels({
start: window.start,
end: window.end,
limit: AppData.UsageModelPageSize,
}),
grabUsageTotals({ start: previous_start, end: window.start }),
grabUsageTotals({ start: window.end - 7 * DayMs, end: window.end }),
]);
return {
window: {
days: window.days,
start: window.start,
end: window.end,
},
totals: totals.totals,
by_harness: totals.by_harness,
days: days_by_day,
models,
previous: previous.totals,
week: week.totals,
};
}
src/functions/backend/db/usage/grab-usage-owner.ts
+47
View File
@@ -0,0 +1,47 @@
import BunSQLite from "@moduletrace/bun-sqlite";
type Params = {
/**
* The bot whose file it is, or the bot the search ran for.
*/
bot_id?: number | string | null;
/**
* The workspace whose file it is, when it shares one.
*/
workspace_id?: number | string | null;
};
/**
* Whose usage a call made outside anyone's run belongs to: a file ingested
* by the queue, or a search that ran with no conversation behind it. The
* workspace's creator when there is a workspace, else the bot's, so an
* embedding never goes uncounted.
*/
export default async function grabUsageOwnerUserId({
bot_id,
workspace_id,
}: Params): Promise<number | null> {
const workspace = Number(workspace_id);
if (Number.isFinite(workspace) && workspace > 0) {
const row = await BunSQLite.sql<{ user_id: number }>({
sql: `SELECT user_id FROM workspaces WHERE id = ? LIMIT 1`,
values: [workspace],
}).then((res) => res.payload?.[0]);
if (row?.user_id) return Number(row.user_id) || null;
}
const bot = Number(bot_id);
if (Number.isFinite(bot) && bot > 0) {
const row = await BunSQLite.sql<{ user_id: number }>({
sql: `SELECT user_id FROM bots WHERE id = ? LIMIT 1`,
values: [bot],
}).then((res) => res.payload?.[0]);
if (row?.user_id) return Number(row.user_id) || null;
}
return null;
}
src/functions/backend/db/usage/grab-usage-totals.ts
+97
View File
@@ -0,0 +1,97 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import type { TokenUsageSummary } from "@/src/harness/usage/types";
type Params = {
/**
* The window's start and end (epoch ms), from `grabUsageWindow`.
*/
start: number;
end: number;
};
type UsageRow = {
calls: number | null;
prompt_tokens: number | null;
completion_tokens: number | null;
cost: number | null;
};
/**
* One total for a whole period, and the same split by the harnesses the
* calls were for.
*/
export type UsageTotals = {
totals: TokenUsageSummary;
by_harness: (TokenUsageSummary & { harness: string })[];
};
/**
* What every model call in the period cost, in total and per harness
* (chat, media reading, transcribing, speaking, images, embeddings).
*/
export default async function grabUsageTotals({ start, end }: Params): Promise<UsageTotals> {
const res = await BunSQLite.sql<UsageRow & { harness: string | null }>({
sql: `SELECT harness,
COUNT(*) AS calls,
SUM(prompt_tokens) AS prompt_tokens,
SUM(completion_tokens) AS completion_tokens,
SUM(cost) AS cost
FROM token_usages
WHERE created_at >= ? AND created_at < ?
GROUP BY harness`,
values: [start, end],
});
const by_harness = (res.payload || []).map((row) => ({
harness: String(row.harness || "text"),
...toSummary({ row }),
}));
return {
totals: sumSummary({ summaries: by_harness }),
by_harness: by_harness.sort((a, b) => b.total_tokens - a.total_tokens),
};
}
/**
* Adds summaries into one. A call that reported no tokens still counts as
* a call.
*/
export function sumSummary({ summaries }: { summaries: TokenUsageSummary[] }): TokenUsageSummary {
let prompt_tokens = 0;
let completion_tokens = 0;
let cost = 0;
let calls = 0;
for (const summary of summaries) {
prompt_tokens += summary.prompt_tokens;
completion_tokens += summary.completion_tokens;
cost += summary.cost || 0;
calls += summary.calls;
}
return {
prompt_tokens,
completion_tokens,
total_tokens: prompt_tokens + completion_tokens,
...(cost > 0 ? { cost } : {}),
calls,
};
}
/**
* One grouped row (any shape with the four sums) as a summary.
*/
export function toSummary({ row }: { row: UsageRow }): TokenUsageSummary {
const prompt_tokens = Number(row.prompt_tokens) || 0;
const completion_tokens = Number(row.completion_tokens) || 0;
const cost = Number(row.cost);
return {
prompt_tokens,
completion_tokens,
total_tokens: prompt_tokens + completion_tokens,
...(Number.isFinite(cost) && cost > 0 ? { cost } : {}),
calls: Number(row.calls) || 0,
};
}
src/functions/backend/db/usage/grab-usage-window.ts
+57
View File
@@ -0,0 +1,57 @@
import { AppData } from "@/src/data/app-data";
type Params = {
/**
* How many days back the window starts, today included. Clamped to
* what the page offers.
*/
days?: number | string | null;
/**
* The moment "now" is, so a whole page (its totals and its days) is
* read at one instant.
*/
now?: number;
};
/**
* The period the usage page totals: a start, an end and the day buckets
* between them, oldest first, each a local midnight.
*/
export default function grabUsageWindow({ days, now = Date.now() }: Params) {
const asked = Number(days);
const count =
Number.isFinite(asked) && asked > 0
? Math.min(Math.round(asked), AppData.UsageMaxDays)
: AppData.UsageDefaultDays;
const end = now;
/**
* Whole days back from the start of today, so every bucket is a day's
* work rather than a sliding window's slice.
*/
const today = startOfDay({ at: now });
const start = today - (count - 1) * DayMs;
return {
days: count,
start,
end,
/**
* One bucket per day in the window, oldest first.
*/
buckets: Array.from({ length: count }, (_, index) => start + index * DayMs),
};
}
export const DayMs = 24 * 60 * 60 * 1000;
/**
* Local midnight of the day the moment falls in.
*/
export function startOfDay({ at }: { at: number }) {
const date = new Date(at);
date.setHours(0, 0, 0, 0);
return date.getTime();
}
src/harness/audio/speak-text.ts
+17 -2
View File
@@ -2,7 +2,9 @@ import grabHarnessSetting from "@/src/functions/backend/db/harness/grab-harness-
import grabHarnessAdapter from "../providers/grab-harness-adapter"; import grabHarnessAdapter from "../providers/grab-harness-adapter";
import resolveHarnessCandidates from "../resolve/resolve-harness-candidates"; import resolveHarnessCandidates from "../resolve/resolve-harness-candidates";
import withCandidates from "../resolve/with-candidates"; import withCandidates from "../resolve/with-candidates";
import type { HarnessCandidate } from "../types"; import type { HarnessAudioStream, HarnessCandidate } from "../types";
import buildUsageCall from "../usage/build-usage-call";
import recordHarnessUsage from "../usage/record-harness-usage";
type Params = { type Params = {
text: string; text: string;
@@ -39,9 +41,22 @@ export default async function speakText({
grabHarnessSetting({ harness: "speech", bot_id, workspace_id }), grabHarnessSetting({ harness: "speech", bot_id, workspace_id }),
]); ]);
const { result } = await withCandidates({ const { result } = await withCandidates<HarnessAudioStream>({
harness: "speech", harness: "speech",
candidates, candidates,
/**
* Speech is billed per character and streamed, so there is no
* completion count to read: the call is recorded as it is, with
* the text's length standing in for what was spoken.
*/
on_success: async ({ candidate }) =>
await recordHarnessUsage({
call: buildUsageCall({
harness: "speech",
candidate,
usage: { prompt_tokens: text.length, completion_tokens: 0 },
}),
}),
run: async (candidate) => run: async (candidate) =>
await grabHarnessAdapter({ kind: candidate.provider.kind }).speak({ await grabHarnessAdapter({ kind: candidate.provider.kind }).speak({
candidate, candidate,
src/harness/audio/transcribe-audio.ts
+15 -5
View File
@@ -4,7 +4,9 @@ import HarnessError from "../harness-error";
import grabHarnessAdapter from "../providers/grab-harness-adapter"; import grabHarnessAdapter from "../providers/grab-harness-adapter";
import resolveHarnessCandidates from "../resolve/resolve-harness-candidates"; import resolveHarnessCandidates from "../resolve/resolve-harness-candidates";
import withCandidates from "../resolve/with-candidates"; import withCandidates from "../resolve/with-candidates";
import type { HarnessAudioInput } from "../types"; import type { HarnessAudioInput, HarnessUsage } from "../types";
import buildUsageCall from "../usage/build-usage-call";
import recordHarnessUsage from "../usage/record-harness-usage";
type Params = { type Params = {
audio: HarnessAudioInput; audio: HarnessAudioInput;
@@ -46,15 +48,23 @@ export default async function transcribeAudio({
const final_signal = signal || new AbortController().signal; const final_signal = signal || new AbortController().signal;
const { result } = await withCandidates({ const { result } = await withCandidates<{ text: string; usage?: HarnessUsage }>({
harness: "transcription", harness: "transcription",
candidates, candidates,
on_success: async ({ candidate, result: transcript }): Promise<void> =>
await recordHarnessUsage({
call: buildUsageCall({
harness: "transcription",
candidate,
usage: transcript.usage,
}),
}),
run: async (candidate) => { run: async (candidate) => {
const adapter = grabHarnessAdapter({ const adapter = grabHarnessAdapter({
kind: candidate.provider.kind, kind: candidate.provider.kind,
}); });
const { text } = await adapter.transcribe({ const { text, usage } = await adapter.transcribe({
candidate, candidate,
audio, audio,
language, language,
@@ -69,9 +79,9 @@ export default async function transcribeAudio({
}); });
} }
return text.trim(); return { text: text.trim(), usage };
}, },
}); });
return result; return result.text;
} }
src/harness/embeddings/embed-text.ts
+12 -1
View File
@@ -1,6 +1,9 @@
import grabHarnessAdapter from "../providers/grab-harness-adapter"; import grabHarnessAdapter from "../providers/grab-harness-adapter";
import resolveHarnessCandidates from "../resolve/resolve-harness-candidates"; import resolveHarnessCandidates from "../resolve/resolve-harness-candidates";
import withCandidates from "../resolve/with-candidates"; import withCandidates from "../resolve/with-candidates";
import type { HarnessEmbedResult } from "../types";
import buildUsageCall from "../usage/build-usage-call";
import recordHarnessUsage from "../usage/record-harness-usage";
import fitVectorDimensions from "./fit-vector-dimensions"; import fitVectorDimensions from "./fit-vector-dimensions";
type Params = { type Params = {
@@ -52,9 +55,17 @@ export default async function embedText({
workspace_id, workspace_id,
}); });
const { result } = await withCandidates({ const { result } = await withCandidates<HarnessEmbedResult>({
harness: "embedding", harness: "embedding",
candidates, candidates,
on_success: async ({ candidate, result: embedded }): Promise<void> =>
await recordHarnessUsage({
call: buildUsageCall({
harness: "embedding",
candidate,
usage: embedded.usage,
}),
}),
run: async (candidate) => run: async (candidate) =>
await grabHarnessAdapter({ kind: candidate.provider.kind }).embed({ await grabHarnessAdapter({ kind: candidate.provider.kind }).embed({
candidate, candidate,
src/harness/image/generate-image.ts
+12 -1
View File
@@ -1,6 +1,9 @@
import grabHarnessAdapter from "../providers/grab-harness-adapter"; import grabHarnessAdapter from "../providers/grab-harness-adapter";
import resolveHarnessCandidates from "../resolve/resolve-harness-candidates"; import resolveHarnessCandidates from "../resolve/resolve-harness-candidates";
import withCandidates from "../resolve/with-candidates"; import withCandidates from "../resolve/with-candidates";
import type { HarnessImageOutput, HarnessUsage } from "../types";
import buildUsageCall from "../usage/build-usage-call";
import recordHarnessUsage from "../usage/record-harness-usage";
type Params = { type Params = {
prompt: string; prompt: string;
@@ -35,9 +38,17 @@ export default async function generateImage({
}: Params) { }: Params) {
const candidates = await resolveHarnessCandidates({ harness: "image", bot_id, workspace_id }); const candidates = await resolveHarnessCandidates({ harness: "image", bot_id, workspace_id });
const { result } = await withCandidates({ const { result } = await withCandidates<{ images: HarnessImageOutput[]; usage?: HarnessUsage }>({
harness: "image", harness: "image",
candidates, candidates,
on_success: async ({ candidate, result: made }): Promise<void> =>
await recordHarnessUsage({
call: buildUsageCall({
harness: "image",
candidate,
usage: made.usage,
}),
}),
run: async (candidate) => run: async (candidate) =>
await grabHarnessAdapter({ await grabHarnessAdapter({
kind: candidate.provider.kind, kind: candidate.provider.kind,
src/harness/media/read-image.ts
+14 -3
View File
@@ -3,7 +3,10 @@ import HarnessError from "../harness-error";
import grabHarnessAdapter from "../providers/grab-harness-adapter"; import grabHarnessAdapter from "../providers/grab-harness-adapter";
import resolveHarnessCandidates from "../resolve/resolve-harness-candidates"; import resolveHarnessCandidates from "../resolve/resolve-harness-candidates";
import withCandidates from "../resolve/with-candidates"; import withCandidates from "../resolve/with-candidates";
import type { HarnessUsage } from "../types";
import grabChatParams from "../text/grab-chat-params"; import grabChatParams from "../text/grab-chat-params";
import buildUsageCall from "../usage/build-usage-call";
import recordHarnessUsage from "../usage/record-harness-usage";
type Params = { type Params = {
/** /**
@@ -42,9 +45,17 @@ export default async function readImage({ url, focus, signal, bot_id, workspace_
workspace_id, workspace_id,
}); });
const { result } = await withCandidates({ const { result } = await withCandidates<{ text: string; usage?: HarnessUsage }>({
harness: "vision", harness: "vision",
candidates, candidates,
on_success: async ({ candidate, result: answer }): Promise<void> =>
await recordHarnessUsage({
call: buildUsageCall({
harness: "vision",
candidate,
usage: answer.usage,
}),
}),
run: async (candidate) => { run: async (candidate) => {
const answer = await grabHarnessAdapter({ const answer = await grabHarnessAdapter({
kind: candidate.provider.kind, kind: candidate.provider.kind,
@@ -83,9 +94,9 @@ export default async function readImage({ url, focus, signal, bot_id, workspace_
}); });
} }
return answer.text.trim(); return { text: answer.text.trim(), usage: answer.usage };
}, },
}); });
return result; return result.text;
} }
src/harness/providers/openai-compatible/embed.ts
+13 -1
View File
@@ -13,6 +13,11 @@ const embed: HarnessAdapter["embed"] = async ({ candidate, input, signal }) => {
const { provider, model } = candidate; const { provider, model } = candidate;
const preset = grabProviderPreset({ kind: provider.kind }); const preset = grabProviderPreset({ kind: provider.kind });
const vectors: number[][] = []; const vectors: number[][] = [];
/**
* What the provider said the batch cost. Most report it once per
* request; the rows are opened before the loop so each batch counts.
*/
let prompt_tokens = 0;
/** /**
* Every OpenAI-compatible API puts embeddings at `/embeddings`. * Every OpenAI-compatible API puts embeddings at `/embeddings`.
@@ -47,6 +52,8 @@ const embed: HarnessAdapter["embed"] = async ({ candidate, input, signal }) => {
}); });
} }
prompt_tokens += Number(json?.usage?.prompt_tokens) || 0;
for (const item of items) { for (const item of items) {
const vector = Array.isArray(item) ? item : item?.embedding; const vector = Array.isArray(item) ? item : item?.embedding;
@@ -62,7 +69,12 @@ const embed: HarnessAdapter["embed"] = async ({ candidate, input, signal }) => {
} }
} }
return { vectors }; return {
vectors,
...(prompt_tokens
? { usage: { prompt_tokens, completion_tokens: 0 } }
: {}),
};
}; };
export default embed; export default embed;
src/harness/providers/openai-compatible/generate-image.ts
+14 -1
View File
@@ -81,7 +81,20 @@ const generateImage: HarnessAdapter["generateImage"] = async ({
}); });
} }
return { images }; /**
* Image models bill per image, not per token: only a provider that
* volunteers a usage figure is recorded with one.
*/
const usage =
json?.usage && typeof json.usage == "object"
? {
prompt_tokens: Number(json.usage.prompt_tokens) || 0,
completion_tokens: Number(json.usage.completion_tokens) || 0,
...(typeof json.usage.cost == "number" ? { cost: json.usage.cost } : {}),
}
: undefined;
return { images, ...(usage ? { usage } : {}) };
}; };
export default generateImage; export default generateImage;
src/harness/providers/openai-compatible/transcribe.ts
+16 -1
View File
@@ -50,7 +50,22 @@ const transcribe: HarnessAdapter["transcribe"] = async ({
}); });
} }
return { text: String(json?.text || "").trim() }; /**
* Providers that bill by audio seconds sometimes report a token count;
* the OpenAI shape puts it under `usage`. Left out when absent, so the
* row records that the call happened without a token count.
*/
const usage =
json?.usage && typeof json.usage == "object"
? {
prompt_tokens: Number(json.usage.prompt_tokens ?? json.usage.input_tokens) || 0,
completion_tokens:
Number(json.usage.completion_tokens ?? json.usage.output_tokens) || 0,
...(typeof json.usage.cost == "number" ? { cost: json.usage.cost } : {}),
}
: undefined;
return { text: String(json?.text || "").trim(), ...(usage ? { usage } : {}) };
}; };
export default transcribe; export default transcribe;
src/harness/resolve/with-candidates.ts
+15 -1
View File
@@ -14,6 +14,15 @@ type Params<T> = {
* uses, so it doesn't switch models mid-task without a reason. * uses, so it doesn't switch models mid-task without a reason.
*/ */
prefer_key?: string; prefer_key?: string;
/**
* Called with what the candidate that worked returned, before the call
* comes back to the caller: where a harness writes its usage down.
* Never fails the call (the caller passes a recorder that can't throw).
*/
on_success?: (params: {
candidate: HarnessCandidate;
result: T;
}) => Promise<void> | void;
cooldowns?: HarnessCooldowns; cooldowns?: HarnessCooldowns;
now?: () => number; now?: () => number;
}; };
@@ -42,6 +51,7 @@ export default async function withCandidates<T>({
candidates, candidates,
run, run,
prefer_key, prefer_key,
on_success,
cooldowns = DefaultCooldowns, cooldowns = DefaultCooldowns,
now = Date.now, now = Date.now,
}: Params<T>): Promise<{ result: T; candidate: HarnessCandidate }> { }: Params<T>): Promise<{ result: T; candidate: HarnessCandidate }> {
@@ -93,7 +103,11 @@ export default async function withCandidates<T>({
if (!isReady(candidate)) continue; if (!isReady(candidate)) continue;
try { try {
return { result: await run(candidate), candidate }; const result = await run(candidate);
await on_success?.({ candidate, result });
return { result, candidate };
} catch (error: unknown) { } catch (error: unknown) {
if (!(error instanceof HarnessError)) throw error; if (!(error instanceof HarnessError)) throw error;
if (error.code == "aborted") throw error; if (error.code == "aborted") throw error;
src/harness/text/generate-text.ts
+19 -1
View File
@@ -8,6 +8,8 @@ import type {
HarnessMessage, HarnessMessage,
HarnessToolDefinition, HarnessToolDefinition,
} from "../types"; } from "../types";
import buildUsageCall from "../usage/build-usage-call";
import recordHarnessUsage from "../usage/record-harness-usage";
import grabChatParams from "./grab-chat-params"; import grabChatParams from "./grab-chat-params";
import prepareMessagesForModel from "./prepare-messages-for-model"; import prepareMessagesForModel from "./prepare-messages-for-model";
import stripThinkTags from "./strip-think-tags"; import stripThinkTags from "./strip-think-tags";
@@ -47,6 +49,14 @@ export type GenerateTextResult = HarnessChatResult & {
candidate: HarnessCandidate; candidate: HarnessCandidate;
}; };
/**
* One turn as `withCandidates` carries it: the answer the provider gave,
* with the reasoning tags stripped and an empty answer turned into null.
*/
type ChatAnswer = Omit<HarnessChatResult, "text"> & {
text: string | null;
};
/** /**
* One model turn on the text harness, falling back across its models. * One model turn on the text harness, falling back across its models.
* Media the model can't take is read by the helper harnesses first. * Media the model can't take is read by the helper harnesses first.
@@ -71,10 +81,18 @@ export default async function generateText({
? undefined ? undefined
: await grabChatParams({ harness: "text", bot_id, workspace_id }); : await grabChatParams({ harness: "text", bot_id, workspace_id });
const { result, candidate } = await withCandidates({ const { result, candidate } = await withCandidates<ChatAnswer>({
harness: "text", harness: "text",
candidates: resolved, candidates: resolved,
prefer_key, prefer_key,
on_success: async ({ candidate: answered, result: answer }): Promise<void> =>
await recordHarnessUsage({
call: buildUsageCall({
harness: "text",
candidate: answered,
usage: answer.usage,
}),
}),
run: async (candidate) => { run: async (candidate) => {
const answer = await grabHarnessAdapter({ const answer = await grabHarnessAdapter({
kind: candidate.provider.kind, kind: candidate.provider.kind,
src/harness/tools/bot/delegate-task-tool.test.ts
+72
View File
@@ -0,0 +1,72 @@
import { describe, expect, test } from "bun:test";
import type { BUN_SQLITE_COBBLE_BOTS, BUN_SQLITE_COBBLE_WORKSPACES } from "@/db/types/db";
import delegateTaskTool from "./delegate-task-tool";
const worker = { id: 5, name: "Researcher", role: "research", kind: "worker" } as BUN_SQLITE_COBBLE_BOTS;
const workspace = { id: 3, name: "Launch", archived: 0 } as BUN_SQLITE_COBBLE_WORKSPACES;
const signal = new AbortController().signal;
function coordinatorTool({
workspace_id,
workers = [worker],
workspaces = [workspace],
}: { workspace_id?: number; workers?: BUN_SQLITE_COBBLE_BOTS[]; workspaces?: BUN_SQLITE_COBBLE_WORKSPACES[] } = {}) {
return delegateTaskTool({
user_id: 1,
coordinator_id: 1,
session_id: 10,
workspace_id: workspace_id ?? null,
coordinator_name: "Coordinator",
workers,
workspaces,
});
}
/**
* The tool's own confines, kept to what needs no database: the argument it
* accepts and the refusal that keeps a workspace hand-over in its
* workspace. Every run that gets past those reaches the DB.
*/
describe("delegateTaskTool", () => {
test("takes an optional workspace the task runs in", () => {
const tool = coordinatorTool();
expect(tool.name).toBe("delegate_task");
expect(tool.schema?.safeParse({ bot_id: 5, title: "Dig", instructions: "Read the docs." }).success).toBe(true);
expect(
tool.schema?.safeParse({ bot_id: 5, title: "Dig", instructions: "Read the docs.", workspace_id: 3 }).success,
).toBe(true);
});
test("names the workspaces it can hand work into", () => {
expect(coordinatorTool().description).toContain("3: Launch");
});
test("in a workspace, says the task runs there and offers no other", () => {
const tool = coordinatorTool({ workspace_id: 3 });
expect(tool.description).toContain("runs there");
expect(tool.schema?.safeParse({ bot_id: 5, title: "Dig", instructions: "x", workspace_id: 9 }).success).toBe(
true,
);
});
test("refuses to move a hand-over out of the workspace it came from", async () => {
await expect(
coordinatorTool({ workspace_id: 3 }).run({
args: { bot_id: 5, title: "Dig", instructions: "x", workspace_id: 9 },
signal,
}),
).rejects.toThrow(/runs in that workspace/);
});
test("accepts the workspace it came from", async () => {
/** Reaches the DB and fails there for a test id, never on the guard. */
await expect(
coordinatorTool({ workspace_id: 3 }).run({
args: { bot_id: 5, title: "Dig", instructions: "x", workspace_id: 3 },
signal,
}),
).rejects.not.toThrow(/runs in that workspace/);
});
});
src/harness/tools/bot/delegate-task-tool.ts
+123 -14
View File
@@ -1,8 +1,12 @@
import { z } from "zod"; import { z } from "zod";
import type { BUN_SQLITE_COBBLE_BOTS } from "@/db/types/db"; import type { BUN_SQLITE_COBBLE_BOTS, BUN_SQLITE_COBBLE_WORKSPACES } from "@/db/types/db";
import pushBotCreated from "@/src/bots-queue/push-bot-created";
import pushBotNotice from "@/src/bots-queue/push-bot-notice"; import pushBotNotice from "@/src/bots-queue/push-bot-notice";
import enqueueBotJob from "@/src/functions/backend/db/bots-queue/enqueue-bot-job"; import enqueueBotJob from "@/src/functions/backend/db/bots-queue/enqueue-bot-job";
import grabOpenBotTask from "@/src/functions/backend/db/bots-queue/grab-open-bot-task"; import grabOpenBotTask from "@/src/functions/backend/db/bots-queue/grab-open-bot-task";
import addBotToWorkspace from "@/src/functions/backend/db/workspaces/add-bot-to-workspace";
import grabUserWorkspaces from "@/src/functions/backend/db/workspaces/grab-user-workspaces";
import grabWorkspaceBots from "@/src/functions/backend/db/workspaces/grab-workspace-bots";
import insertBotMessage from "@/src/functions/backend/db/bots/insert-bot-message"; import insertBotMessage from "@/src/functions/backend/db/bots/insert-bot-message";
import grabCrewWorkers from "@/src/functions/backend/db/bots/grab-crew-workers"; import grabCrewWorkers from "@/src/functions/backend/db/bots/grab-crew-workers";
import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers"; import grabBotComputers from "@/src/functions/backend/db/computers/grab-bot-computers";
@@ -32,19 +36,32 @@ type Params = {
* The worker bots it can hand work to (in a workspace, its members). * The worker bots it can hand work to (in a workspace, its members).
*/ */
workers: BUN_SQLITE_COBBLE_BOTS[]; workers: BUN_SQLITE_COBBLE_BOTS[];
/**
* The user's live workspaces, for the picker. In a workspace there's
* nothing to pick: the task runs there.
*/
workspaces?: BUN_SQLITE_COBBLE_WORKSPACES[];
}; };
type Args = { type Args = {
bot_id: number; bot_id: number;
title: string; title: string;
instructions: string; instructions: string;
/**
* The workspace the worker runs the task in, from the coordinator's own
* chat. Left out, the worker runs it in its own chat, on its own
* memory and its direct computers only.
*/
workspace_id?: number;
}; };
/** /**
* Lets the coordinator hand a task to a worker bot. The worker runs it on * Lets the coordinator hand a task to a worker bot: in its own chat, or in
* the computer it's on (if it has one) and posts the result in its own * one of the user's workspaces, which decides the work's context entirely.
* conversation, where a note says who asked; the result (or failure) is * The worker runs it on the computer it's on there (if it has one) and
* also reported back in the coordinator's conversation. * posts the result in that conversation, where a note says who asked; the
* result (or failure) is also reported back in the coordinator's
* conversation.
*/ */
export default function delegateTaskTool({ export default function delegateTaskTool({
user_id, user_id,
@@ -53,33 +70,79 @@ export default function delegateTaskTool({
workspace_id, workspace_id,
coordinator_name, coordinator_name,
workers, workers,
workspaces = [],
}: Params): HarnessTool<Args> { }: Params): HarnessTool<Args> {
const crew = workers const crew = workers
.map((bot) => `${bot.id}: ${bot.name}${bot.role ? ` (${bot.role})` : ""}`) .map((bot) => `${bot.id}: ${bot.name}${bot.role ? ` (${bot.role})` : ""}`)
.join("; "); .join("; ");
const places = workspaces
.map((w) => `${w.id}: ${w.name}`)
.join("; ");
return { return {
name: "delegate_task", name: "delegate_task",
description: `Hand a task to a crew member. They run it in the background and post the result in their own conversation; it's also reported back here when they finish or fail. Write the instructions as a full brief: what to do, where, and how they'll know it's done. Crew: ${crew}.`, description: `Hand a task to a crew member. They run it in the background and post the result in their own conversation; it's also reported back here when they finish or fail. Write the instructions as a full brief: what to do, where, and how they'll know it's done. Crew: ${crew}.${
workspace_id
? ` You are working in a workspace, so the task runs there: in this workspace's context.`
: ` Leave workspace_id out to run it in the crew member's own chat: only its own memory, files and directly assigned computers apply there, and the user finds the result by talking to it on its own. Add workspace_id to run it in one of the user's workspaces instead: the workspace's shared context, the member's instructions and memory there, its files and its computers (the workspace's own and the member's) all apply, and the result lands in its conversation in that workspace, beside the project it belongs to. Pick a workspace when the work belongs to that project; the member is added to it when it isn't in it yet. Workspaces: ${places || "none"}.`
}`,
schema: z.object({ schema: z.object({
bot_id: z.number().int(), bot_id: z.number().int(),
title: z.string().min(1).max(80), title: z.string().min(1).max(80),
instructions: z.string().min(1).max(8000), instructions: z.string().min(1).max(8000),
workspace_id: z
.number()
.int()
.optional()
.describe(
workspace_id
? "Not available here: a task handed over in a workspace runs in that workspace."
: "Run the task in this workspace: its shared context, memory, files and computers apply. Leave it out to run it in the crew member's own chat.",
),
}), }),
run: async ({ args }) => { run: async ({ args }) => {
const fresh = await grabCrewWorkers({ workspace_id }); /**
* In a workspace the task can't leave it: the run is already
* there, and that is the whole point of the hand-over.
*/
if (workspace_id && args.workspace_id && args.workspace_id != Number(workspace_id)) {
throw new Error(
`A task handed over from a workspace runs in that workspace. Leave workspace_id out, or hand it over from your own chat.`,
);
}
const target_workspace_id = workspace_id
? Number(workspace_id)
: args.workspace_id
? Number(args.workspace_id)
: null;
const fresh = await grabCrewWorkers({ workspace_id: target_workspace_id });
const worker = fresh.find((bot) => bot.id == args.bot_id); const worker = fresh.find((bot) => bot.id == args.bot_id);
if (!worker?.id) { if (!worker?.id) {
throw new Error( throw new Error(
`No crew member has id ${args.bot_id}. Crew: ${fresh `No crew member has id ${args.bot_id}${
.map((bot) => `${bot.id}: ${bot.name}`) target_workspace_id ? ` in workspace ${target_workspace_id}` : ""
.join("; ") || "none"}.`, }. Crew: ${
fresh.map((bot) => `${bot.id}: ${bot.name}`).join("; ") || "none"
}.`,
); );
} }
const computers = await grabBotComputers({ bot_id: worker.id, workspace_id }); const joined = target_workspace_id
const worker_session = await ensureBotSession({ user_id, bot_id: worker.id, workspace_id }); ? await ensureWorkerInWorkspace({ user_id, worker, target_workspace_id })
: "";
const computers = await grabBotComputers({
bot_id: worker.id,
workspace_id: target_workspace_id,
});
const worker_session = await ensureBotSession({
user_id,
bot_id: worker.id,
workspace_id: target_workspace_id,
});
const open = await grabOpenBotTask({ const open = await grabOpenBotTask({
bot_id: worker.id, bot_id: worker.id,
session_id: Number(worker_session.id), session_id: Number(worker_session.id),
@@ -108,12 +171,58 @@ export default function delegateTaskTool({
message: await insertBotMessage({ message: await insertBotMessage({
bot_id: worker.id, bot_id: worker.id,
session_id: Number(worker_session.id), session_id: Number(worker_session.id),
workspace_id: workspace_id ? Number(workspace_id) : null, workspace_id: target_workspace_id,
text: `${coordinator_name} asked me to: **${args.title}**`, text: `${coordinator_name} asked me to: **${args.title}**`,
}), }),
}); });
return `Task #${job_id} is with ${worker.name}. Their result will be in their conversation and reported back here.`; return `Task #${job_id} is with ${worker.name}${
target_workspace_id ? ` in workspace ${target_workspace_id}` : " in their own chat"
}.${joined} Their result will be in that conversation and reported back here.`;
}, },
}; };
} }
type EnsureParams = {
user_id: number | string;
worker: BUN_SQLITE_COBBLE_BOTS;
target_workspace_id: number;
};
/**
* A worker can only run a task in a workspace it's in, and a workspace's
* members were already listed by `grabCrewWorkers`. Anyone else is added
* first, since delegation is exactly the moment the broker decides who
* does the work where.
*/
async function ensureWorkerInWorkspace({
user_id,
worker,
target_workspace_id,
}: EnsureParams) {
if (!worker.id) return "";
const workspaces = (await grabUserWorkspaces({ user_id })).filter((w) => !w.archived);
const workspace = workspaces.find((w) => w.id == target_workspace_id);
if (!workspace) {
throw new Error(
`No workspace has id ${target_workspace_id}. Workspaces: ${
workspaces.map((w) => `${w.id}: ${w.name}`).join("; ") || "none"
}. Hand the task over without workspace_id to run it in the crew member's own chat.`,
);
}
const members = await grabWorkspaceBots({ user_id, workspace_id: target_workspace_id });
if (members.some((bot) => bot.id == worker.id)) return "";
await addBotToWorkspace({
user_id,
workspace_id: target_workspace_id,
bot_id: Number(worker.id),
});
await pushBotCreated({ bot: worker });
return ` ${worker.name} wasn't in "${workspace.name}", so it was added to it first.`;
}
src/harness/tools/bot/grab-bot-chat-tools.ts
+4 -1
View File
@@ -61,7 +61,9 @@ type Params = {
* for the coordinator (the super admin's alone) create_workspace and * for the coordinator (the super admin's alone) create_workspace and
* update_workspace (a workspace and its shared context), create_bot to * update_workspace (a workspace and its shared context), create_bot to
* grow the crew, delegate_task to every worker (the workspace's in a * grow the crew, delegate_task to every worker (the workspace's in a
* workspace) plus, in its own chat, find_bot and find_workspace, * workspace, where the task runs; from its own chat, in the worker's own
* chat or in a workspace it names) plus, in its own chat, find_bot and
* find_workspace,
* add_bot_to_workspace and remove_bot_from_workspace, assign_computer, * add_bot_to_workspace and remove_bot_from_workspace, assign_computer,
* update_bot, delete_bot and delete_workspace (which workspace, which * update_bot, delete_bot and delete_workspace (which workspace, which
* computers, and which name, role and core instructions each crew member * computers, and which name, role and core instructions each crew member
@@ -150,6 +152,7 @@ export default async function grabBotChatTools({
workspace_id, workspace_id,
coordinator_name: bot.name || "The coordinator", coordinator_name: bot.name || "The coordinator",
workers, workers,
workspaces,
}), }),
); );
src/harness/tools/bot/tasks/create-task-tool.ts
+2 -2
View File
@@ -101,7 +101,7 @@ export default function createTaskTool({
name: "create_task", name: "create_task",
description: is_coordinator description: is_coordinator
? `Add a task to a list: a worker's own (${crew || "no workers yet"}), a workspace's shared one (${places || "no workspaces yet"}), or your own (leave both ids out). Write the description as a brief: what to do, where, and how you'll know it's done. A workspace's tasks are visible to every bot working there.` ? `Add a task to a list: a worker's own (${crew || "no workers yet"}), a workspace's shared one (${places || "no workspaces yet"}), or your own (leave both ids out). Write the description as a brief: what to do, where, and how you'll know it's done. A workspace's tasks are visible to every bot working there.`
: `Add a task to a list: ${workspace_id ? "this workspace's shared one (leave the ids out), or your own (bot_id: " + bot_id + ")" : "your own (leave the ids out)"}. Write the description as a brief: what to do, where, and how you'll know it's done.`, : `Add a task to a list: ${workspace_id ? `this workspace's shared one (leave the ids out), or your own (bot_id: ${bot_id})` : `your own, bot_id ${bot_id} (or leave the id out, since it defaults to yours)`}. Write the description as a brief: what to do, where, and how you'll know it's done.`,
schema: z.object({ schema: z.object({
title: z.string().trim().min(1).max(AppData.TaskTitleMaxLength).describe(`A short label`), title: z.string().trim().min(1).max(AppData.TaskTitleMaxLength).describe(`A short label`),
description: z description: z
@@ -111,7 +111,7 @@ export default function createTaskTool({
bot_id: z.number().int().optional().describe( bot_id: z.number().int().optional().describe(
is_coordinator is_coordinator
? `Whose own list, when it isn't a workspace's. Leave out for your own.` ? `Whose own list, when it isn't a workspace's. Leave out for your own.`
: `Your own id (${bot_id}), to keep it in your list rather than the workspace's.`, : `Your own id (${bot_id}), to keep it in your list rather than the workspace's; leaving it out has the same effect here.`,
), ),
workspace_id: z workspace_id: z
.number() .number()
src/harness/types.ts
+25 -2
View File
@@ -138,6 +138,18 @@ export type HarnessUsage = {
cost?: number; cost?: number;
}; };
/**
* What a harness does with a model, for its usage rows and the totals
* that read them. Mirrors the `harness` column of `token_usages`.
*/
export type HarnessUsageKind =
| "text"
| "vision"
| "transcription"
| "speech"
| "image"
| "embedding";
export type HarnessChatParams = { export type HarnessChatParams = {
candidate: HarnessCandidate; candidate: HarnessCandidate;
messages: HarnessMessage[]; messages: HarnessMessage[];
@@ -238,11 +250,22 @@ export type HarnessEmbedResult = {
*/ */
export type HarnessAdapter = { export type HarnessAdapter = {
chat: (params: HarnessChatParams) => Promise<HarnessChatResult>; chat: (params: HarnessChatParams) => Promise<HarnessChatResult>;
transcribe: (params: HarnessTranscribeParams) => Promise<{ text: string }>; /**
* `usage` when the provider says how many tokens it heard, which
* OpenAI-style speech-to-text endpoints do only sometimes.
*/
transcribe: (params: HarnessTranscribeParams) => Promise<{
text: string;
usage?: HarnessUsage;
}>;
speak: (params: HarnessSpeakParams) => Promise<HarnessAudioStream>; speak: (params: HarnessSpeakParams) => Promise<HarnessAudioStream>;
/**
* Image models report what they made rather than how many tokens it
* took, so `usage` is only whatever the provider volunteers.
*/
generateImage: ( generateImage: (
params: HarnessImageParams, params: HarnessImageParams,
) => Promise<{ images: HarnessImageOutput[] }>; ) => Promise<{ images: HarnessImageOutput[]; usage?: HarnessUsage }>;
embed: (params: HarnessEmbedParams) => Promise<HarnessEmbedResult>; embed: (params: HarnessEmbedParams) => Promise<HarnessEmbedResult>;
listModels: (params: { listModels: (params: {
provider: HarnessProviderConfig; provider: HarnessProviderConfig;
src/harness/usage/build-usage-call.test.ts
+102
View File
@@ -0,0 +1,102 @@
import { describe, expect, test } from "bun:test";
import type { HarnessCandidate } from "../types";
import buildUsageCall from "./build-usage-call";
function model(overrides: { row_id?: number } = {}) {
return {
slug: "deepseek/deepseek-chat",
name: "DeepSeek Chat",
input_modalities: ["text"],
output_modalities: ["text"],
supports_tools: true,
...overrides,
} as HarnessCandidate["model"];
}
function candidate(overrides: Partial<HarnessCandidate> = {}): HarnessCandidate {
return {
provider: {
id: 2,
name: "OpenRouter",
kind: "openrouter",
} as HarnessCandidate["provider"],
key: "openrouter:deepseek/deepseek-chat",
source: "auto",
model: model({ row_id: 41 }),
...overrides,
};
}
describe("buildUsageCall", () => {
test("names the candidate that really answered and what it spent", () => {
const call = buildUsageCall({
harness: "text",
candidate: candidate(),
usage: { prompt_tokens: 1200, completion_tokens: 340, cost: 0.0042 },
});
expect(call).toEqual({
harness: "text",
provider_id: 2,
provider_name: "OpenRouter",
model_id: 41,
model_slug: "deepseek/deepseek-chat",
prompt_tokens: 1200,
completion_tokens: 340,
cost: 0.0042,
});
});
test("a provider with no name reads as its kind", () => {
const call = buildUsageCall({
harness: "image",
candidate: candidate({
provider: { id: 4, name: "", kind: "openai" } as HarnessCandidate["provider"],
}),
usage: { prompt_tokens: 0, completion_tokens: 0 },
});
expect(call.provider_name).toBe("openai");
});
test("a provider that reports no usage still gets a row", () => {
const call = buildUsageCall({ harness: "transcription", candidate: candidate() });
expect(call.prompt_tokens).toBe(0);
expect(call.completion_tokens).toBe(0);
expect(call.cost).toBeUndefined();
});
test("a candidate with no catalog row has no model id to point at", () => {
const call = buildUsageCall({
harness: "text",
candidate: candidate({
model: model({ slug: "made-up-model" } as any),
}),
usage: { prompt_tokens: 10, completion_tokens: 5 },
});
expect(call.model_id).toBeUndefined();
expect(call.model_slug).toBe("made-up-model");
});
test("a cost the provider did not report is left out, not zeroed", () => {
const call = buildUsageCall({
harness: "text",
candidate: candidate(),
usage: { prompt_tokens: 10, completion_tokens: 5, cost: Number.NaN },
});
expect(call.cost).toBeUndefined();
});
test("a free provider's zero cost is kept as zero", () => {
const call = buildUsageCall({
harness: "text",
candidate: candidate(),
usage: { prompt_tokens: 10, completion_tokens: 5, cost: 0 },
});
expect(call.cost).toBe(0);
});
});
src/harness/usage/build-usage-call.ts
+37
View File
@@ -0,0 +1,37 @@
import type { HarnessCandidate, HarnessUsage } from "../types";
import type { HarnessUsageCall } from "./types";
type Params = {
harness: HarnessUsageCall["harness"];
candidate: HarnessCandidate;
usage?: HarnessUsage;
};
/**
* The candidate that really answered a call, and what it reported, as the
* row `recordHarnessUsage` writes. Providers that report no usage at all
* still get a row (no tokens), so a call is never invisible.
*/
export default function buildUsageCall({ harness, candidate, usage }: Params): HarnessUsageCall {
return {
harness,
provider_id: candidate.provider.id,
provider_name: candidate.provider.name || candidate.provider.kind,
model_id: modelIdOf(candidate),
model_slug: candidate.model.slug,
prompt_tokens: Number(usage?.prompt_tokens) || 0,
completion_tokens: Number(usage?.completion_tokens) || 0,
cost: typeof usage?.cost == "number" && Number.isFinite(usage.cost) ? usage.cost : undefined,
};
}
/**
* The `harness_models` row id, when the candidate carries the catalog row
* it was resolved from (`withCandidates` doesn't drop it; a candidate made
* by hand may not have one).
*/
function modelIdOf(candidate: HarnessCandidate) {
const row_id = (candidate.model as { row_id?: number }).row_id;
return typeof row_id == "number" ? row_id : undefined;
}
src/harness/usage/harness-usage-scope.test.ts
+106
View File
@@ -0,0 +1,106 @@
import { describe, expect, test } from "bun:test";
import { grabHarnessUsageScope, runWithHarnessUsageScope } from "./harness-usage-scope";
describe("harness usage scope", () => {
test("nothing is attributed outside a run", () => {
expect(grabHarnessUsageScope()).toBeUndefined();
});
test("a run's scope is visible to everything it calls", async () => {
const scope = await runWithHarnessUsageScope({
scope: { user_id: 7, bot_id: 3, session_id: 11 },
task: async () => grabHarnessUsageScope(),
});
expect(scope).toEqual({ user_id: 7, bot_id: 3, session_id: 11 });
});
test("a nested run keeps what the outer one knew", async () => {
const scope = await runWithHarnessUsageScope({
scope: { user_id: 7, bot_id: 3, session_id: 11, job_id: 5 },
task: () =>
runWithHarnessUsageScope({
scope: { message_id: 42 },
task: async () => grabHarnessUsageScope(),
}),
});
expect(scope).toEqual({
user_id: 7,
bot_id: 3,
session_id: 11,
job_id: 5,
message_id: 42,
});
});
test("a nested run overrides only what it sets", async () => {
const scope = await runWithHarnessUsageScope({
scope: { user_id: 7, bot_id: 3, workspace_id: 9 },
task: () =>
runWithHarnessUsageScope({
scope: { bot_id: 4 },
task: async () => grabHarnessUsageScope(),
}),
});
expect(scope).toEqual({ user_id: 7, bot_id: 4, workspace_id: 9 });
});
test("an undefined field leaves the outer value alone", async () => {
const scope = await runWithHarnessUsageScope({
scope: { user_id: 7, bot_id: 3, job_id: 5 },
task: () =>
runWithHarnessUsageScope({
scope: { job_id: undefined, message_id: 42 },
task: async () => grabHarnessUsageScope(),
}),
});
expect(scope).toEqual({ user_id: 7, bot_id: 3, job_id: 5, message_id: 42 });
});
test("a null field clears the outer value, since it means 'none'", async () => {
const scope = await runWithHarnessUsageScope({
scope: { user_id: 7, bot_id: 3, workspace_id: 9, job_id: 5 },
task: () =>
runWithHarnessUsageScope({
scope: { workspace_id: null, job_id: null },
task: async () => grabHarnessUsageScope(),
}),
});
expect(scope).toEqual({ user_id: 7, bot_id: 3, workspace_id: null, job_id: null });
});
test("the scope is gone once the run returns", async () => {
await runWithHarnessUsageScope({
scope: { user_id: 7 },
task: async () => undefined,
});
expect(grabHarnessUsageScope()).toBeUndefined();
});
test("two concurrent runs never see each other's scope", async () => {
const [first, second] = await Promise.all([
runWithHarnessUsageScope({
scope: { user_id: 1 },
task: async () => {
await Bun.sleep(5);
return grabHarnessUsageScope();
},
}),
runWithHarnessUsageScope({
scope: { user_id: 2 },
task: async () => {
await Bun.sleep(1);
return grabHarnessUsageScope();
},
}),
]);
expect(first).toEqual({ user_id: 1 });
expect(second).toEqual({ user_id: 2 });
});
});
src/harness/usage/harness-usage-scope.ts
+63
View File
@@ -0,0 +1,63 @@
import { AsyncLocalStorage } from "node:async_hooks";
import type { HarnessUsageScope } from "./types";
/**
* The attribution a run's model calls carry.
*
* A bot's run is entered once (`runWithHarnessUsageScope`) and everything
* it calls underneath - the text loop's turns, a session summary, a
* transcription, an image, an embedding - is written down for that user,
* bot, session and job without threading the ids through every harness
* function. One of those calls is itself entered by a `run` that has to
* swap the scope it reports under (`withHarnessUsageScope`): the session
* namer runs outside the job's scope, on purpose, since the job it named
* is already complete.
*
* The store is merged, not replaced: a call that only knows the message id
* (the text loop attaching a turn's usage) keeps the run's user, bot and
* job.
*/
const storage = new AsyncLocalStorage<HarnessUsageScope>();
/**
* Runs `task` with `scope` as its attribution. Ambient values are merged
* under it, so a nested call keeps what the outer one set and overrides
* only what it knows better (null clears a field).
*/
export async function runWithHarnessUsageScope<T>(
{ scope, task }: { scope: HarnessUsageScope; task: () => Promise<T> },
): Promise<T> {
return await storage.run(mergeScopes({ base: storage.getStore(), overlay: scope }), task);
}
/**
* The attribution a call right now carries, if any.
*/
export function grabHarnessUsageScope(): HarnessUsageScope | undefined {
return storage.getStore();
}
function mergeScopes({
base,
overlay,
}: {
base?: HarnessUsageScope;
overlay: HarnessUsageScope;
}): HarnessUsageScope {
const merged: HarnessUsageScope = { ...base };
for (const [key, value] of Object.entries(overlay) as [
keyof HarnessUsageScope,
HarnessUsageScope[keyof HarnessUsageScope],
][]) {
if (value === undefined) continue;
/**
* A null is a real value here: it says "none", e.g. a bot's own
* chat has no workspace, and must clear what the outer scope had.
*/
(merged as Record<string, unknown>)[key] = value;
}
return merged;
}
src/harness/usage/record-harness-usage.ts
+92
View File
@@ -0,0 +1,92 @@
import BunSQLite from "@moduletrace/bun-sqlite";
import type {
BUN_SQLITE_COBBLE_TOKEN_USAGES,
BunSQLiteTables,
} from "@/db/types/db";
import { logError } from "@/src/functions/backend/logger";
import { grabHarnessUsageScope } from "./harness-usage-scope";
import type { HarnessUsageCall, HarnessUsageScope } from "./types";
type Params = {
/**
* The scope the call ran under. Left out, the ambient one
* (`grabHarnessUsageScope`) is used, which is what every entry point
* does: the ids came in through `runWithHarnessUsageScope`.
*/
scope?: HarnessUsageScope;
/**
* Which harness and which model answered, with what the provider
* reported it cost.
*/
call: HarnessUsageCall;
};
/**
* Writes one model call into `token_usages`: the tokens it spent, on which
* model and provider, for whom, and which message or job it was for. This
* is what a message's token line, a bot's and a provider's totals and the
* usage page all read.
*
* Never throws: a model that answered must not have its answer lost
* because the bookkeeping failed. A call without an owner (no scope, no
* user) is not written at all, since nothing could show it.
*/
export default async function recordHarnessUsage({ scope, call }: Params) {
const resolved = scope || grabHarnessUsageScope();
const user_id = Number(resolved?.user_id) || null;
if (!user_id) return;
const row: BUN_SQLITE_COBBLE_TOKEN_USAGES = {
user_id,
bot_id: numberOrNull(resolved?.bot_id),
workspace_id: numberOrNull(resolved?.workspace_id),
session_id: numberOrNull(resolved?.session_id),
message_id: numberOrNull(resolved?.message_id),
job_id: numberOrNull(resolved?.job_id),
harness: call.harness,
provider_id: call.provider_id,
provider_name: call.provider_name,
model_id: call.model_id,
model_slug: call.model_slug,
prompt_tokens: Math.max(0, Math.round(call.prompt_tokens || 0)),
completion_tokens: Math.max(0, Math.round(call.completion_tokens || 0)),
cost: typeof call.cost == "number" && Number.isFinite(call.cost) ? call.cost : undefined,
};
try {
const res = await BunSQLite.insert<
BUN_SQLITE_COBBLE_TOKEN_USAGES,
(typeof BunSQLiteTables)[number]
>({
table: "token_usages",
data: [row],
});
if (!res.success) {
throw new Error(res.msg || `The usage wasn't saved.`);
}
} catch (error: unknown) {
await logError({
title: "Model usage not recorded",
error,
metadata: {
harness: call.harness,
model_slug: call.model_slug || null,
provider_name: call.provider_name || null,
user_id,
},
}).catch(() => {});
}
}
/**
* SQLite stores NULL for what isn't given; `Number(undefined)` is NaN,
* which bun-sqlite would refuse. An absent id is `undefined`, not `null`:
* bun-sqlite's row types spell "no value" that way.
*/
function numberOrNull(value: unknown): number | undefined {
const number = Number(value);
return Number.isFinite(number) && number > 0 ? number : undefined;
}
src/harness/usage/run-with-fallback-scope.test.ts
+81
View File
@@ -0,0 +1,81 @@
import { describe, expect, test } from "bun:test";
import { grabHarnessUsageScope, runWithHarnessUsageScope } from "./harness-usage-scope";
import runWithFallbackScope from "./run-with-fallback-scope";
const owner = { user_id: 9, bot_id: 3, session_id: null, message_id: null, job_id: null };
describe("runWithFallbackScope", () => {
test("the enclosing run keeps the work it already owns", async () => {
const seen = await runWithHarnessUsageScope({
scope: { user_id: 7, bot_id: 3, job_id: 5 },
task: () =>
runWithFallbackScope({
scope: owner,
task: async () => grabHarnessUsageScope(),
}),
});
expect(seen).toEqual({ user_id: 7, bot_id: 3, job_id: 5 });
});
test("work with no run is attributed to its owner", async () => {
const seen = await runWithFallbackScope({
scope: owner,
task: async () => grabHarnessUsageScope(),
});
expect(seen).toEqual(owner);
});
test("nobody to charge leaves the work unattributed rather than failing", async () => {
const seen = await runWithFallbackScope({
scope: null,
task: async () => grabHarnessUsageScope(),
});
expect(seen).toBeUndefined();
});
test("a scope with no user is not an owner", async () => {
const seen = await runWithFallbackScope({
scope: { ...owner, user_id: undefined },
task: async () => grabHarnessUsageScope(),
});
expect(seen).toBeUndefined();
});
test("work that must be charged on its own overrides the enclosing run", async () => {
const seen = await runWithHarnessUsageScope({
scope: { user_id: 7, bot_id: 3, job_id: 5 },
task: () =>
runWithFallbackScope({
scope: owner,
mode: "own",
task: async () => grabHarnessUsageScope(),
}),
});
expect(seen).toEqual(owner);
});
test("the work's result is passed through", async () => {
const result = await runWithFallbackScope({
scope: owner,
task: async () => "the answer",
});
expect(result).toBe("the answer");
});
test("a failure inside the work is not swallowed", async () => {
expect(
runWithFallbackScope({
scope: owner,
task: async () => {
throw new Error("the provider refused");
},
}),
).rejects.toThrow("the provider refused");
});
});
src/harness/usage/run-with-fallback-scope.ts
+41
View File
@@ -0,0 +1,41 @@
import { grabHarnessUsageScope, runWithHarnessUsageScope } from "./harness-usage-scope";
import type { HarnessUsageScope } from "./types";
type Params<T> = {
/**
* What to attribute the work to when nothing already owns it: a file
* ingested by the queue has no run behind it, but its owner must still
* see what the embedding cost. Null when there is nobody to charge.
*/
scope: HarnessUsageScope | null;
task: () => Promise<T>;
/**
* How the work is attributed: `enclosing` keeps whatever run it is in,
* `own` always reports under `scope` (the session namer, whose job is
* already complete and must not be charged to it).
*/
mode?: "enclosing" | "own";
};
/**
* Runs work under an attribution the run may not have. Nothing is entered
* when a run is already in charge, so an ingest a bot asked for is charged
* to that bot's user and not to the file's owner.
*/
export default async function runWithFallbackScope<T>({
scope,
task,
mode = "enclosing",
}: Params<T>): Promise<T> {
if (mode == "enclosing" && grabHarnessUsageScope()?.user_id) {
return await task();
}
const user_id = Number(scope?.user_id);
if (!Number.isFinite(user_id) || user_id <= 0 || user_id === null) {
return await task();
}
return await runWithHarnessUsageScope({ scope: { ...scope, user_id }, task });
}
src/harness/usage/types.ts
+60
View File
@@ -0,0 +1,60 @@
/**
* Who and what a model call was for. Every harness call that reports its
* usage is written down with this (`/api/auth/crud` no: `token_usages`),
* so a message, a session, a bot or a provider can be totaled later.
*
* It is deliberately shaped slightly differently from the table's
* columns: `user_id` is required here (a row without an owner can't be
* shown or totaled for anyone), the rest are optional.
*/
export type HarnessUsageScope = {
/**
* Whose work it was: the user whose conversation the call ran for.
*/
user_id?: number | string;
bot_id?: number | string | null;
workspace_id?: number | string | null;
session_id?: number | string | null;
/**
* The queued job it ran for, when it ran for one.
*/
job_id?: number | string | null;
/**
* The message it produced, for the ones that produce one.
*/
message_id?: number | string | null;
};
/**
* What the caller knows about the call beyond its scope: which harness
* asked, and which candidate answered. Both are filled in by the call's
* own entry point, never by the code that runs it.
*/
export type HarnessUsageCall = {
harness: "text" | "vision" | "transcription" | "speech" | "image" | "embedding";
provider_id?: number;
provider_name?: string;
model_id?: number;
model_slug?: string;
prompt_tokens: number;
completion_tokens: number;
cost?: number;
};
/**
* A message's usage, as its thread shows it and as `token_usages` stores
* it: what one answer cost, summed over its model calls.
*/
export type TokenUsageSummary = {
prompt_tokens: number;
completion_tokens: number;
total_tokens: number;
/**
* In USD, when the provider reported it.
*/
cost?: number;
/**
* How many model calls it took.
*/
calls: number;
};
src/harness/usage/with-usage-scope.ts
+19
View File
@@ -0,0 +1,19 @@
import { runWithHarnessUsageScope } from "./harness-usage-scope";
import type { HarnessUsageScope } from "./types";
/**
* Runs one call the app makes on a user's behalf inside the scope its
* usage is written down with (transcribing a dictation, speaking a
* message, previewing a voice). The bot's run is already inside a scope of
* its own, so this only adds what the run didn't know: no job, no message,
* and the session the call belongs to.
*/
export default async function withUsageScope<T>({
scope,
task,
}: {
scope: HarnessUsageScope;
task: () => Promise<T>;
}): Promise<T> {
return await runWithHarnessUsageScope({ scope, task });
}
src/layouts/bots/(dict)/rail-user-menu-dict.ts
+8 -3
View File
@@ -1,4 +1,5 @@
import { LogOut, SlidersHorizontal, type LucideIcon } from "lucide-react"; import { LogOut, SlidersHorizontal, type LucideIcon } from "lucide-react";
import type { BUN_SQLITE_COBBLE_USER_TYPES } from "@/db/types/db";
import { SettingsDefaultPath } from "@/src/dict/settings-pages-dict"; import { SettingsDefaultPath } from "@/src/dict/settings-pages-dict";
import { RailPages } from "@/src/dict/rail-pages-dict"; import { RailPages } from "@/src/dict/rail-pages-dict";
@@ -18,12 +19,16 @@ export type RailUserMenuItem = {
* Shown to super admins only. * Shown to super admins only.
*/ */
super_admin_only?: boolean; super_admin_only?: boolean;
/**
* Who sees it, when that's an RBAC rule rather than super admins only.
*/
can_view?: (params: { user_types?: Pick<BUN_SQLITE_COBBLE_USER_TYPES, "user_type">[] | null }) => boolean;
}; };
/** /**
* The popover items: the rail pages (routines, activity, help), settings, * The popover items: the rail pages (routines, files, usage, activity,
* sign out. Nothing else lives in the rail: these destinations are only * help), settings, sign out. Nothing else lives in the rail: these
* reachable from here. * destinations are only reachable from here.
*/ */
export const RailUserMenuItems: RailUserMenuItem[] = [ export const RailUserMenuItems: RailUserMenuItem[] = [
...RailPages, ...RailPages,
src/layouts/bots/(partials)/rail-pages-list.tsx
+10 -4
View File
@@ -2,6 +2,7 @@ import { useContext } from "react";
import Divider from "@moduletrace/twui/layout/Divider"; import Divider from "@moduletrace/twui/layout/Divider";
import { RailPages } from "@/src/dict/rail-pages-dict"; import { RailPages } from "@/src/dict/rail-pages-dict";
import { AppContext } from "@/src/pages/__root"; import { AppContext } from "@/src/pages/__root";
import canSeeRailItem from "@/src/utils/rbac/can-see-rail-item";
import isPassThroughUser from "@/src/utils/rbac/is-pass-through-user"; import isPassThroughUser from "@/src/utils/rbac/is-pass-through-user";
import RailBackLink from "./rail-back-link"; import RailBackLink from "./rail-back-link";
import RailSettingsLink from "./rail-settings-link"; import RailSettingsLink from "./rail-settings-link";
@@ -11,8 +12,9 @@ type Props = {
}; };
/** /**
* The rail on routines, activity and help: back to the bots, then those * The rail on routines, usage, activity and help: back to the bots, then
* pages (super admin ones for super admins only). * those pages (super admin ones for super admins only, crew managers for
* usage).
*/ */
export default function RailPagesList({ pathname }: Props) { export default function RailPagesList({ pathname }: Props) {
const { pageProps } = useContext(AppContext); const { pageProps } = useContext(AppContext);
@@ -25,8 +27,12 @@ export default function RailPagesList({ pathname }: Props) {
<> <>
<RailBackLink /> <RailBackLink />
{/* <Divider className="shrink-0 mx-1 @rail:mx-3 my-2 w-auto border-hairline dark:border-hairline" /> */} {/* <Divider className="shrink-0 mx-1 @rail:mx-3 my-2 w-auto border-hairline dark:border-hairline" /> */}
{RailPages.filter( {RailPages.filter((page) =>
(page) => is_super_admin || !page.super_admin_only, canSeeRailItem({
item: page,
is_super_admin,
user_types: pageProps?.user_types,
}),
).map((page) => ( ).map((page) => (
<RailSettingsLink <RailSettingsLink
key={page.url} key={page.url}
src/layouts/bots/(partials)/rail-settings-list.tsx
+6 -3
View File
@@ -2,6 +2,7 @@ import { useContext } from "react";
import Divider from "@moduletrace/twui/layout/Divider"; import Divider from "@moduletrace/twui/layout/Divider";
import { SettingsPages } from "@/src/dict/settings-pages-dict"; import { SettingsPages } from "@/src/dict/settings-pages-dict";
import { AppContext } from "@/src/pages/__root"; import { AppContext } from "@/src/pages/__root";
import canSeeRailItem from "@/src/utils/rbac/can-see-rail-item";
import isPassThroughUser from "@/src/utils/rbac/is-pass-through-user"; import isPassThroughUser from "@/src/utils/rbac/is-pass-through-user";
import RailBackLink from "./rail-back-link"; import RailBackLink from "./rail-back-link";
import RailSettingsLink from "./rail-settings-link"; import RailSettingsLink from "./rail-settings-link";
@@ -27,9 +28,11 @@ export default function RailSettingsList({ pathname }: Props) {
<RailBackLink /> <RailBackLink />
{/* <Divider className="shrink-0 mx-1 @rail:mx-3 my-2 w-auto border-hairline dark:border-hairline" /> */} {/* <Divider className="shrink-0 mx-1 @rail:mx-3 my-2 w-auto border-hairline dark:border-hairline" /> */}
{SettingsPages.filter((page) => {SettingsPages.filter((page) =>
page.can_view canSeeRailItem({
? page.can_view({ user_types: pageProps?.user_types }) item: page,
: is_super_admin || !page.super_admin_only, is_super_admin,
user_types: pageProps?.user_types,
}),
).map((page) => ( ).map((page) => (
<RailSettingsLink <RailSettingsLink
key={page.url} key={page.url}
src/layouts/bots/(partials)/rail-user-menu.tsx
+9 -4
View File
@@ -9,18 +9,23 @@ import RailUserMenuEntry from "./rail-user-menu-entry";
import RailUserMenuAction from "./rail-user-menu-action"; import RailUserMenuAction from "./rail-user-menu-action";
import InstallAppModal from "./install-app-modal"; import InstallAppModal from "./install-app-modal";
import { AppContext } from "@/src/pages/__root"; import { AppContext } from "@/src/pages/__root";
import canSeeRailItem from "@/src/utils/rbac/can-see-rail-item";
import isPassThroughUser from "@/src/utils/rbac/is-pass-through-user"; import isPassThroughUser from "@/src/utils/rbac/is-pass-through-user";
import useInstallPrompt from "@/src/hooks/use-install-prompt"; import useInstallPrompt from "@/src/hooks/use-install-prompt";
/** /**
* 220px popover: the only element in the shell with elevation. Super * 220px popover: the only element in the shell with elevation. Items
* admin items show to super admins only. Install app shows while the * follow the same rule as the rail's lists: super admin ones show to
* browser can install it; Safari gets the Share menu steps instead. * super admins only, RBAC-gated ones by their own rule. Install app shows
* while the browser can install it; Safari gets the Share menu steps
* instead.
*/ */
export default function RailUserMenu() { export default function RailUserMenu() {
const { pageProps } = useContext(AppContext); const { pageProps } = useContext(AppContext);
const is_super_admin = isPassThroughUser({ user_types: pageProps?.user_types }); const is_super_admin = isPassThroughUser({ user_types: pageProps?.user_types });
const shown = RailUserMenuItems.filter((item) => is_super_admin || !item.super_admin_only); const shown = RailUserMenuItems.filter((item) =>
canSeeRailItem({ item, is_super_admin, user_types: pageProps?.user_types }),
);
const items = shown.filter((item) => !item.destructive); const items = shown.filter((item) => !item.destructive);
const destructive = shown.filter((item) => item.destructive); const destructive = shown.filter((item) => item.destructive);
src/pages/admin/bots/[bot_id]/(hooks)/use-thread-scroll.ts
+162
View File
@@ -0,0 +1,162 @@
import { useLayoutEffect, useRef, useState, type RefObject } from "react";
import type { BotMessageWithAttachments } from "@/src/types";
import {
ThreadJumpWindowMs,
scrollThreadToBottom,
shouldFollowThread,
threadOffset,
} from "../(utils)/thread-scroll";
type Params = {
/**
* The thread's messages, oldest first, pending ones included.
*/
messages: BotMessageWithAttachments[];
};
/**
* Keeps the thread's bottom in view: a chat sitting at the bottom follows
* every new message, a chat scrolled up stays where it is (except for a
* message the user just sent, which brings the thread to it) and offers a
* button to get back down. Whether it sits at the bottom is read from where
* the content ends, not from `scrollTop`, whose sign a reversed column
* reports differently across engines. Content that grows while the thread
* is at the bottom (images landing, a wider window) is followed too.
*/
export default function useThreadScroll({ messages }: Params) {
const column_ref = useRef<HTMLDivElement>(null);
const content_ref = useRef<HTMLDivElement>(null);
const [jump_visible, setJumpVisible] = useState(false);
/** Whether the bottom is being held, so nothing fights the user's scroll. */
const following = useRef(true);
/**
* A pressed jump is still animating, so the scrolling it causes is its
* own. It ends when the thread arrives, when a scroll moves away from
* the bottom (the user took over), or after the window, whichever first.
*/
const jump = useRef({ active: false, closest: 0, until: 0 });
const previous = useRef<{
last_id?: number | string | null;
count: number;
}>({ count: 0 });
const measure = () => {
const column = column_ref.current;
const content = content_ref.current;
if (!column || !content) return null;
return threadOffset({
content_bottom: content.getBoundingClientRect().bottom,
column_bottom: column.getBoundingClientRect().bottom,
});
};
/** The user's own scrolling: it decides whether the thread follows. */
const onScroll = () => {
const offset = measure();
if (!offset) return;
if (jump.current.active) {
const animating =
Date.now() < jump.current.until &&
offset.distance <= jump.current.closest;
jump.current.closest = offset.distance;
jump.current.active = animating;
if (animating) return;
}
following.current = offset.near_bottom;
setJumpVisible(!offset.near_bottom);
};
const jumpToBottom = () => {
const column = column_ref.current;
if (!column) return;
const reduced = Boolean(
globalThis.matchMedia?.("(prefers-reduced-motion: reduce)").matches,
);
jump.current = {
active: !reduced,
closest: Number.POSITIVE_INFINITY,
until: Date.now() + ThreadJumpWindowMs,
};
following.current = true;
setJumpVisible(false);
scrollThreadToBottom({
element: column,
behavior: reduced ? "auto" : "smooth",
});
};
useLayoutEffect(() => {
const column = column_ref.current;
const newest = messages[messages.length - 1];
const before = previous.current;
/**
* The list was replaced (a new session, the first render) when the
* message that was last is gone.
*/
const kept_last_message =
before.count > 0 &&
messages.some((message) => message.id == before.last_id);
previous.current = { last_id: newest?.id, count: messages.length };
if (!column) return;
const follow = shouldFollowThread({
near_bottom: following.current,
newest_author: newest?.author,
kept_last_message,
});
if (!follow) {
setJumpVisible(true);
return;
}
following.current = true;
jump.current.active = false;
setJumpVisible(false);
scrollThreadToBottom({ element: column });
}, [messages]);
useLayoutEffect(() => {
const column = column_ref.current;
const content = content_ref.current;
if (!column || !content) return;
/**
* A message that grows after it lands (an image, a widened window)
* keeps the bottom in view while the thread is there, and leaves a
* chat scrolled up alone.
*/
const observer = new ResizeObserver(() => {
if (!following.current) return;
scrollThreadToBottom({ element: column });
});
observer.observe(content);
return () => observer.disconnect();
}, []);
return {
column_ref,
content_ref: content_ref as RefObject<HTMLDivElement>,
jump_visible,
jump_to_bottom: jumpToBottom,
on_scroll: onScroll,
};
}
src/pages/admin/bots/[bot_id]/(partials)/bot-message-line.tsx
+2
View File
@@ -12,6 +12,7 @@ import GroupTimestamp from "./group-timestamp";
import MessageActions from "./message-actions"; import MessageActions from "./message-actions";
import MessageReactions from "./message-reactions"; import MessageReactions from "./message-reactions";
import MessageReplyQuote from "./message-reply-quote"; import MessageReplyQuote from "./message-reply-quote";
import MessageUsageChip from "./message-usage-chip";
import NeedsYouChip from "./needs-you-chip"; import NeedsYouChip from "./needs-you-chip";
import SpeakMessageButton from "./speak-message-button"; import SpeakMessageButton from "./speak-message-button";
@@ -74,6 +75,7 @@ function BotMessageLine({ message, bot_name, at }: Props) {
<Row className="gap-3! items-center"> <Row className="gap-3! items-center">
{message.needs_you ? <NeedsYouChip /> : null} {message.needs_you ? <NeedsYouChip /> : null}
{at ? <GroupTimestamp at={at} /> : null} {at ? <GroupTimestamp at={at} /> : null}
<MessageUsageChip usage={message.usage} />
{can_speak ? ( {can_speak ? (
<SpeakMessageButton message_id={Number(message.id)} /> <SpeakMessageButton message_id={Number(message.id)} />
) : null} ) : null}
src/pages/admin/bots/[bot_id]/(partials)/computer-ref-chip.render.test.tsx
+82
View File
@@ -0,0 +1,82 @@
import { describe, expect, test } from "bun:test";
import { renderToString } from "react-dom/server";
import type { BUN_SQLITE_COBBLE_COMPUTERS } from "@/db/types/db";
import type { BotComputers } from "../(hooks)/use-bot-computers";
import { BotComputersContext } from "../(data)/bot-computers-context";
import ComputerRefChip from "./computer-ref-chip";
const main: BUN_SQLITE_COBBLE_COMPUTERS = { id: 1, name: "Main computer" };
const sandbox: BUN_SQLITE_COBBLE_COMPUTERS = { id: 2, name: "SEO box" };
type RenderProps = {
computer_id: number;
computers: BUN_SQLITE_COBBLE_COMPUTERS[];
};
function render({ computer_id, computers }: RenderProps) {
const value = makeComputers({ computers });
const html = renderToString(
<BotComputersContext.Provider value={value}>
<ComputerRefChip computer_id={computer_id} />
</BotComputersContext.Provider>,
);
return { html, value };
}
type MakeComputersProps = {
computers: BUN_SQLITE_COBBLE_COMPUTERS[];
};
/**
* A computers context recording what the chip asked for: the viewer it
* opened and the switches it made.
*/
function makeComputers({ computers }: MakeComputersProps) {
const switches: number[] = [];
const opened: { computer_id: number; mode: string }[] = [];
const value: BotComputers = {
computers,
switchComputer: async ({ computer_id }) => {
switches.push(computer_id);
},
openComputer: (params) => {
opened.push(params);
},
closeComputer: () => {},
};
return Object.assign(value, { switches, opened });
}
describe("ComputerRefChip", () => {
test("opens the viewer in its browser and switches nothing", async () => {
const { html, value } = render({
computer_id: 2,
computers: [main, sandbox],
});
expect(html).toContain("SEO box");
// The chip's only job is opening, so no switch affordance survives.
expect(html).not.toContain("Switch to");
expect(value.switches).toHaveLength(0);
});
test("the computer the bot is on is still a chip, sunken with a check", () => {
const { html } = render({ computer_id: 1, computers: [main, sandbox] });
expect(html).toContain("Main computer");
expect(html).toContain("<button");
expect(html).toContain("bg-sunken");
expect(html).toContain('title="Open Main computer"');
});
test("a computer the bot can't use stays a plain name", () => {
const { html } = render({ computer_id: 9, computers: [main] });
expect(html).toContain("computer 9");
expect(html).not.toContain("<button");
});
});
src/pages/admin/bots/[bot_id]/(partials)/computer-ref-chip.tsx
+20 -24
View File
@@ -2,6 +2,7 @@ import { useContext } from "react";
import Button from "@moduletrace/twui/layout/Button"; import Button from "@moduletrace/twui/layout/Button";
import Span from "@moduletrace/twui/layout/Span"; import Span from "@moduletrace/twui/layout/Span";
import { Check, Monitor } from "lucide-react"; import { Check, Monitor } from "lucide-react";
import { twMerge } from "tailwind-merge";
import { BotComputersContext } from "../(data)/bot-computers-context"; import { BotComputersContext } from "../(data)/bot-computers-context";
type Props = { type Props = {
@@ -9,33 +10,22 @@ type Props = {
}; };
/** /**
* A computer the bot named in a message (`computerRef`): a button that * A computer the bot named in a message (`computerRef`): a chip that opens
* switches the bot to it at once, or a checked chip when it's already on * it in the viewer, in its browser. The one the bot is already on says so,
* it. A computer the bot can't use (any more) is shown by name only. * sunken with a check. A computer the bot can't use (any more) is shown by
* name only.
*/ */
export default function ComputerRefChip({ computer_id }: Props) { export default function ComputerRefChip({ computer_id }: Props) {
const { computers, switching_id, error, switchComputer } = useContext(BotComputersContext); const { computers, openComputer } = useContext(BotComputersContext);
const index = computers.findIndex((c) => c.id == computer_id); const computer = computers.find((c) => c.id == computer_id);
const computer = computers[index];
const name = computer?.name || `computer ${computer_id}`; const name = computer?.name || `computer ${computer_id}`;
if (!computer) { if (!computer?.id) {
return <Span className="font-bold">{name}</Span>; return <Span className="font-bold">{name}</Span>;
} }
if (index == 0) { const current = computers[0]?.id == computer.id;
return (
<Span
title="The computer this bot is on"
className="inline-flex h-6 gap-1.5 items-center px-2 align-middle rounded-default-xs bg-sunken text-[13px] font-bold leading-4 text-ink whitespace-nowrap"
>
<Monitor size={14} aria-hidden />
{name}
<Check size={14} aria-hidden />
</Span>
);
}
return ( return (
<Button <Button
@@ -43,14 +33,20 @@ export default function ComputerRefChip({ computer_id }: Props) {
variant="outlined" variant="outlined"
color="gray" color="gray"
size="smaller" size="smaller"
title={error && switching_id == undefined ? error : `Switch to ${name}`} title={`Open ${name}`}
loading={switching_id == computer_id} onClick={() =>
disabled={Boolean(switching_id)} openComputer({ computer_id: computer.id || 0, mode: "browser" })
onClick={() => switchComputer({ computer_id })} }
className="inline-flex h-6 gap-1.5! flex-nowrap align-middle px-2! text-[13px] font-bold text-ink whitespace-nowrap" className={twMerge(
"inline-flex h-6 gap-1.5! flex-nowrap align-middle px-2! text-[13px] font-bold whitespace-nowrap text-ink dark:text-ink",
current
? "bg-sunken hover:bg-sunken outline-0"
: "bg-transparent hover:bg-sunken",
)}
> >
<Monitor size={14} aria-hidden className="shrink-0" /> <Monitor size={14} aria-hidden className="shrink-0" />
{name} {name}
{current ? <Check size={14} aria-hidden className="shrink-0" /> : null}
</Button> </Button>
); );
} }
src/pages/admin/bots/[bot_id]/(partials)/message-thread.tsx
+45 -31
View File
@@ -1,15 +1,17 @@
import { useMemo, useRef, type RefObject } from "react"; import { useMemo, type RefObject } from "react";
import Stack from "@moduletrace/twui/layout/Stack"; import Stack from "@moduletrace/twui/layout/Stack";
import type { import type {
BotMessageWithAttachments, BotMessageWithAttachments,
BotWithLastMessage, BotWithLastMessage,
} from "@/src/types"; } from "@/src/types";
import useMessageSelection from "../(hooks)/use-message-selection"; import useMessageSelection from "../(hooks)/use-message-selection";
import useThreadScroll from "../(hooks)/use-thread-scroll";
import groupMessages from "../(utils)/group-messages"; import groupMessages from "../(utils)/group-messages";
import BotThinkingRow from "./bot-thinking-row"; import BotThinkingRow from "./bot-thinking-row";
import MessageGroupBlock from "./message-group-block"; import MessageGroupBlock from "./message-group-block";
import OlderMessagesLoader from "./older-messages-loader"; import OlderMessagesLoader from "./older-messages-loader";
import QuoteSelectionButton from "./quote-selection-button"; import QuoteSelectionButton from "./quote-selection-button";
import ScrollToBottomButton from "./scroll-to-bottom-button";
type Props = { type Props = {
bot: BotWithLastMessage; bot: BotWithLastMessage;
@@ -51,7 +53,9 @@ type Props = {
* While the bot works on a reply or a task, a status row sits under the * While the bot works on a reply or a task, a status row sits under the
* last one. Text selected in a message can be quoted in the composer. * last one. Text selected in a message can be quoted in the composer.
* Older messages load as it scrolls to the top; the reversed column keeps * Older messages load as it scrolls to the top; the reversed column keeps
* the view where it was as they land above it. * the view where it was as they land above it. The thread follows new
* messages while it sits at its bottom, stays put while it's scrolled up
* and then offers a button to get back down.
*/ */
export default function MessageThread({ export default function MessageThread({
bot, bot,
@@ -64,42 +68,52 @@ export default function MessageThread({
older, older,
}: Props) { }: Props) {
const groups = useMemo(() => groupMessages({ messages }), [messages]); const groups = useMemo(() => groupMessages({ messages }), [messages]);
const ref = useRef<HTMLDivElement>(null); const scroll = useThreadScroll({ messages });
const { selection, clear } = useMessageSelection({ ref }); const { selection, clear } = useMessageSelection({ ref: scroll.column_ref });
return ( return (
<Stack <Stack className="relative grow min-h-0 gap-0! items-stretch">
componentRef={ref as RefObject<HTMLDivElement>}
className="grow min-h-0 gap-0! items-stretch overflow-y-auto flex-col-reverse">
<Stack <Stack
role="log" componentRef={scroll.column_ref as RefObject<HTMLDivElement>}
aria-live="polite" className="grow min-h-0 gap-0! items-stretch overflow-y-auto flex-col-reverse"
aria-label={`Conversation with ${bot.name}`} onScroll={scroll.on_scroll}
className="w-full max-w-container mx-auto gap-4! items-stretch px-3 md:px-6 pt-6 pb-6"
> >
{older?.has_more ? ( <Stack
<OlderMessagesLoader componentRef={scroll.content_ref as RefObject<HTMLDivElement>}
loading={older.loading} role="log"
error={older.error} aria-live="polite"
onLoad={older.loadOlder} aria-label={`Conversation with ${bot.name}`}
/> className="w-full max-w-container mx-auto gap-4! items-stretch px-3 md:px-6 pt-6 pb-6"
) : null} >
{groups.map((group) => ( {older?.has_more ? (
<MessageGroupBlock <OlderMessagesLoader
key={group.key} loading={older.loading}
error={older.error}
onLoad={older.loadOlder}
/>
) : null}
{groups.map((group) => (
<MessageGroupBlock
key={group.key}
bot={bot}
group={group}
/>
))}
<BotThinkingRow
bot={bot} bot={bot}
group={group} progress={progress}
reply_progress={reply_progress}
awaiting_reply={
messages[messages.length - 1]?.author == "user"
}
receiving={receiving}
has_task={has_task}
/> />
))} </Stack>
<BotThinkingRow
bot={bot}
progress={progress}
reply_progress={reply_progress}
awaiting_reply={messages[messages.length - 1]?.author == "user"}
receiving={receiving}
has_task={has_task}
/>
</Stack> </Stack>
{scroll.jump_visible ? (
<ScrollToBottomButton onClick={scroll.jump_to_bottom} />
) : null}
{selection && onQuote ? ( {selection && onQuote ? (
<QuoteSelectionButton <QuoteSelectionButton
selection={selection} selection={selection}
src/pages/admin/bots/[bot_id]/(partials)/message-usage-chip.tsx
+35
View File
@@ -0,0 +1,35 @@
import Span from "@moduletrace/twui/layout/Span";
import type { TokenUsageSummary } from "@/src/harness/usage/types";
import formatTokens from "@/src/utils/format-tokens";
type Props = {
usage?: TokenUsageSummary | null;
};
/**
* What one answer cost, in the row under it beside its timestamp: the
* tokens every model call behind it spent (a chat turn, reading an image,
* making one), and the price when the provider reported it. The hover
* title breaks it down.
*/
export default function MessageUsageChip({ usage }: Props) {
if (!usage || usage.total_tokens <= 0) return null;
const { prompt_tokens, completion_tokens, total_tokens, cost, calls } = usage;
const title = [
`${formatTokens({ tokens: prompt_tokens })} in, ${formatTokens({ tokens: completion_tokens })} out`,
`${calls} model ${calls == 1 ? "call" : "calls"}`,
typeof cost == "number" && cost > 0 ? `$${cost.toFixed(4)}` : "",
]
.filter(Boolean)
.join(" · ");
return (
<Span
title={title}
className="text-[11px] leading-4 text-ink-secondary tabular-nums whitespace-nowrap cursor-help"
>
{formatTokens({ tokens: total_tokens })} tokens
</Span>
);
}
src/pages/admin/bots/[bot_id]/(partials)/scroll-to-bottom-button.tsx
+28
View File
@@ -0,0 +1,28 @@
import Button from "@moduletrace/twui/layout/Button";
import { ArrowDown } from "lucide-react";
type Props = {
onClick: () => void;
};
/**
* "Jump to latest", floating over the bottom of the thread while it sits
* above it: a bot's reply that landed out of sight, or a long scroll back
* through the conversation.
*/
export default function ScrollToBottomButton({ onClick }: Props) {
return (
<Button
type="button"
color="secondary"
title="Jump to the latest message"
aria-label="Jump to the latest message"
variant="outlined"
onClick={onClick}
className="absolute bottom-3 left-1/2 z-20 shrink-0 -translate-x-1/2 gap-1.5! px-2.5! py-1.5! rounded-default-sm border border-solid border-hairline bg-surface shadow-popover text-xs font-bold text-ink hover:bg-sunken focus-visible:ring-2 focus-visible:ring-ink"
>
<ArrowDown size={13} aria-hidden />
Jump to latest
</Button>
);
}
src/pages/admin/bots/[bot_id]/(utils)/thread-scroll.test.ts
+98
View File
@@ -0,0 +1,98 @@
import { describe, expect, test } from "bun:test";
import {
ThreadBottomSlack,
shouldFollowThread,
threadOffset,
} from "./thread-scroll";
describe("threadOffset", () => {
test("content whose bottom edge is the column's is at the bottom", () => {
expect(
threadOffset({ content_bottom: 800, column_bottom: 800 }),
).toEqual({ distance: 0, near_bottom: true });
});
test("content ending just short of the column's bottom is at the bottom", () => {
expect(
threadOffset({ content_bottom: 780, column_bottom: 800 }),
).toEqual({ distance: 0, near_bottom: true });
});
test("a small offset past the bottom still counts as being there", () => {
const offset = threadOffset({
content_bottom: 800 + ThreadBottomSlack - 1,
column_bottom: 800,
});
expect(offset.near_bottom).toBe(true);
expect(offset.distance).toBe(ThreadBottomSlack - 1);
});
test("one pixel past the slack is not the bottom anymore", () => {
const offset = threadOffset({
content_bottom: 800 + ThreadBottomSlack + 1,
column_bottom: 800,
});
expect(offset.near_bottom).toBe(false);
expect(offset.distance).toBe(ThreadBottomSlack + 1);
});
test("scrolled up by a long way is far from the bottom", () => {
expect(
threadOffset({ content_bottom: 4800, column_bottom: 800 }),
).toEqual({ distance: 4000, near_bottom: false });
});
test("the slack is configurable", () => {
expect(
threadOffset({
content_bottom: 1000,
column_bottom: 800,
slack: 200,
}).near_bottom,
).toBe(true);
});
});
describe("shouldFollowThread", () => {
test("a chat at the bottom follows a bot's reply", () => {
expect(
shouldFollowThread({
near_bottom: true,
newest_author: "bot",
kept_last_message: true,
}),
).toBe(true);
});
test("a chat scrolled up stays put when a bot's reply lands", () => {
expect(
shouldFollowThread({
near_bottom: false,
newest_author: "bot",
kept_last_message: true,
}),
).toBe(false);
});
test("a message the user sent always brings the thread to it", () => {
expect(
shouldFollowThread({
near_bottom: false,
newest_author: "user",
kept_last_message: true,
}),
).toBe(true);
});
test("a list that replaced the old one opens at its bottom", () => {
expect(
shouldFollowThread({
near_bottom: false,
newest_author: "bot",
kept_last_message: false,
}),
).toBe(true);
});
});
src/pages/admin/bots/[bot_id]/(utils)/thread-scroll.ts
+75
View File
@@ -0,0 +1,75 @@
/**
* Pixels from the bottom that still count as being at it, so a mouse wheel
* tick or a rounding difference doesn't hide the latest message.
*/
export const ThreadBottomSlack = 48;
/**
* How long a smooth jump to the bottom runs, in milliseconds. The scrolling
* it causes is ignored for that long (so the button doesn't flicker back
* mid-animation), and the window ends on its own however it lands.
*/
export const ThreadJumpWindowMs = 800;
/**
* How far the thread's content sits below the bottom of its column, in
* pixels, and whether that still counts as being at the bottom. It reads
* the two rects rather than `scrollTop`, whose sign a reversed column
* (`flex-col-reverse`) reports differently across engines: the content's
* bottom edge is the column's when the thread is at its bottom, and past it
* by however far the thread has been scrolled up.
*/
export function threadOffset({
content_bottom,
column_bottom,
slack = ThreadBottomSlack,
}: {
content_bottom: number;
column_bottom: number;
slack?: number;
}) {
const distance = content_bottom - column_bottom;
const at_bottom = distance <= slack;
return {
distance: at_bottom && distance < 0 ? 0 : distance,
near_bottom: at_bottom,
};
}
/**
* Whether the thread should show a change of its messages, which keeps the
* bottom in view while the user is there: it follows while at the bottom, a
* message the user sent always counts, and a list that replaced the old one
* (a new session, a first render) opens at its bottom. A chat scrolled up
* stays where it is when a bot answers.
*/
export function shouldFollowThread({
near_bottom,
newest_author,
kept_last_message,
}: {
near_bottom: boolean;
newest_author?: string | null;
kept_last_message: boolean;
}) {
if (!kept_last_message) return true;
if (newest_author == "user") return true;
return near_bottom;
}
/**
* Puts the thread at its bottom, animated or at once. A reversed column
* clamps a past-the-end `scrollTop` back onto its bottom, so the same call
* serves either direction.
*/
export function scrollThreadToBottom({
element,
behavior = "auto",
}: {
element: HTMLElement;
behavior?: ScrollBehavior;
}) {
element.scrollTo({ top: element.scrollHeight, behavior });
}
src/pages/admin/settings/(functions)/grab-settings-server-props.ts
+2
View File
@@ -5,6 +5,7 @@ import sanitizeUserBio from "@/src/functions/backend/db/users/sanitize-user-bio"
import grabUserPasswordHash from "@/src/functions/backend/auth/grab-user-password-hash"; import grabUserPasswordHash from "@/src/functions/backend/auth/grab-user-password-hash";
import type { PagePropsType } from "@/src/types"; import type { PagePropsType } from "@/src/types";
import type { BunextPageServerFn } from "@moduletrace/bunext/types"; import type { BunextPageServerFn } from "@moduletrace/bunext/types";
import grabRunLimitsOverview from "@/src/functions/backend/db/app-settings/grab-run-limits";
import grabStorageQuotaOverview from "@/src/functions/backend/db/app-settings/grab-storage-quota-overview"; import grabStorageQuotaOverview from "@/src/functions/backend/db/app-settings/grab-storage-quota-overview";
import { grabSystemPromptAppendOverview } from "@/src/functions/backend/db/app-settings/grab-system-prompt-append"; import { grabSystemPromptAppendOverview } from "@/src/functions/backend/db/app-settings/grab-system-prompt-append";
import runningVersion from "@/src/utils/running-version"; import runningVersion from "@/src/utils/running-version";
@@ -45,6 +46,7 @@ export default async function grabSettingsServerProps({ req }: Params) {
has_password, has_password,
storage_quota: await grabStorageQuotaOverview().catch(() => null), storage_quota: await grabStorageQuotaOverview().catch(() => null),
system_prompt_append: await grabSystemPromptAppendOverview().catch(() => null), system_prompt_append: await grabSystemPromptAppendOverview().catch(() => null),
run_limits: await grabRunLimitsOverview().catch(() => null),
running_version: runningVersion({ env: process.env }), running_version: runningVersion({ env: process.env }),
}, },
}; };
src/pages/admin/settings/(partials)/admin-breadcrumbs.render.test.tsx
+94
View File
@@ -0,0 +1,94 @@
import { describe, expect, test } from "bun:test";
import { renderToString } from "react-dom/server";
import type { AppContextObject, PagePropsType } from "@/src/types";
import { AppContext } from "@/src/pages/__root";
import SettingsShell from "./settings-shell";
type RenderProps = {
pathname: string;
/**
* The page's own title, the name the header shows.
*/
title: string;
};
function render({ pathname, title }: RenderProps) {
const pageProps: PagePropsType = {
url: { pathname } as PagePropsType["url"],
};
const context = { pageProps } as AppContextObject;
return renderToString(
<AppContext.Provider value={context}>
<SettingsShell
title={title}
description="Work your bots do on a schedule."
>
<span>PAGE_CONTENT</span>
</SettingsShell>
</AppContext.Provider>,
);
}
describe("SettingsShell breadcrumbs", () => {
test("a top-level page shows Home and itself", () => {
const html = render({ pathname: "/admin/routines", title: "Routines" });
expect(html).toContain('aria-label="Breadcrumb"');
expect(html).toContain(">Home<");
expect(html).toContain(">Routines<");
expect(html).toContain('href="/admin/routines"');
});
test("a settings page walks through Settings up to the page", () => {
const html = render({
pathname: "/admin/settings/general",
title: "General",
});
expect(html).toContain('href="/admin"');
expect(html).toContain('href="/admin/settings"');
expect(html).toContain('href="/admin/settings/general"');
});
test("a record page reads its name, not the id in its address", () => {
const html = render({
pathname: "/admin/settings/bots/3",
title: "Scout",
});
expect(html).toContain(">Scout<");
expect(html).not.toContain(">3<");
});
test("a page whose slug differs from its name shows the name", () => {
const html = render({
pathname: "/admin/settings/change-password",
title: "Change password",
});
expect(html).toContain(">Change password<");
expect(html).not.toContain(">Change Password<");
});
test("Activity keeps its name though its address says logs", () => {
const html = render({ pathname: "/admin/logs", title: "Activity" });
expect(html).toContain(">Activity<");
expect(html).not.toContain(">Logs<");
});
test("the trail sits above the page's content, right after the header", () => {
const html = render({ pathname: "/admin/routines", title: "Routines" });
expect(html.indexOf('aria-label="Breadcrumb"')).toBeLessThan(
html.indexOf("PAGE_CONTENT"),
);
});
test("the crew of bots, which is home, has no trail above it", () => {
const html = render({ pathname: "/admin/bots", title: "Bots" });
expect(html).not.toContain('aria-label="Breadcrumb"');
});
});
src/pages/admin/settings/(partials)/settings-breadcrumbs.tsx
-50
View File
@@ -1,50 +0,0 @@
import { useContext } from "react";
import Breadcrumbs from "@moduletrace/twui/elements/Breadcrumbs";
import Stack from "@moduletrace/twui/layout/Stack";
import { ChevronRight } from "lucide-react";
import { AppContext } from "@/src/pages/__root";
type Props = {
/**
* The record's name, shown where the URL has its id.
*/
title: string;
};
/**
* The hierarchy above a nested settings page: Settings / Bots / Scout.
* Built from the path, with the record's name replacing its id.
*/
export default function SettingsBreadcrumbs({ title }: Props) {
const { pageProps } = useContext(AppContext);
const pathname = pageProps?.url?.pathname;
if (!pathname) return null;
return (
<Stack className="w-full shrink-0 pt-5 pb-1">
<Breadcrumbs
pageUrl={pathname}
currentTitle={title}
skipHome
excludeRegexMatch={/^admin$/}
linkProps={{
className:
"border-0 text-[13px] leading-[18px] text-ink-secondary no-underline hover:text-ink hover:opacity-100 dark:text-ink-secondary",
}}
currentLinkProps={{
title,
className:
"border-0 text-[13px] leading-[18px] text-ink no-underline pointer-events-none dark:text-ink",
}}
divider={
<ChevronRight
size={12}
aria-hidden
className="shrink-0 text-ink-secondary"
/>
}
/>
</Stack>
);
}
src/pages/admin/settings/(partials)/settings-shell.tsx
+17 -12
View File
@@ -1,7 +1,8 @@
import type { ReactNode } from "react"; import { useContext, type ReactNode } from "react";
import { twMerge } from "tailwind-merge"; import { twMerge } from "tailwind-merge";
import Stack from "@moduletrace/twui/layout/Stack"; import Stack from "@moduletrace/twui/layout/Stack";
import SettingsBreadcrumbs from "./settings-breadcrumbs"; import AdminBreadcrumbs from "@/src/components/general/admin-breadcrumbs";
import { AppContext } from "@/src/pages/__root";
import SettingsPaneHeader from "./settings-pane-header"; import SettingsPaneHeader from "./settings-pane-header";
type Props = { type Props = {
@@ -12,10 +13,6 @@ type Props = {
* A wider column, for tables (activity). * A wider column, for tables (activity).
*/ */
wide?: boolean; wide?: boolean;
/**
* A nested record page's name, for its breadcrumb trail.
*/
breadcrumb_title?: string;
/** /**
* Controls at the end of the pane header, e.g. a settings link. * Controls at the end of the pane header, e.g. a settings link.
*/ */
@@ -23,17 +20,19 @@ type Props = {
}; };
/** /**
* The settings pane beside the rail: a 64px header like the conversation * The pane beside the rail: a 64px header like the conversation header,
* header, then one readable column that scrolls on its own. * then one readable column that scrolls on its own, opening with the
* breadcrumb trail above the page.
*/ */
export default function SettingsShell({ export default function SettingsShell({
title, title,
description, description,
children, children,
wide, wide,
breadcrumb_title,
action, action,
}: Props) { }: Props) {
const { pageProps } = useContext(AppContext);
return ( return (
<Stack className="grow min-w-0 h-full gap-0! items-stretch"> <Stack className="grow min-w-0 h-full gap-0! items-stretch">
<SettingsPaneHeader <SettingsPaneHeader
@@ -48,9 +47,15 @@ export default function SettingsShell({
wide && "max-w-[1200px]", wide && "max-w-[1200px]",
)} )}
> >
{breadcrumb_title ? ( {/**
<SettingsBreadcrumbs title={breadcrumb_title} /> * The page's own `title` names the last crumb: the
) : null} * address would say the id of a record, or a slug that
* differs from the name the page shows.
*/}
<AdminBreadcrumbs
pathname={pageProps?.url?.pathname}
title={title}
/>
{children} {children}
</Stack> </Stack>
</Stack> </Stack>
src/pages/admin/settings/bots/[bot_id]/(sections)/bot-settings-section.tsx
-1
View File
@@ -39,7 +39,6 @@ export default function BotSettingsSection() {
? "Coordinates your crew and hands work to it" ? "Coordinates your crew and hands work to it"
: "A worker bot.") : "A worker bot.")
} }
breadcrumb_title={name}
> >
<BotOverviewGroup bot={bot} /> <BotOverviewGroup bot={bot} />
<BotDetailsGroup <BotDetailsGroup
src/pages/admin/settings/computers/[computer_id]/(sections)/computer-settings-section.tsx
-1
View File
@@ -45,7 +45,6 @@ export default function ComputerSettingsSection() {
computer.description || computer.description ||
"Its container, the bots that reach it, and its workspaces." "Its container, the bots that reach it, and its workspaces."
} }
breadcrumb_title={name}
> >
<ComputerOverviewGroup computer={computer} /> <ComputerOverviewGroup computer={computer} />
{is_super_admin ? ( {is_super_admin ? (
src/pages/admin/settings/general/(partials)/run-limits-form.tsx
+134
View File
@@ -0,0 +1,134 @@
import { useState } from "react";
import fetchApi from "@moduletrace/twui/utils/fetch/fetchApi";
import type { APIResponseObject } from "@moduletrace/bunext/types";
import Form from "@moduletrace/twui/form/Form";
import Input from "@moduletrace/twui/form/Input";
import Stack from "@moduletrace/twui/layout/Stack";
import type { RunLimitsOverview } from "@/src/functions/backend/db/app-settings/grab-run-limits";
import SettingsSaveBar from "../../(partials)/settings-save-bar";
type Props = {
limits?: RunLimitsOverview | null;
};
/**
* How many model turns a task run and a chat reply may take, saved on its
* own without a reload. Zero puts the code's default back for that one.
*/
export default function RunLimitsForm({ limits }: Props) {
const [overview, setOverview] = useState(limits || undefined);
const [dirty, setDirty] = useState(false);
const [task_steps, setTaskSteps] = useState(String(limits?.task_steps ?? ""));
const [reply_steps, setReplySteps] = useState(String(limits?.reply_steps ?? ""));
/**
* The inputs only read `defaultValue` when they mount, so discarding or
* saving remounts them with the saved numbers.
*/
const [form_key, setFormKey] = useState(0);
const [loading, setLoading] = useState(false);
const [status, setStatus] = useState<{
msg?: string;
error?: boolean;
success?: boolean;
}>();
const save = async () => {
setLoading(true);
setStatus(undefined);
const res = await fetchApi<{}, APIResponseObject<RunLimitsOverview>>(
"/api/admin/settings/run-limits",
{
method: "POST",
body: {
task_steps: Number(task_steps) || 0,
reply_steps: Number(reply_steps) || 0,
},
},
).catch(() => null);
setLoading(false);
if (!res?.success || !res.singleRes) {
setStatus({
error: true,
msg: res?.msg || `Couldn't save the limits.`,
});
return;
}
setOverview(res.singleRes);
setTaskSteps(String(res.singleRes.task_steps));
setReplySteps(String(res.singleRes.reply_steps));
setFormKey((key) => key + 1);
setDirty(false);
setStatus({
success: true,
msg: `Saved: ${res.singleRes.task_steps} steps per task, ${res.singleRes.reply_steps} per reply.`,
});
};
const discard = () => {
setTaskSteps(String(overview?.task_steps ?? ""));
setReplySteps(String(overview?.reply_steps ?? ""));
setFormKey((key) => key + 1);
setDirty(false);
setStatus(undefined);
};
return (
<Form
onSubmit={(event) => {
event.preventDefault();
save().catch(() => {});
}}
className="w-full gap-3! items-stretch"
>
<Stack className="w-full gap-3! items-stretch">
<Input
key={`task-${form_key}`}
label="Tool steps per task"
type="number"
min={0}
max={overview?.max}
step={1}
numberText
rawNumber
defaultValue={task_steps}
onChange={(event) => {
setTaskSteps(event.target.value);
setDirty(true);
setStatus(undefined);
}}
suffix={`steps`}
info={`How many turns the model may take before it must stop and report, when it works on a task. 0 uses the default (${overview?.task_default ?? ""}).`}
/>
<Input
key={`reply-${form_key}`}
label="Tool steps per chat reply"
type="number"
min={0}
max={overview?.max}
step={1}
numberText
rawNumber
defaultValue={reply_steps}
onChange={(event) => {
setReplySteps(event.target.value);
setDirty(true);
setStatus(undefined);
}}
suffix={`steps`}
info={`How many turns a chat reply may take before it answers. Long work belongs in a task. 0 uses the default (${overview?.reply_default ?? ""}).`}
/>
</Stack>
<SettingsSaveBar
dirty={dirty}
loading={loading}
status={status}
save_label="Save limits"
onDiscard={discard}
/>
</Form>
);
}
src/pages/admin/settings/general/(sections)/run-limits-section.tsx
+25
View File
@@ -0,0 +1,25 @@
import { useContext } from "react";
import { AppContext } from "@/src/pages/__root";
import isPassThroughUser from "@/src/utils/rbac/is-pass-through-user";
import SettingsGroup from "../../(partials)/settings-group";
import RunLimitsForm from "../(partials)/run-limits-form";
/**
* How long a bot may work on one task or reply before it reports. Only the
* super admin sees it, since it changes every bot on the install.
*/
export default function RunLimitsSection() {
const { pageProps } = useContext(AppContext);
if (!isPassThroughUser({ user_types: pageProps?.user_types })) return null;
return (
<SettingsGroup
title="Run limits"
description="How many turns a bot may take with its tools before it must stop and report. A task may take far more than a chat reply, since it does the long work."
id="run-limits"
>
<RunLimitsForm limits={pageProps?.run_limits} />
</SettingsGroup>
);
}
src/pages/admin/settings/general/index.tsx
+2
View File
@@ -1,6 +1,7 @@
import AppearanceSection from "./(sections)/appearance-section"; import AppearanceSection from "./(sections)/appearance-section";
import NotificationsSection from "./(sections)/notifications-section"; import NotificationsSection from "./(sections)/notifications-section";
import PromptSection from "./(sections)/prompt-section"; import PromptSection from "./(sections)/prompt-section";
import RunLimitsSection from "./(sections)/run-limits-section";
import StorageSection from "./(sections)/storage-section"; import StorageSection from "./(sections)/storage-section";
import VersionSection from "./(sections)/version-section"; import VersionSection from "./(sections)/version-section";
import SettingsShell from "../(partials)/settings-shell"; import SettingsShell from "../(partials)/settings-shell";
@@ -15,6 +16,7 @@ export default function AdminSettingsGeneralPage() {
<AppearanceSection /> <AppearanceSection />
<NotificationsSection /> <NotificationsSection />
<PromptSection /> <PromptSection />
<RunLimitsSection />
<StorageSection /> <StorageSection />
<VersionSection /> <VersionSection />
</SettingsShell> </SettingsShell>
src/pages/admin/settings/mcps/index.tsx
+4 -1
View File
@@ -4,7 +4,10 @@ import settingsPageMeta from "../(functions)/settings-page-meta";
export default function AdminSettingsMcpsPage() { export default function AdminSettingsMcpsPage() {
return ( return (
<SettingsShell title="MCPs" description="MCP servers whose tools your bots can use."> <SettingsShell
title="MCPs"
description="MCP servers whose tools your bots can use."
>
<McpsSettingsSection /> <McpsSettingsSection />
</SettingsShell> </SettingsShell>
); );
src/pages/admin/settings/users/[user_id]/(sections)/user-settings-section.tsx
-1
View File
@@ -39,7 +39,6 @@ export default function UserSettingsSection() {
description={ description={
types_title || person.email || "A user on this install." types_title || person.email || "A user on this install."
} }
breadcrumb_title={name}
> >
<UserOverviewGroup <UserOverviewGroup
person={person} person={person}
src/pages/admin/settings/workspaces/[workspace_id]/(sections)/workspace-overview.tsx
-1
View File
@@ -50,7 +50,6 @@ export default function WorkspaceOverview() {
workspace.description || workspace.description ||
"Context, bots, computers and models for this workspace." "Context, bots, computers and models for this workspace."
} }
breadcrumb_title={name}
> >
<WorkspaceImageGroup workspace={workspace} /> <WorkspaceImageGroup workspace={workspace} />
<WorkspaceDetailsGroup workspace={workspace} onSaved={setWorkspace} /> <WorkspaceDetailsGroup workspace={workspace} onSaved={setWorkspace} />
src/pages/admin/usage/(data)/usage-bar-heights.ts
+37
View File
@@ -0,0 +1,37 @@
/**
* The heights a day's bar can take, shortest first. A day's tokens are
* placed on this scale rather than given a height in pixels, so every bar
* is a static utility class the stylesheet already carries: no inline
* style, and the theme still decides the colour.
*/
export const UsageBarHeights = [
"h-0.5",
"h-1",
"h-1.5",
"h-2",
"h-2.5",
"h-3",
"h-4",
"h-5",
"h-6",
"h-8",
"h-10",
"h-12",
"h-14",
"h-16",
"h-20",
"h-24",
];
/**
* Where a value sits on that scale, against the window's busiest day. A
* day with work on it is never shorter than one step, so it is visible.
*/
export function usageBarHeight({ value, max }: { value: number; max: number }) {
if (value <= 0) return UsageBarHeights[0]!;
const ratio = max > 0 ? value / max : 0;
const step = Math.max(1, Math.round(ratio * (UsageBarHeights.length - 1)));
return UsageBarHeights[Math.min(step, UsageBarHeights.length - 1)]!;
}
src/pages/admin/usage/(data)/usage-harness-titles.ts
+12
View File
@@ -0,0 +1,12 @@
/**
* What each harness is called on the usage page, in words rather than the
* harness's own key.
*/
export const UsageHarnessTitles: Record<string, string> = {
text: "Chat and tasks",
vision: "Reading media",
transcription: "Transcription",
speech: "Speech",
image: "Image generation",
embedding: "Embeddings",
};
src/pages/admin/usage/(data)/usage-ranges.ts
+27
View File
@@ -0,0 +1,27 @@
export type UsageRange = {
/**
* The days back the range totals, today included.
*/
days: number;
title: string;
};
/**
* The periods the usage page offers, shortest first. A week and a month
* are what most questions need; a quarter is for spotting a trend.
*/
export const UsageRanges: UsageRange[] = [
{ days: 7, title: "7 days" },
{ days: 30, title: "30 days" },
{ days: 90, title: "90 days" },
];
/**
* The range asked for in the address, or the page's default when it isn't
* one of the offered ones.
*/
export function parseUsageRange({ value }: { value?: string | null }): UsageRange | null {
const days = Number(value);
return UsageRanges.find((range) => range.days == days) || null;
}
src/pages/admin/usage/(functions)/grab-usage-server-props.ts
+35
View File
@@ -0,0 +1,35 @@
import type { PagePropsType } from "@/src/types";
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
import { AdminHomePath } from "@/src/dict/admin-sections-dict";
import userAuth from "@/src/functions/backend/auth/user-auth";
import grabUsageOverview from "@/src/functions/backend/db/usage/grab-usage-overview";
import canManageCrew from "@/src/utils/rbac/can-manage-crew";
import { parseUsageRange } from "../(data)/usage-ranges";
type Params = Parameters<BunextPageServerFn<PagePropsType>>[0];
/**
* The usage page's data for the period in the address (the default when
* it carries none). Crew managers only, since it is the whole install's
* work rather than one user's conversation; everyone else lands on the
* bots.
*/
export default async function grabUsageServerProps(params: Params) {
const { user, user_types } = await userAuth({ req: params.req });
if (!user?.id) {
return { redirect: { destination: "/auth/login" } };
}
if (!canManageCrew({ user_types })) {
return { redirect: { destination: AdminHomePath } };
}
const range = parseUsageRange({ value: params.query?.range });
return {
props: {
usage: await grabUsageOverview({ days: range?.days }).catch(() => null),
},
};
}
src/pages/admin/usage/(functions)/sum-tokens.ts
+13
View File
@@ -0,0 +1,13 @@
import type { TokenUsageSummary } from "@/src/harness/usage/types";
type Params = {
rows: TokenUsageSummary[];
};
/**
* The tokens every row of a breakdown spent together, which is the
* denominator of its shares.
*/
export default function sumTokens({ rows }: Params) {
return rows.reduce((total, row) => total + row.total_tokens, 0);
}
src/pages/admin/usage/(functions)/usage-href.ts
+15
View File
@@ -0,0 +1,15 @@
import { UsagePath } from "@/src/dict/rail-pages-dict";
type Params = {
days?: number;
};
/**
* The usage page's address for the period it is showing, so a range can be
* linked to and the browser's own back button walks back through them.
*/
export default function usageHref({ days }: Params): string {
if (!days) return UsagePath;
return `${UsagePath}?range=${days}`;
}
src/pages/admin/usage/(hooks)/use-usage-overview.ts
+63
View File
@@ -0,0 +1,63 @@
import fetchApi from "@moduletrace/twui/utils/fetch/fetchApi";
import type { APIResponseObject } from "@moduletrace/bunext/types";
import { useEffect, useState } from "react";
import type { UsageOverview } from "@/src/types";
type Params = {
days: number;
/**
* The totals the page already carries, when it covers this period.
*/
page_usage?: UsageOverview | null;
};
/**
* The usage totals for one period. The page's own copy is used when it
* covers that period; another one loads from the API, so switching the
* range doesn't reload the page.
*/
export default function useUsageOverview({ days, page_usage }: Params) {
const from_page = Boolean(page_usage && page_usage.window.days == days);
const [usage, set_usage] = useState<UsageOverview | null>(from_page ? page_usage || null : null);
const [loading, set_loading] = useState(!from_page);
const [error, set_error] = useState<string>();
useEffect(() => {
if (from_page) {
set_usage(page_usage || null);
set_loading(false);
set_error(undefined);
return;
}
const controller = new AbortController();
const load = async () => {
set_loading(true);
set_error(undefined);
const res = await fetchApi<{}, APIResponseObject<UsageOverview>>(
`/api/admin/usage?days=${days}`,
{ method: "GET" },
).catch(() => null);
if (controller.signal.aborted) return;
set_loading(false);
if (!res?.success || !res.singleRes) {
set_usage(null);
set_error(res?.msg || `Couldn't read the usage.`);
return;
}
set_usage(res.singleRes);
};
load().catch(() => {});
return () => controller.abort();
}, [days, from_page, page_usage]);
return { usage, loading, error };
}
src/pages/admin/usage/(partials)/usage-day-bar.tsx
+39
View File
@@ -0,0 +1,39 @@
import Stack from "@moduletrace/twui/layout/Stack";
import { twMerge } from "tailwind-merge";
import dayjs from "dayjs";
import type { UsageOverview } from "@/src/types";
import formatTokens from "@/src/utils/format-tokens";
import { usageBarHeight } from "../(data)/usage-bar-heights";
type Props = {
day: UsageOverview["days"][number];
max: number;
};
/**
* One day's bar in the chart: its tokens above nothing, the bar itself,
* and the title spells the day out with what it spent. Hovering it shows
* that title.
*/
export default function UsageDayBar({ day, max }: Props) {
const title = `${dayjs(day.at).format("ddd, MMM D")}: ${formatTokens({ tokens: day.total_tokens })} tokens, ${daysCalls({ calls: day.calls })}`;
return (
<Stack
title={title}
className="grow basis-[3px] min-w-[3px] max-w-10 h-24 justify-end items-stretch gap-0!"
>
<Stack
className={twMerge(
"w-full rounded-t-[2px]",
day.total_tokens ? "bg-ink" : "bg-hairline",
usageBarHeight({ value: day.total_tokens, max }),
)}
/>
</Stack>
);
}
function daysCalls({ calls }: { calls: number }) {
return `${calls} ${calls == 1 ? "call" : "calls"}`;
}
src/pages/admin/usage/(partials)/usage-day-chart.tsx
+51
View File
@@ -0,0 +1,51 @@
import Row from "@moduletrace/twui/layout/Row";
import Span from "@moduletrace/twui/layout/Span";
import Stack from "@moduletrace/twui/layout/Stack";
import dayjs from "dayjs";
import type { UsageOverview } from "@/src/types";
import formatTokens from "@/src/utils/format-tokens";
import SettingsGroup from "@/src/pages/admin/settings/(partials)/settings-group";
import UsageDayBar from "./usage-day-bar";
type Props = {
days: UsageOverview["days"];
};
/**
* The period day by day, one bar a day, so a spike or a quiet stretch is
* read at a glance. The busiest day's total labels the top.
*/
export default function UsageDayChart({ days }: Props) {
const max = days.reduce((highest, day) => Math.max(highest, day.total_tokens), 0);
const first = days[0];
const last = days[days.length - 1];
return (
<SettingsGroup
title="Day by day"
description={
max
? `Busiest day: ${formatTokens({ tokens: max })} tokens.`
: "Nothing spent in this period yet."
}
>
<Stack className="w-full gap-2!">
<Row className="w-full gap-1! items-end">
{days.map((day) => (
<UsageDayBar day={day} max={max} key={day.at} />
))}
</Row>
{first && last ? (
<Row className="w-full gap-2! items-center justify-between">
<Span className="text-[11px] leading-4 text-ink-secondary">
{dayjs(first.at).format("MMM D")}
</Span>
<Span className="text-[11px] leading-4 text-ink-secondary">
{dayjs(last.at).format("MMM D")}
</Span>
</Row>
) : null}
</Stack>
</SettingsGroup>
);
}
src/pages/admin/usage/(partials)/usage-harness-breakdown.tsx
+55
View File
@@ -0,0 +1,55 @@
import Center from "@moduletrace/twui/layout/Center";
import Row from "@moduletrace/twui/layout/Row";
import Span from "@moduletrace/twui/layout/Span";
import Stack from "@moduletrace/twui/layout/Stack";
import type { UsageOverview } from "@/src/types";
import SettingsGroup from "@/src/pages/admin/settings/(partials)/settings-group";
import sumTokens from "../(functions)/sum-tokens";
import UsageHarnessRow from "./usage-harness-row";
type Props = {
rows: UsageOverview["by_harness"];
};
/**
* What each kind of model call spent: a chat turn, reading an image,
* transcribing, speaking, making a picture, embedding a file. Their totals
* add up to the period's.
*/
export default function UsageHarnessBreakdown({ rows }: Props) {
if (!rows.length) {
return (
<SettingsGroup title="By kind of call" description="No model calls in this period yet.">
<Center className="p-8">
<Span size="small">Nothing to break down.</Span>
</Center>
</SettingsGroup>
);
}
const total_tokens = sumTokens({ rows });
return (
<SettingsGroup title="By kind of call" description="Where the period's tokens went.">
<Stack className="w-full gap-0!">
<Row className="w-full pb-2 gap-3! items-center border-0 border-b border-solid border-hairline">
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary grow">
Kind
</Span>
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary w-10 text-right">
Share
</Span>
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary w-20 text-right">
Tokens
</Span>
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary w-16 text-right">
Cost
</Span>
</Row>
{rows.map((row) => (
<UsageHarnessRow row={row} total_tokens={total_tokens} key={row.harness} />
))}
</Stack>
</SettingsGroup>
);
}
src/pages/admin/usage/(partials)/usage-harness-row.tsx
+38
View File
@@ -0,0 +1,38 @@
import Row from "@moduletrace/twui/layout/Row";
import Span from "@moduletrace/twui/layout/Span";
import type { UsageOverview } from "@/src/types";
import formatTokens from "@/src/utils/format-tokens";
import { UsageHarnessTitles } from "../(data)/usage-harness-titles";
type Props = {
row: UsageOverview["by_harness"][number];
/**
* Every harness's tokens in the period, for the share.
*/
total_tokens: number;
};
/**
* One harness's line in the breakdown: what it is, its share of the
* period's tokens, and what it spent.
*/
export default function UsageHarnessRow({ row, total_tokens }: Props) {
const share = total_tokens ? Math.round((row.total_tokens / total_tokens) * 100) : 0;
return (
<Row className="w-full py-2 gap-3! items-center border-0 border-b border-solid border-hairline last:border-b-0">
<Span className="text-[13px] leading-5 text-ink grow min-w-0 truncate">
{UsageHarnessTitles[row.harness] || row.harness}
</Span>
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-10 text-right">
{share}%
</Span>
<Span className="text-[13px] leading-5 text-ink tabular-nums w-20 text-right">
{formatTokens({ tokens: row.total_tokens })}
</Span>
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
{row.cost ? `$${row.cost.toFixed(2)}` : "-"}
</Span>
</Row>
);
}
src/pages/admin/usage/(partials)/usage-model-head-cell.tsx
+23
View File
@@ -0,0 +1,23 @@
import Span from "@moduletrace/twui/layout/Span";
import { twMerge } from "tailwind-merge";
type Props = {
title: string;
className?: string;
};
/**
* A column heading of the models table, right-aligned over its figures.
*/
export default function UsageModelHeadCell({ title, className }: Props) {
return (
<Span
className={twMerge(
"text-[11px] uppercase tracking-wide text-ink-secondary text-right shrink-0",
className,
)}
>
{title}
</Span>
);
}
src/pages/admin/usage/(partials)/usage-model-row.tsx
+52
View File
@@ -0,0 +1,52 @@
import Row from "@moduletrace/twui/layout/Row";
import Span from "@moduletrace/twui/layout/Span";
import Stack from "@moduletrace/twui/layout/Stack";
import type { UsageOverview } from "@/src/types";
import formatTokens from "@/src/utils/format-tokens";
import { UsageHarnessTitles } from "../(data)/usage-harness-titles";
type Props = {
row: UsageOverview["models"][number];
};
/**
* One model's line: which provider and model it was, what it was used for,
* how many calls and tokens it took, and the price of a thousand tokens
* worked out from both, which is what makes models comparable.
*/
export default function UsageModelRow({ row }: Props) {
const per_thousand = row.cost && row.total_tokens
? `$${((row.cost / row.total_tokens) * 1000).toFixed(3)}`
: "-";
return (
<Row className="w-full py-2.5 gap-3! items-center border-0 border-b border-solid border-hairline last:border-b-0">
<Stack className="grow min-w-0 gap-0!">
<Span className="text-[13px] leading-5 text-ink truncate" title={row.model_slug}>
{row.model_name || row.model_slug}
</Span>
<Span className="text-[11px] leading-4 text-ink-secondary truncate">
{row.provider_name} - {UsageHarnessTitles[row.harness] || row.harness}
</Span>
</Stack>
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-14 text-right">
{formatTokens({ tokens: row.calls })}
</Span>
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
{formatTokens({ tokens: row.prompt_tokens })}
</Span>
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
{formatTokens({ tokens: row.completion_tokens })}
</Span>
<Span className="text-[13px] leading-5 text-ink tabular-nums w-16 text-right">
{formatTokens({ tokens: row.total_tokens })}
</Span>
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
{per_thousand}
</Span>
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-14 text-right">
{row.cost ? `$${row.cost.toFixed(2)}` : "-"}
</Span>
</Row>
);
}

Some files were not shown because too many files have changed in this diff Show More