Record what every model call spends, and show it
Nothing anywhere recorded what a model call cost: the harness used a provider's usage numbers only to decide whether a turn had finished, then dropped them. So a reply could not say what it took to make, one model could not be compared with another on price, and there was no way to ask what a day, a week or a month of the crew's work came to. - `token_usages` records one row per call: user, bot, workspace, session, job and message, the harness, the provider and model that actually answered, the prompt and completion counts, and the cost when the provider reports one. - `harness-usage-scope.ts` carries those ids in an AsyncLocalStorage scope that `run-bot-job.ts` enters once, so a run's turns, its session summary, its transcriptions, images and embeddings are all attributed without threading ids through every harness function. Nested runs merge rather than replace, and a null genuinely clears a field (a bot's own chat has no workspace). - Work with no run behind it is charged to its owner instead: `run-with-fallback-scope.ts` reaches the workspace's creator else the bot's, and deliberately does not override a real run, so an ingest a bot asked for is not misattributed to the bot's owner. - All six harnesses record through `withCandidates`'s `on_success`. Recording never throws: a model that answered must not lose its answer to a failed insert. - A job's calls land before its message exists, so `complete-bot-job.ts` back-fills `message_id` once the message is inserted. - The message's actions row shows what it cost, beside the timestamp and the play button, with the breakdown in its title. - `/admin/usage` totals the install's spending for 7, 30 or 90 days: the total against the previous period, a day-by-day chart, a split per kind of call, and a table per model with the price per thousand tokens worked out from cost and tokens, which is what makes two models comparable. Crew managers only, since it is the whole install's work rather than one user's conversation. It sits with Routines and Activity in the user popover rather than under Settings, because it is a record of what happened, not something to configure. Days are grouped in local time (`date(created_at / 1000, 'unixepoch', 'localtime')`), matching the local midnights the window builds and the `YYYY-MM-DD` buckets it looks up. Grouping by the bare UTC day put a late evening's calls under a name no bucket carried, so they vanished and the chart's total silently disagreed with the totals card; a test pins it. `canSeeRailItem` now decides who sees a popover or rail entry. The two each filtered with their own inline rule and only one understood `can_view`, so a gated page could be offered in the popover to everyone.
This commit is contained in:
@@ -12,6 +12,7 @@ import GroupTimestamp from "./group-timestamp";
|
||||
import MessageActions from "./message-actions";
|
||||
import MessageReactions from "./message-reactions";
|
||||
import MessageReplyQuote from "./message-reply-quote";
|
||||
import MessageUsageChip from "./message-usage-chip";
|
||||
import NeedsYouChip from "./needs-you-chip";
|
||||
import SpeakMessageButton from "./speak-message-button";
|
||||
|
||||
@@ -74,6 +75,7 @@ function BotMessageLine({ message, bot_name, at }: Props) {
|
||||
<Row className="gap-3! items-center">
|
||||
{message.needs_you ? <NeedsYouChip /> : null}
|
||||
{at ? <GroupTimestamp at={at} /> : null}
|
||||
<MessageUsageChip usage={message.usage} />
|
||||
{can_speak ? (
|
||||
<SpeakMessageButton message_id={Number(message.id)} />
|
||||
) : null}
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import type { TokenUsageSummary } from "@/src/harness/usage/types";
|
||||
import formatTokens from "@/src/utils/format-tokens";
|
||||
|
||||
type Props = {
|
||||
usage?: TokenUsageSummary | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* What one answer cost, in the row under it beside its timestamp: the
|
||||
* tokens every model call behind it spent (a chat turn, reading an image,
|
||||
* making one), and the price when the provider reported it. The hover
|
||||
* title breaks it down.
|
||||
*/
|
||||
export default function MessageUsageChip({ usage }: Props) {
|
||||
if (!usage || usage.total_tokens <= 0) return null;
|
||||
|
||||
const { prompt_tokens, completion_tokens, total_tokens, cost, calls } = usage;
|
||||
const title = [
|
||||
`${formatTokens({ tokens: prompt_tokens })} in, ${formatTokens({ tokens: completion_tokens })} out`,
|
||||
`${calls} model ${calls == 1 ? "call" : "calls"}`,
|
||||
typeof cost == "number" && cost > 0 ? `$${cost.toFixed(4)}` : "",
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join(" · ");
|
||||
|
||||
return (
|
||||
<Span
|
||||
title={title}
|
||||
className="text-[11px] leading-4 text-ink-secondary tabular-nums whitespace-nowrap cursor-help"
|
||||
>
|
||||
{formatTokens({ tokens: total_tokens })} tokens
|
||||
</Span>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
/**
|
||||
* The heights a day's bar can take, shortest first. A day's tokens are
|
||||
* placed on this scale rather than given a height in pixels, so every bar
|
||||
* is a static utility class the stylesheet already carries: no inline
|
||||
* style, and the theme still decides the colour.
|
||||
*/
|
||||
export const UsageBarHeights = [
|
||||
"h-0.5",
|
||||
"h-1",
|
||||
"h-1.5",
|
||||
"h-2",
|
||||
"h-2.5",
|
||||
"h-3",
|
||||
"h-4",
|
||||
"h-5",
|
||||
"h-6",
|
||||
"h-8",
|
||||
"h-10",
|
||||
"h-12",
|
||||
"h-14",
|
||||
"h-16",
|
||||
"h-20",
|
||||
"h-24",
|
||||
];
|
||||
|
||||
/**
|
||||
* Where a value sits on that scale, against the window's busiest day. A
|
||||
* day with work on it is never shorter than one step, so it is visible.
|
||||
*/
|
||||
export function usageBarHeight({ value, max }: { value: number; max: number }) {
|
||||
if (value <= 0) return UsageBarHeights[0]!;
|
||||
|
||||
const ratio = max > 0 ? value / max : 0;
|
||||
const step = Math.max(1, Math.round(ratio * (UsageBarHeights.length - 1)));
|
||||
|
||||
return UsageBarHeights[Math.min(step, UsageBarHeights.length - 1)]!;
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
/**
|
||||
* What each harness is called on the usage page, in words rather than the
|
||||
* harness's own key.
|
||||
*/
|
||||
export const UsageHarnessTitles: Record<string, string> = {
|
||||
text: "Chat and tasks",
|
||||
vision: "Reading media",
|
||||
transcription: "Transcription",
|
||||
speech: "Speech",
|
||||
image: "Image generation",
|
||||
embedding: "Embeddings",
|
||||
};
|
||||
@@ -0,0 +1,27 @@
|
||||
export type UsageRange = {
|
||||
/**
|
||||
* The days back the range totals, today included.
|
||||
*/
|
||||
days: number;
|
||||
title: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* The periods the usage page offers, shortest first. A week and a month
|
||||
* are what most questions need; a quarter is for spotting a trend.
|
||||
*/
|
||||
export const UsageRanges: UsageRange[] = [
|
||||
{ days: 7, title: "7 days" },
|
||||
{ days: 30, title: "30 days" },
|
||||
{ days: 90, title: "90 days" },
|
||||
];
|
||||
|
||||
/**
|
||||
* The range asked for in the address, or the page's default when it isn't
|
||||
* one of the offered ones.
|
||||
*/
|
||||
export function parseUsageRange({ value }: { value?: string | null }): UsageRange | null {
|
||||
const days = Number(value);
|
||||
|
||||
return UsageRanges.find((range) => range.days == days) || null;
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
import type { PagePropsType } from "@/src/types";
|
||||
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
|
||||
import { AdminHomePath } from "@/src/dict/admin-sections-dict";
|
||||
import userAuth from "@/src/functions/backend/auth/user-auth";
|
||||
import grabUsageOverview from "@/src/functions/backend/db/usage/grab-usage-overview";
|
||||
import canManageCrew from "@/src/utils/rbac/can-manage-crew";
|
||||
import { parseUsageRange } from "../(data)/usage-ranges";
|
||||
|
||||
type Params = Parameters<BunextPageServerFn<PagePropsType>>[0];
|
||||
|
||||
/**
|
||||
* The usage page's data for the period in the address (the default when
|
||||
* it carries none). Crew managers only, since it is the whole install's
|
||||
* work rather than one user's conversation; everyone else lands on the
|
||||
* bots.
|
||||
*/
|
||||
export default async function grabUsageServerProps(params: Params) {
|
||||
const { user, user_types } = await userAuth({ req: params.req });
|
||||
|
||||
if (!user?.id) {
|
||||
return { redirect: { destination: "/auth/login" } };
|
||||
}
|
||||
|
||||
if (!canManageCrew({ user_types })) {
|
||||
return { redirect: { destination: AdminHomePath } };
|
||||
}
|
||||
|
||||
const range = parseUsageRange({ value: params.query?.range });
|
||||
|
||||
return {
|
||||
props: {
|
||||
usage: await grabUsageOverview({ days: range?.days }).catch(() => null),
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
import type { TokenUsageSummary } from "@/src/harness/usage/types";
|
||||
|
||||
type Params = {
|
||||
rows: TokenUsageSummary[];
|
||||
};
|
||||
|
||||
/**
|
||||
* The tokens every row of a breakdown spent together, which is the
|
||||
* denominator of its shares.
|
||||
*/
|
||||
export default function sumTokens({ rows }: Params) {
|
||||
return rows.reduce((total, row) => total + row.total_tokens, 0);
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
import { UsagePath } from "@/src/dict/rail-pages-dict";
|
||||
|
||||
type Params = {
|
||||
days?: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* The usage page's address for the period it is showing, so a range can be
|
||||
* linked to and the browser's own back button walks back through them.
|
||||
*/
|
||||
export default function usageHref({ days }: Params): string {
|
||||
if (!days) return UsagePath;
|
||||
|
||||
return `${UsagePath}?range=${days}`;
|
||||
}
|
||||
@@ -0,0 +1,63 @@
|
||||
import fetchApi from "@moduletrace/twui/utils/fetch/fetchApi";
|
||||
import type { APIResponseObject } from "@moduletrace/bunext/types";
|
||||
import { useEffect, useState } from "react";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
days: number;
|
||||
/**
|
||||
* The totals the page already carries, when it covers this period.
|
||||
*/
|
||||
page_usage?: UsageOverview | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* The usage totals for one period. The page's own copy is used when it
|
||||
* covers that period; another one loads from the API, so switching the
|
||||
* range doesn't reload the page.
|
||||
*/
|
||||
export default function useUsageOverview({ days, page_usage }: Params) {
|
||||
const from_page = Boolean(page_usage && page_usage.window.days == days);
|
||||
const [usage, set_usage] = useState<UsageOverview | null>(from_page ? page_usage || null : null);
|
||||
const [loading, set_loading] = useState(!from_page);
|
||||
const [error, set_error] = useState<string>();
|
||||
|
||||
useEffect(() => {
|
||||
if (from_page) {
|
||||
set_usage(page_usage || null);
|
||||
set_loading(false);
|
||||
set_error(undefined);
|
||||
return;
|
||||
}
|
||||
|
||||
const controller = new AbortController();
|
||||
|
||||
const load = async () => {
|
||||
set_loading(true);
|
||||
set_error(undefined);
|
||||
|
||||
const res = await fetchApi<{}, APIResponseObject<UsageOverview>>(
|
||||
`/api/admin/usage?days=${days}`,
|
||||
{ method: "GET" },
|
||||
).catch(() => null);
|
||||
|
||||
if (controller.signal.aborted) return;
|
||||
|
||||
set_loading(false);
|
||||
|
||||
if (!res?.success || !res.singleRes) {
|
||||
set_usage(null);
|
||||
set_error(res?.msg || `Couldn't read the usage.`);
|
||||
return;
|
||||
}
|
||||
|
||||
set_usage(res.singleRes);
|
||||
};
|
||||
|
||||
load().catch(() => {});
|
||||
|
||||
return () => controller.abort();
|
||||
}, [days, from_page, page_usage]);
|
||||
|
||||
return { usage, loading, error };
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import dayjs from "dayjs";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
import formatTokens from "@/src/utils/format-tokens";
|
||||
import { usageBarHeight } from "../(data)/usage-bar-heights";
|
||||
|
||||
type Props = {
|
||||
day: UsageOverview["days"][number];
|
||||
max: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* One day's bar in the chart: its tokens above nothing, the bar itself,
|
||||
* and the title spells the day out with what it spent. Hovering it shows
|
||||
* that title.
|
||||
*/
|
||||
export default function UsageDayBar({ day, max }: Props) {
|
||||
const title = `${dayjs(day.at).format("ddd, MMM D")}: ${formatTokens({ tokens: day.total_tokens })} tokens, ${daysCalls({ calls: day.calls })}`;
|
||||
|
||||
return (
|
||||
<Stack
|
||||
title={title}
|
||||
className="grow basis-[3px] min-w-[3px] max-w-10 h-24 justify-end items-stretch gap-0!"
|
||||
>
|
||||
<Stack
|
||||
className={twMerge(
|
||||
"w-full rounded-t-[2px]",
|
||||
day.total_tokens ? "bg-ink" : "bg-hairline",
|
||||
usageBarHeight({ value: day.total_tokens, max }),
|
||||
)}
|
||||
/>
|
||||
</Stack>
|
||||
);
|
||||
}
|
||||
|
||||
function daysCalls({ calls }: { calls: number }) {
|
||||
return `${calls} ${calls == 1 ? "call" : "calls"}`;
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import dayjs from "dayjs";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
import formatTokens from "@/src/utils/format-tokens";
|
||||
import SettingsGroup from "@/src/pages/admin/settings/(partials)/settings-group";
|
||||
import UsageDayBar from "./usage-day-bar";
|
||||
|
||||
type Props = {
|
||||
days: UsageOverview["days"];
|
||||
};
|
||||
|
||||
/**
|
||||
* The period day by day, one bar a day, so a spike or a quiet stretch is
|
||||
* read at a glance. The busiest day's total labels the top.
|
||||
*/
|
||||
export default function UsageDayChart({ days }: Props) {
|
||||
const max = days.reduce((highest, day) => Math.max(highest, day.total_tokens), 0);
|
||||
const first = days[0];
|
||||
const last = days[days.length - 1];
|
||||
|
||||
return (
|
||||
<SettingsGroup
|
||||
title="Day by day"
|
||||
description={
|
||||
max
|
||||
? `Busiest day: ${formatTokens({ tokens: max })} tokens.`
|
||||
: "Nothing spent in this period yet."
|
||||
}
|
||||
>
|
||||
<Stack className="w-full gap-2!">
|
||||
<Row className="w-full gap-1! items-end">
|
||||
{days.map((day) => (
|
||||
<UsageDayBar day={day} max={max} key={day.at} />
|
||||
))}
|
||||
</Row>
|
||||
{first && last ? (
|
||||
<Row className="w-full gap-2! items-center justify-between">
|
||||
<Span className="text-[11px] leading-4 text-ink-secondary">
|
||||
{dayjs(first.at).format("MMM D")}
|
||||
</Span>
|
||||
<Span className="text-[11px] leading-4 text-ink-secondary">
|
||||
{dayjs(last.at).format("MMM D")}
|
||||
</Span>
|
||||
</Row>
|
||||
) : null}
|
||||
</Stack>
|
||||
</SettingsGroup>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import Center from "@moduletrace/twui/layout/Center";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
import SettingsGroup from "@/src/pages/admin/settings/(partials)/settings-group";
|
||||
import sumTokens from "../(functions)/sum-tokens";
|
||||
import UsageHarnessRow from "./usage-harness-row";
|
||||
|
||||
type Props = {
|
||||
rows: UsageOverview["by_harness"];
|
||||
};
|
||||
|
||||
/**
|
||||
* What each kind of model call spent: a chat turn, reading an image,
|
||||
* transcribing, speaking, making a picture, embedding a file. Their totals
|
||||
* add up to the period's.
|
||||
*/
|
||||
export default function UsageHarnessBreakdown({ rows }: Props) {
|
||||
if (!rows.length) {
|
||||
return (
|
||||
<SettingsGroup title="By kind of call" description="No model calls in this period yet.">
|
||||
<Center className="p-8">
|
||||
<Span size="small">Nothing to break down.</Span>
|
||||
</Center>
|
||||
</SettingsGroup>
|
||||
);
|
||||
}
|
||||
|
||||
const total_tokens = sumTokens({ rows });
|
||||
|
||||
return (
|
||||
<SettingsGroup title="By kind of call" description="Where the period's tokens went.">
|
||||
<Stack className="w-full gap-0!">
|
||||
<Row className="w-full pb-2 gap-3! items-center border-0 border-b border-solid border-hairline">
|
||||
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary grow">
|
||||
Kind
|
||||
</Span>
|
||||
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary w-10 text-right">
|
||||
Share
|
||||
</Span>
|
||||
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary w-20 text-right">
|
||||
Tokens
|
||||
</Span>
|
||||
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary w-16 text-right">
|
||||
Cost
|
||||
</Span>
|
||||
</Row>
|
||||
{rows.map((row) => (
|
||||
<UsageHarnessRow row={row} total_tokens={total_tokens} key={row.harness} />
|
||||
))}
|
||||
</Stack>
|
||||
</SettingsGroup>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
import formatTokens from "@/src/utils/format-tokens";
|
||||
import { UsageHarnessTitles } from "../(data)/usage-harness-titles";
|
||||
|
||||
type Props = {
|
||||
row: UsageOverview["by_harness"][number];
|
||||
/**
|
||||
* Every harness's tokens in the period, for the share.
|
||||
*/
|
||||
total_tokens: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* One harness's line in the breakdown: what it is, its share of the
|
||||
* period's tokens, and what it spent.
|
||||
*/
|
||||
export default function UsageHarnessRow({ row, total_tokens }: Props) {
|
||||
const share = total_tokens ? Math.round((row.total_tokens / total_tokens) * 100) : 0;
|
||||
|
||||
return (
|
||||
<Row className="w-full py-2 gap-3! items-center border-0 border-b border-solid border-hairline last:border-b-0">
|
||||
<Span className="text-[13px] leading-5 text-ink grow min-w-0 truncate">
|
||||
{UsageHarnessTitles[row.harness] || row.harness}
|
||||
</Span>
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-10 text-right">
|
||||
{share}%
|
||||
</Span>
|
||||
<Span className="text-[13px] leading-5 text-ink tabular-nums w-20 text-right">
|
||||
{formatTokens({ tokens: row.total_tokens })}
|
||||
</Span>
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
|
||||
{row.cost ? `$${row.cost.toFixed(2)}` : "-"}
|
||||
</Span>
|
||||
</Row>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
|
||||
type Props = {
|
||||
title: string;
|
||||
className?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* A column heading of the models table, right-aligned over its figures.
|
||||
*/
|
||||
export default function UsageModelHeadCell({ title, className }: Props) {
|
||||
return (
|
||||
<Span
|
||||
className={twMerge(
|
||||
"text-[11px] uppercase tracking-wide text-ink-secondary text-right shrink-0",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
{title}
|
||||
</Span>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
import formatTokens from "@/src/utils/format-tokens";
|
||||
import { UsageHarnessTitles } from "../(data)/usage-harness-titles";
|
||||
|
||||
type Props = {
|
||||
row: UsageOverview["models"][number];
|
||||
};
|
||||
|
||||
/**
|
||||
* One model's line: which provider and model it was, what it was used for,
|
||||
* how many calls and tokens it took, and the price of a thousand tokens
|
||||
* worked out from both, which is what makes models comparable.
|
||||
*/
|
||||
export default function UsageModelRow({ row }: Props) {
|
||||
const per_thousand = row.cost && row.total_tokens
|
||||
? `$${((row.cost / row.total_tokens) * 1000).toFixed(3)}`
|
||||
: "-";
|
||||
|
||||
return (
|
||||
<Row className="w-full py-2.5 gap-3! items-center border-0 border-b border-solid border-hairline last:border-b-0">
|
||||
<Stack className="grow min-w-0 gap-0!">
|
||||
<Span className="text-[13px] leading-5 text-ink truncate" title={row.model_slug}>
|
||||
{row.model_name || row.model_slug}
|
||||
</Span>
|
||||
<Span className="text-[11px] leading-4 text-ink-secondary truncate">
|
||||
{row.provider_name} - {UsageHarnessTitles[row.harness] || row.harness}
|
||||
</Span>
|
||||
</Stack>
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-14 text-right">
|
||||
{formatTokens({ tokens: row.calls })}
|
||||
</Span>
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
|
||||
{formatTokens({ tokens: row.prompt_tokens })}
|
||||
</Span>
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
|
||||
{formatTokens({ tokens: row.completion_tokens })}
|
||||
</Span>
|
||||
<Span className="text-[13px] leading-5 text-ink tabular-nums w-16 text-right">
|
||||
{formatTokens({ tokens: row.total_tokens })}
|
||||
</Span>
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-16 text-right">
|
||||
{per_thousand}
|
||||
</Span>
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary tabular-nums w-14 text-right">
|
||||
{row.cost ? `$${row.cost.toFixed(2)}` : "-"}
|
||||
</Span>
|
||||
</Row>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
import Center from "@moduletrace/twui/layout/Center";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
import SettingsGroup from "@/src/pages/admin/settings/(partials)/settings-group";
|
||||
import UsageModelHeadCell from "./usage-model-head-cell";
|
||||
import UsageModelRow from "./usage-model-row";
|
||||
|
||||
type Props = {
|
||||
rows: UsageOverview["models"];
|
||||
days: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* Every model the period ran, busiest first. A row's price per thousand
|
||||
* tokens is what makes one model's cost comparable with another's, and
|
||||
* how many calls each took says which one earned it.
|
||||
*/
|
||||
export default function UsageModelsTable({ rows, days }: Props) {
|
||||
if (!rows.length) {
|
||||
return (
|
||||
<SettingsGroup title="By model" description="No model calls in this period yet.">
|
||||
<Center className="p-8">
|
||||
<Span size="small">Nothing to list.</Span>
|
||||
</Center>
|
||||
</SettingsGroup>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<SettingsGroup
|
||||
title="By model"
|
||||
description={`The models this install ran in the last ${days} days, busiest first. The price per thousand tokens is read from what the provider reported.`}
|
||||
>
|
||||
<Stack className="w-full gap-0! overflow-x-auto">
|
||||
<Row className="w-full pb-2 gap-3! items-center border-0 border-b border-solid border-hairline min-w-[640px]">
|
||||
<UsageModelHeadCell className="grow text-left" title="Model" />
|
||||
<UsageModelHeadCell className="w-14" title="Calls" />
|
||||
<UsageModelHeadCell className="w-16" title="In" />
|
||||
<UsageModelHeadCell className="w-16" title="Out" />
|
||||
<UsageModelHeadCell className="w-16" title="Tokens" />
|
||||
<UsageModelHeadCell className="w-16" title="Per 1K" />
|
||||
<UsageModelHeadCell className="w-14" title="Cost" />
|
||||
</Row>
|
||||
{rows.map((row) => (
|
||||
<UsageModelRow
|
||||
row={row}
|
||||
key={`${row.provider_name}-${row.model_slug}-${row.harness}`}
|
||||
/>
|
||||
))}
|
||||
</Stack>
|
||||
</SettingsGroup>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
import Button from "@moduletrace/twui/layout/Button";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import { UsageRanges } from "../(data)/usage-ranges";
|
||||
|
||||
type Props = {
|
||||
days: number;
|
||||
loading?: boolean;
|
||||
onChange: (days: number) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* The periods the page totals, one button each. The one showing is the
|
||||
* one that reads as pressed.
|
||||
*/
|
||||
export default function UsageRangePicker({ days, loading, onChange }: Props) {
|
||||
return (
|
||||
<Row className="gap-1! items-center">
|
||||
{UsageRanges.map((range) => (
|
||||
<Button
|
||||
key={range.days}
|
||||
title={range.title}
|
||||
type="button"
|
||||
size="small"
|
||||
variant={range.days == days ? "outlined" : "ghost"}
|
||||
color="text"
|
||||
disabled={loading}
|
||||
onClick={() => onChange(range.days)}
|
||||
className="px-2.5! py-1! text-[12px]! rounded-default-xs"
|
||||
>
|
||||
{range.title}
|
||||
</Button>
|
||||
))}
|
||||
</Row>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
|
||||
type Props = {
|
||||
label: string;
|
||||
value: string;
|
||||
/**
|
||||
* A line under the figure, e.g. the comparison with an earlier period.
|
||||
*/
|
||||
hint?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* One figure of the page's totals: what it is above, the number itself
|
||||
* big, and a quiet line under it.
|
||||
*/
|
||||
export default function UsageStat({ label, value, hint }: Props) {
|
||||
return (
|
||||
<Stack className="grow basis-[160px] min-w-0 gap-1!">
|
||||
<Span className="text-[11px] uppercase tracking-wide text-ink-secondary">{label}</Span>
|
||||
<Span className="text-2xl leading-8 font-bold text-ink tabular-nums">{value}</Span>
|
||||
{hint ? (
|
||||
<Span className="text-[12px] leading-4 text-ink-secondary">{hint}</Span>
|
||||
) : null}
|
||||
</Stack>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import type { UsageOverview } from "@/src/types";
|
||||
import formatTokens from "@/src/utils/format-tokens";
|
||||
import { UsageHarnessTitles } from "../(data)/usage-harness-titles";
|
||||
import UsageStat from "./usage-stat";
|
||||
|
||||
type Props = {
|
||||
usage: UsageOverview;
|
||||
};
|
||||
|
||||
/**
|
||||
* The page's headline figures for the period: what it spent, how many
|
||||
* calls that took, the price when providers report it, the busiest
|
||||
* harness, and how the total compares with the period before it.
|
||||
*/
|
||||
export default function UsageTotals({ usage }: Props) {
|
||||
const { totals, previous, by_harness } = usage;
|
||||
const busiest = by_harness[0];
|
||||
|
||||
return (
|
||||
<Row className="w-full py-8 gap-6! items-start flex-wrap border-0 border-b border-solid border-hairline">
|
||||
<UsageStat
|
||||
label="Tokens"
|
||||
value={formatTokens({ tokens: totals.total_tokens })}
|
||||
hint={`${formatTokens({ tokens: totals.prompt_tokens })} in, ${formatTokens({ tokens: totals.completion_tokens })} out`}
|
||||
/>
|
||||
<UsageStat
|
||||
label="Model calls"
|
||||
value={formatTokens({ tokens: totals.calls })}
|
||||
hint={busiest ? `${UsageHarnessTitles[busiest.harness] || busiest.harness} most of it` : undefined}
|
||||
/>
|
||||
<UsageStat
|
||||
label="Cost"
|
||||
value={totals.cost ? `$${totals.cost.toFixed(2)}` : "Not reported"}
|
||||
hint={totals.cost ? "As the providers reported it" : "Free models, or no price"}
|
||||
/>
|
||||
<UsageStat
|
||||
label="Previous period"
|
||||
value={changeLabel({ current: totals.total_tokens, previous: previous.total_tokens })}
|
||||
hint={`${formatTokens({ tokens: previous.total_tokens })} then`}
|
||||
/>
|
||||
</Row>
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* How this period compares with the one before it: a share up or down, or
|
||||
* that there is nothing to compare with yet.
|
||||
*/
|
||||
function changeLabel({ current, previous }: { current: number; previous: number }) {
|
||||
if (!previous) return current ? "New" : "Nothing yet";
|
||||
|
||||
const change = Math.round(((current - previous) / previous) * 100);
|
||||
|
||||
if (change == 0) return "Level";
|
||||
|
||||
return `${change > 0 ? "+" : ""}${change}%`;
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
import Center from "@moduletrace/twui/layout/Center";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import { useContext, useState } from "react";
|
||||
import { AppContext } from "@/src/pages/__root";
|
||||
import { UsageRanges } from "../(data)/usage-ranges";
|
||||
import usageHref from "../(functions)/usage-href";
|
||||
import useUsageOverview from "../(hooks)/use-usage-overview";
|
||||
import UsageDayChart from "../(partials)/usage-day-chart";
|
||||
import UsageHarnessBreakdown from "../(partials)/usage-harness-breakdown";
|
||||
import UsageModelsTable from "../(partials)/usage-models-table";
|
||||
import UsageRangePicker from "../(partials)/usage-range-picker";
|
||||
import UsageTotals from "../(partials)/usage-totals";
|
||||
|
||||
/**
|
||||
* The usage page's state: which period is totalled, kept in the address so
|
||||
* a range can be linked to. The default period comes with the page;
|
||||
* another one loads through the API.
|
||||
*/
|
||||
export default function UsageSection() {
|
||||
const { pageProps } = useContext(AppContext);
|
||||
const page_usage = pageProps?.usage;
|
||||
const [days, set_days] = useState(page_usage?.window.days || UsageRanges[1]!.days);
|
||||
const { usage, loading, error } = useUsageOverview({ days, page_usage });
|
||||
|
||||
const changeRange = (next_days: number) => {
|
||||
set_days(next_days);
|
||||
|
||||
const url = new URL(usageHref({ days: next_days }), window.location.origin);
|
||||
|
||||
history.pushState(null, "", url);
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<Row className="w-full pt-6 gap-4! items-center justify-between flex-wrap">
|
||||
<Span className="text-[13px] leading-[18px] text-ink-secondary">
|
||||
Every model call the crew made, whether it answered in chat, read an image,
|
||||
spoke or made one.
|
||||
</Span>
|
||||
<UsageRangePicker days={days} loading={loading} onChange={changeRange} />
|
||||
</Row>
|
||||
{error ? (
|
||||
<Center className="p-16">
|
||||
<Span size="small">{error}</Span>
|
||||
</Center>
|
||||
) : !usage ? (
|
||||
<Center className="p-16">
|
||||
<Span size="small">Reading the usage...</Span>
|
||||
</Center>
|
||||
) : (
|
||||
<UsageTotals usage={usage} />
|
||||
)}
|
||||
{usage ? <UsageDayChart days={usage.days} /> : null}
|
||||
{usage ? <UsageHarnessBreakdown rows={usage.by_harness} /> : null}
|
||||
{usage ? <UsageModelsTable rows={usage.models} days={usage.window.days} /> : null}
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
import type { PagePropsType } from "@/src/types";
|
||||
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
|
||||
import grabUsageServerProps from "./(functions)/grab-usage-server-props";
|
||||
|
||||
export const server: BunextPageServerFn<PagePropsType> = async (params) => {
|
||||
return await grabUsageServerProps(params);
|
||||
};
|
||||
@@ -0,0 +1,23 @@
|
||||
import type { BunextPageModuleMetaFn } from "@moduletrace/bunext/types";
|
||||
import { SiteData } from "@/src/data/site-data";
|
||||
import SettingsShell from "@/src/pages/admin/settings/(partials)/settings-shell";
|
||||
import UsageSection from "./(sections)/usage-section";
|
||||
|
||||
export default function AdminUsagePage() {
|
||||
return (
|
||||
<SettingsShell
|
||||
title="Usage"
|
||||
description="What the crew's model calls spent: in total, day by day, and per model, so a model's cost and the work of a day are both readable."
|
||||
wide
|
||||
>
|
||||
<UsageSection />
|
||||
</SettingsShell>
|
||||
);
|
||||
}
|
||||
|
||||
export const meta: BunextPageModuleMetaFn = async () => {
|
||||
return {
|
||||
title: `Usage | ${SiteData["SiteName"]}`,
|
||||
description: "What the crew's model calls spent.",
|
||||
};
|
||||
};
|
||||
Reference in New Issue
Block a user