Quiet the logs and capture the errors that were missing
- Activity page: "Delete all" empties the open tab (api/admin/logs/clear-logs, super admin only, logged under system) - Stop logging every WebSocket connect - Log unexpected API errors (safeErrorMessage -> logUnexpectedError), every failed bot job attempt and every job the queue lost - Drop the cron jobs' console output; a failed model sync goes to the logs Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
Claude Opus 5.5
@@ -77,7 +77,7 @@ Inspect live data with `bun llm-code-assistant/query-db.ts "<sql>"` - a read-onl
|
||||
- **Auth layout:** `/auth/*` (except `/auth/logout`) renders in `src/layouts/auth/` with no site header/footer.
|
||||
- **User types** (`UserTypes` in `src/dict/user-types-dict.ts`, the only place they're defined): `super_admin` (the first account, runs the install), `admin` (manages members; only the super admin grants it), `member` (a regular user: new users start with it), `read_only`, `revoked` (can't sign in). A user needs at least one to sign in. Admins grant and remove `member`, `read_only` and `revoked` on anyone but admins and the super admin.
|
||||
- **RBAC:** `src/utils/check-user-access.ts` (includes/excludes primitive, `super_admin` bypass) plus the declarative engine in `src/utils/rbac/` - `userTypeAssignment` interprets `UserTypeAssignmentMatrix`/`UserTypeDeletionMatrix` in `src/dict/user-types-dict.ts`. `read_only` users are blocked from non-GET CRUD in the router.
|
||||
- **Audit trail:** `src/functions/backend/logger.ts` - `logEvent`/`logError` write to the `logs` table. Signup, login, Google login, logout, cron runs, and WebSocket connects are all logged.
|
||||
- **Audit trail:** `src/functions/backend/logger.ts` - `logEvent`/`logError` write to the `logs` table. Signup, login, Google login, logout and cron runs are logged (WebSocket connects are not: one per tab and reconnect drowned the page). Errors land under `error`: every unexpected one an API route catches (`safeErrorMessage` calls `logUnexpectedError` for anything but an `AppError`, skipped under tests; pass `log: false` where the caller already logs it), every failed bot job attempt with its bot, kind and try (`processBotJob`), and every job the queue lost (`reportLostBotJobs`). The Activity page's "Delete all" empties the open tab (`api/admin/logs/clear-logs`, super admin only, logged under `system`).
|
||||
|
||||
### WebSocket
|
||||
|
||||
|
||||
@@ -290,6 +290,38 @@ export default async function processBotJob({
|
||||
return null;
|
||||
});
|
||||
|
||||
/**
|
||||
* Every failed try is on the Activity page, so a model or provider
|
||||
* that keeps failing shows there and not only in one chat.
|
||||
*/
|
||||
if (failed) {
|
||||
const final = failed.status == "failed";
|
||||
|
||||
await logError({
|
||||
title: final
|
||||
? `Bot job #${job_id} failed`
|
||||
: `Bot job #${job_id} failed, retrying (try ${tries})`,
|
||||
error: timed_out ? new Error(timeout_message) : error,
|
||||
user_id: user_id || undefined,
|
||||
details: [
|
||||
{ label: "Bot", value: bot_id },
|
||||
{ label: "Kind", value: job.kind },
|
||||
{ label: "Title", value: job.title || "" },
|
||||
{ label: "Try", value: tries },
|
||||
],
|
||||
metadata: {
|
||||
job_id,
|
||||
bot_id,
|
||||
kind: job.kind,
|
||||
session_id: job.session_id,
|
||||
workspace_id: job.workspace_id,
|
||||
schedule_id,
|
||||
fatal: error instanceof BotJobFatalError,
|
||||
harness_code: error instanceof HarnessError ? error.code : undefined,
|
||||
},
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
if (failed?.notice) {
|
||||
await pushBotNotice({ bot_id, ...failed.notice });
|
||||
}
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
import grabUnreportedFailedBotJobs from "@/src/functions/backend/db/bots-queue/grab-unreported-failed-bot-jobs";
|
||||
import notifyBotJobFailed from "@/src/functions/backend/db/bots-queue/notify-bot-job-failed";
|
||||
import { logError } from "@/src/functions/backend/logger";
|
||||
import pushBotNotice from "./push-bot-notice";
|
||||
|
||||
/**
|
||||
* Tells the user about jobs that ran out of attempts by going silent
|
||||
* (bun-sqlite fails those itself, without a message).
|
||||
* (bun-sqlite fails those itself, without a message), and logs each.
|
||||
*/
|
||||
export default async function reportLostBotJobs() {
|
||||
const jobs = await grabUnreportedFailedBotJobs();
|
||||
@@ -18,5 +19,16 @@ export default async function reportLostBotJobs() {
|
||||
});
|
||||
|
||||
await pushBotNotice({ bot_id: job.bot_id, ...notice });
|
||||
|
||||
await logError({
|
||||
title: `Bot job #${job.id} lost`,
|
||||
error: job.last_error || "The job's lock timed out.",
|
||||
details: [
|
||||
{ label: "Bot", value: job.bot_id },
|
||||
{ label: "Kind", value: job.kind },
|
||||
{ label: "Title", value: job.title || "" },
|
||||
],
|
||||
metadata: { job_id: job.id, bot_id: job.bot_id, kind: job.kind },
|
||||
}).catch(() => {});
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import bunext from "@moduletrace/bunext";
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
|
||||
@@ -15,9 +14,5 @@ export default async function pruneBotsQueue() {
|
||||
values: [cutoff],
|
||||
});
|
||||
|
||||
bunext.bunextLog.info(
|
||||
`Cron: pruned ${res.payload?.length || 0} finished bot job(s)`,
|
||||
);
|
||||
|
||||
return res;
|
||||
}
|
||||
|
||||
@@ -1,23 +1,23 @@
|
||||
import bunext from "@moduletrace/bunext";
|
||||
import syncAllHarnessModels from "@/src/functions/backend/db/harness/sync-all-harness-models";
|
||||
import { logError } from "@/src/functions/backend/logger";
|
||||
|
||||
/**
|
||||
* Refreshes every enabled provider's model list. Free models on OpenRouter
|
||||
* come and go, and automatic picks follow them from this list.
|
||||
* come and go, and automatic picks follow them from this list. A provider
|
||||
* that fails doesn't fail the run, so its error goes to the logs here.
|
||||
*/
|
||||
export default async function cronSyncHarnessModels() {
|
||||
const results = await syncAllHarnessModels();
|
||||
|
||||
for (const result of results) {
|
||||
if (result.error) {
|
||||
bunext.bunextLog.error(
|
||||
`Cron: model sync for provider #${result.provider_id} failed: ${result.error}`,
|
||||
);
|
||||
} else {
|
||||
bunext.bunextLog.info(
|
||||
`Cron: provider #${result.provider_id} lists ${result.count} model(s)`,
|
||||
);
|
||||
}
|
||||
if (!result.error) continue;
|
||||
|
||||
await logError({
|
||||
title: `Cron: model sync for provider #${result.provider_id} failed`,
|
||||
category: "cron",
|
||||
error: result.error,
|
||||
metadata: { provider_id: result.provider_id },
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
return results;
|
||||
|
||||
@@ -87,7 +87,7 @@ export default async function fireCronJob({ cron_job, now = Date.now() }: Params
|
||||
metadata: { cron_job_id: id, job_id, bot_id: cron_job.bot_id },
|
||||
});
|
||||
} catch (error) {
|
||||
await saveRun({ id, last_error: safeErrorMessage({ error, fallback: `Couldn't queue the run.` }) });
|
||||
await saveRun({ id, last_error: safeErrorMessage({ error, fallback: `Couldn't queue the run.`, log: false }) });
|
||||
|
||||
await logError({
|
||||
title: `Schedule #${id} couldn't run`,
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import { logError } from "./logger";
|
||||
|
||||
type Params = {
|
||||
error: unknown;
|
||||
};
|
||||
|
||||
/**
|
||||
* Writes an error nobody meant to show (anything but an `AppError`) to the
|
||||
* logs, so a failure the client only sees as "Something went wrong" is
|
||||
* still on the Activity page with its stack. Never during tests, which
|
||||
* throw plain errors on purpose, and never throws.
|
||||
*/
|
||||
export default function logUnexpectedError({ error }: Params) {
|
||||
if (process.env.NODE_ENV == "test") return;
|
||||
|
||||
const name = error instanceof Error ? error.name : typeof error;
|
||||
|
||||
void logError({
|
||||
title: `Unexpected server error (${name})`,
|
||||
error,
|
||||
}).catch(() => {});
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import fetchApi from "@moduletrace/twui/utils/fetch/fetchApi";
|
||||
import type { APIResponseObject } from "@moduletrace/bunext/types";
|
||||
import type { ClearLogsRequest } from "@/src/schemas/log-schema";
|
||||
|
||||
type Params = {
|
||||
category?: ClearLogsRequest["category"];
|
||||
/**
|
||||
* The tab's name, for the confirmation.
|
||||
*/
|
||||
title: string;
|
||||
count?: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* Deletes every log in the open tab, after asking.
|
||||
*/
|
||||
export default async function clearLogs({ category, title, count }: Params) {
|
||||
const amount = typeof count == "number" ? `${count} ` : "";
|
||||
const confirm_msg = category
|
||||
? `Delete all ${amount}${title} logs? This cannot be undone.`
|
||||
: `Delete all ${amount}logs? This cannot be undone.`;
|
||||
|
||||
if (!window.confirm(confirm_msg)) {
|
||||
return { success: false, cancelled: true };
|
||||
}
|
||||
|
||||
const res = await fetchApi<ClearLogsRequest, APIResponseObject<{ deleted: number }>>(
|
||||
`/api/admin/logs/clear-logs`,
|
||||
{ method: "POST", body: { category } },
|
||||
);
|
||||
|
||||
return {
|
||||
success: Boolean(res?.success),
|
||||
msg: res?.success ? undefined : res?.msg || `Couldn't delete logs`,
|
||||
};
|
||||
}
|
||||
@@ -78,6 +78,20 @@ export default function useLogsListInit(_props?: Props) {
|
||||
setRes(undefined);
|
||||
};
|
||||
|
||||
/**
|
||||
* The tab was emptied: back to its first page, reloaded.
|
||||
*/
|
||||
const afterClear = () => {
|
||||
setSelectedIds([]);
|
||||
|
||||
if (page > 1) {
|
||||
setPage(1);
|
||||
return;
|
||||
}
|
||||
|
||||
setRes(undefined);
|
||||
};
|
||||
|
||||
const all_selected =
|
||||
Boolean(logs?.[0]) && selected_ids.length == logs?.length;
|
||||
|
||||
@@ -100,6 +114,7 @@ export default function useLogsListInit(_props?: Props) {
|
||||
toggleSelectAll,
|
||||
all_selected,
|
||||
afterDelete,
|
||||
afterClear,
|
||||
active_log,
|
||||
openLog,
|
||||
log_modal_open,
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
import { Trash2 } from "lucide-react";
|
||||
import Button from "@moduletrace/twui/layout/Button";
|
||||
import Tag from "@moduletrace/twui/elements/Tag";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import useStatus from "@moduletrace/twui/hooks/useStatus";
|
||||
import { LogCategoryFilters } from "../(data)/log-category-filters";
|
||||
import type { LogCategoryValue } from "../(data)/log-category-styles";
|
||||
import clearLogs from "../(functions)/clear-logs";
|
||||
|
||||
type Props = {
|
||||
category?: LogCategoryValue;
|
||||
count?: number;
|
||||
afterClear: () => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* Deletes every log in the open tab ("All" deletes every log).
|
||||
*/
|
||||
export default function ClearLogsButton({ category, count, afterClear }: Props) {
|
||||
const { loading, setLoading, status, setStatus } = useStatus();
|
||||
|
||||
const title = LogCategoryFilters.find((filter) => filter.value == category)?.title || "All";
|
||||
|
||||
const clear_logs = async () => {
|
||||
setStatus(undefined);
|
||||
setLoading(true);
|
||||
|
||||
const res = await clearLogs({ category, title, count });
|
||||
|
||||
setLoading(false);
|
||||
|
||||
if (res.success) {
|
||||
afterClear();
|
||||
} else if (res.msg) {
|
||||
setStatus({ error: true, msg: res.msg });
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<Row className="gap-2! flex-nowrap">
|
||||
{status?.error && <Tag color="error">{status.msg}</Tag>}
|
||||
<Button
|
||||
title={category ? `Delete all ${title} logs` : "Delete all logs"}
|
||||
variant="ghost"
|
||||
color="error"
|
||||
size="small"
|
||||
loading={loading}
|
||||
disabled={!count}
|
||||
onClick={() => {
|
||||
clear_logs();
|
||||
}}
|
||||
>
|
||||
<Row className="gap-2! flex-nowrap">
|
||||
<Trash2 size={15} />
|
||||
<Span>{category ? `Delete all ${title}` : "Delete all"}</Span>
|
||||
</Row>
|
||||
</Button>
|
||||
</Row>
|
||||
);
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import type useLogsListInit from "../(hooks)/use-logs-list-init";
|
||||
import ClearLogsButton from "./clear-logs-button";
|
||||
import DeleteLogsButton from "./delete-logs-button";
|
||||
import LogCategoryFilter from "./log-category-filter";
|
||||
|
||||
@@ -18,6 +19,7 @@ export default function LogsToolbar({ init }: Props) {
|
||||
selectCategory,
|
||||
selected_ids,
|
||||
afterDelete,
|
||||
afterClear,
|
||||
} = init;
|
||||
|
||||
return (
|
||||
@@ -33,20 +35,27 @@ export default function LogsToolbar({ init }: Props) {
|
||||
? `${count} log${count == 1 ? "" : "s"}`
|
||||
: "--"}
|
||||
</Span>
|
||||
{selected_ids[0] ? (
|
||||
<DeleteLogsButton
|
||||
ids={selected_ids}
|
||||
afterDelete={afterDelete}
|
||||
title="Delete Selected Logs"
|
||||
variant="outlined"
|
||||
size="small"
|
||||
>
|
||||
<Row className="gap-2! flex-nowrap">
|
||||
<Trash2 size={15} />
|
||||
<Span>Delete selected ({selected_ids.length})</Span>
|
||||
</Row>
|
||||
</DeleteLogsButton>
|
||||
) : null}
|
||||
<Row className="gap-2! flex-nowrap">
|
||||
{selected_ids[0] ? (
|
||||
<DeleteLogsButton
|
||||
ids={selected_ids}
|
||||
afterDelete={afterDelete}
|
||||
title="Delete Selected Logs"
|
||||
variant="outlined"
|
||||
size="small"
|
||||
>
|
||||
<Row className="gap-2! flex-nowrap">
|
||||
<Trash2 size={15} />
|
||||
<Span>Delete selected ({selected_ids.length})</Span>
|
||||
</Row>
|
||||
</DeleteLogsButton>
|
||||
) : null}
|
||||
<ClearLogsButton
|
||||
category={category}
|
||||
count={count}
|
||||
afterClear={afterClear}
|
||||
/>
|
||||
</Row>
|
||||
</Row>
|
||||
</Stack>
|
||||
</Paper>
|
||||
|
||||
@@ -114,7 +114,7 @@ export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
|
||||
|
||||
return {
|
||||
success: false,
|
||||
msg: safeErrorMessage({ error }),
|
||||
msg: safeErrorMessage({ error, log: false }),
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
@@ -89,7 +89,7 @@ export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
|
||||
|
||||
return {
|
||||
success: false,
|
||||
msg: safeErrorMessage({ error }),
|
||||
msg: safeErrorMessage({ error, log: false }),
|
||||
};
|
||||
}
|
||||
};
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
import { z } from "zod";
|
||||
import { LogCategories } from "@/src/dict/log-categories";
|
||||
|
||||
const LogCategoryValues = LogCategories.map((lc) => lc.value) as [
|
||||
(typeof LogCategories)[number]["value"],
|
||||
...(typeof LogCategories)[number]["value"][],
|
||||
];
|
||||
|
||||
/**
|
||||
* `POST api/admin/logs/clear-logs`: deletes every log in one category, or
|
||||
* every log when it is left out (the "All" tab).
|
||||
*/
|
||||
export const ClearLogsRequestSchema = z.object({
|
||||
category: z.enum(LogCategoryValues).optional(),
|
||||
});
|
||||
|
||||
export type ClearLogsRequest = z.infer<typeof ClearLogsRequestSchema>;
|
||||
@@ -1,23 +1,33 @@
|
||||
import AppError from "./app-error";
|
||||
import logUnexpectedError from "@/src/functions/backend/log-unexpected-error";
|
||||
|
||||
type Params = {
|
||||
error: unknown;
|
||||
fallback?: string;
|
||||
/**
|
||||
* Write an unexpected error to the logs. Off where the caller already
|
||||
* logs it.
|
||||
*/
|
||||
log?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* # Safe Error Message
|
||||
* Returns the message of an `AppError` (deliberate, user-facing) and a
|
||||
* generic fallback for everything else, so internal details (SQL errors,
|
||||
* file paths, stack messages) never reach the client.
|
||||
* file paths, stack messages) never reach the client. The unexpected ones
|
||||
* are logged (`logUnexpectedError`), since the client never sees why.
|
||||
*/
|
||||
export default function safeErrorMessage({
|
||||
error,
|
||||
fallback = "Something went wrong. Please try again.",
|
||||
log = true,
|
||||
}: Params): string {
|
||||
if (error instanceof AppError) {
|
||||
return error.message;
|
||||
}
|
||||
|
||||
if (log) logUnexpectedError({ error });
|
||||
|
||||
return fallback;
|
||||
}
|
||||
|
||||
@@ -9,7 +9,6 @@ import { getAllClients, getAllRooms } from "./(data)/rooms";
|
||||
import type { WebSocketDataType } from "@/src/types";
|
||||
import { wsSendData } from "./(functions)/ws-send-data";
|
||||
import checkSessionAccess from "./(functions)/check-session-access";
|
||||
import logEvent from "@/src/functions/backend/logger";
|
||||
import { computerProxySocket } from "@/src/functions/server/computers/computer-proxy-socket";
|
||||
import touchUserPresence from "@/src/functions/backend/db/push/touch-user-presence";
|
||||
|
||||
@@ -46,16 +45,6 @@ export const websocket = {
|
||||
message: "WebSocket connected",
|
||||
},
|
||||
});
|
||||
|
||||
logEvent({
|
||||
title: "WebSocket connected",
|
||||
category: "websocket",
|
||||
user_id: user.id,
|
||||
plain_text: `User #${user.id} [${user.first_name} ${user.last_name}] connected via WebSocket.`,
|
||||
metadata: {
|
||||
user_id: user.id,
|
||||
},
|
||||
});
|
||||
},
|
||||
|
||||
async message(
|
||||
|
||||
Reference in New Issue
Block a user