Moduletrace/cobble
2
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 13 Wiki Activity

Run whole-data archives and restores in the background

Browse Source 
A data archive or restore ran inside its request, which sent nothing
until tar finished: Bun closes a silent socket after 10s (its default
idleTimeout in production) and a proxy cuts it at about 100s, so the
page showed a network error while the archive kept going, and a retry
said another backup was already running.

The routes now start the run (startDataBackupRun, refused while the
backup lock is held) and answer at once. The page follows it by polling
api/admin/backups/data-status: an archive reloads once written, a
restore once the status comes from a new process after its restart,
and a failure shows its error. A run in progress survives a reload.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
tben
2026-10-25 08:34:29 +01:00
co-authored by Claude Opus 5.5
parent 86becd50bc
commit 4161863587
22 changed files with 630 additions and 179 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/pages/admin/settings/backups/(data)/data-backup-run-context.ts
+26
View File
@@ -0,0 +1,26 @@
import React from "react";
import type { DataBackupRun } from "@/src/types";
export type DataBackupRunState = {
/**
* The archive or restore running now, while it runs.
*/
run?: DataBackupRun;
/**
* Why the last run failed.
*/
error?: string;
/**
* Follows a run a request just started until it ends.
*/
track: (run: DataBackupRun) => void;
};
/**
* The whole-data run the backups page is waiting on, shared by the list's
* actions, every archive row and the upload modal, so none of them starts
* another while one is going.
*/
export const DataBackupRunContext = React.createContext<DataBackupRunState>({
track: () => {},
});
src/pages/admin/settings/backups/(functions)/create-data-backup.ts
+6
View File
@@ -1,5 +1,10 @@
import adminFetchHandler from "@/src/functions/frontend/admin-fetch-handler";
import type { DataBackupRun } from "@/src/types";
/**
* Starts the archive, which runs in the background on the server, and
* hands back its run for the page to follow.
*/
export default async function createDataBackup() {
if (
!window.confirm(
@@ -17,5 +22,6 @@ export default async function createDataBackup() {
return {
success: Boolean(res.success),
msg: res.success ? undefined : res.msg || `Couldn't create archive`,
run: res.success ? (res.singleRes as DataBackupRun | undefined) : undefined,
};
}
src/pages/admin/settings/backups/(functions)/grab-backups-server-props.ts
+4 -1
View File
@@ -1,6 +1,7 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import grabDbBackups from "@/src/functions/backend/backups/grab-db-backups";
import grabDataBackups from "@/src/functions/backend/backups/grab-data-backups";
import { grabDataBackupRun } from "@/src/functions/backend/backups/data-backup-run";
import { SettingsDefaultPath } from "@/src/dict/settings-pages-dict";
import type { PagePropsType } from "@/src/types";
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
@@ -10,7 +11,8 @@ type Params = Parameters<BunextPageServerFn<PagePropsType>>[0];
/**
* Shared server props for the `/admin/settings/backups` sub-pages: the
* saved database snapshots and the whole-data archives. Super admins
* saved database snapshots, the whole-data archives and the archive or
* restore running now, so a reload mid-run picks it back up. Super admins
* only; everyone else lands on the general settings.
*/
export default async function grabBackupsServerProps(params: Params) {
@@ -32,6 +34,7 @@ export default async function grabBackupsServerProps(params: Params) {
...settings.props,
backups: grabDbBackups(),
data_backups: grabDataBackups(),
data_backup_run: grabDataBackupRun(),
},
};
}
src/pages/admin/settings/backups/(functions)/restore-data-backup.ts
+5 -4
View File
@@ -1,5 +1,5 @@
import adminFetchHandler from "@/src/functions/frontend/admin-fetch-handler";
import type { DataBackup } from "@/src/types";
import type { DataBackup, DataBackupRun } from "@/src/types";
import dayjs from "dayjs";
type Params = {
@@ -7,9 +7,9 @@ type Params = {
};
/**
* Replaces the whole data volume with the archive. The server takes a
* safety archive first and restarts afterwards, so this only sends the
* request and leaves the reload to the button.
* Starts replacing the whole data volume with the archive. The server
* takes a safety archive first and restarts afterwards, all in the
* background, so this hands back the run for the page to follow.
*/
export default async function restoreDataBackup({ backup }: Params) {
const confirm_msg = `Restore the data volume from the archive of ${dayjs(backup.created_at).format("MMM D, YYYY HH:mm:ss")}?\n\nEvery bot, workspace, computer home, file, media item and database snapshot is replaced by the archive's, and the app restarts. A safety archive of the current data is made first.`;
@@ -29,5 +29,6 @@ export default async function restoreDataBackup({ backup }: Params) {
return {
success: Boolean(res.success),
msg: res.success ? undefined : res.msg || `Couldn't restore archive`,
run: res.success ? (res.singleRes as DataBackupRun | undefined) : undefined,
};
}
src/pages/admin/settings/backups/(functions)/upload-restore-data-backup.ts
+6 -2
View File
@@ -1,11 +1,14 @@
import type { DataBackupRun } from "@/src/types";
type Params = {
file: File;
};
/**
* Sends the file as multipart form data (not base64 JSON) so large archives
* aren't inflated. The app replaces the data volume and restarts, so the
* caller reloads the page once it is back.
* aren't inflated. Once the file is saved the server restores it in the
* background and restarts, so this hands back the run for the page to
* follow.
*/
export default async function uploadRestoreDataBackup({ file }: Params) {
const confirm_msg = `Restore the data volume from "${file.name}"?\n\nEvery bot, workspace, computer home, file, media item and database snapshot is replaced by the archive's, and the app restarts. A safety archive of the current data is made first.`;
@@ -28,6 +31,7 @@ export default async function uploadRestoreDataBackup({ file }: Params) {
return {
success: Boolean(json.success),
msg: json.success ? undefined : json.msg || `Couldn't restore archive`,
run: json.success ? (json.singleRes as DataBackupRun | undefined) : undefined,
};
} catch (error: any) {
return {
src/pages/admin/settings/backups/(hooks)/use-data-backup-run.ts
+98
View File
@@ -0,0 +1,98 @@
import { useContext, useEffect, useState } from "react";
import { AppContext } from "@/src/pages/__root";
import { AppData } from "@/src/data/app-data";
import adminFetchHandler from "@/src/functions/frontend/admin-fetch-handler";
import type { DataBackupRun, DataBackupStatus } from "@/src/types";
import type { DataBackupRunState } from "../(data)/data-backup-run-context";
/**
* Follows a whole-data archive or restore, which runs in the background
* on the server, by polling `data-status` every `AppData.DataBackupPollMs`.
* An archive reloads the page once it is written. A restore restarts the
* app, so failed polls are expected and the page reloads once the status
* comes from a new boot (or after `AppData.DataRestoreWaitMs` anyway).
* Picks a run already going back up from the page props.
*/
export default function useDataBackupRun(): DataBackupRunState {
const { pageProps } = useContext(AppContext);
const initial = pageProps?.data_backup_run;
const [run, setRun] = useState<DataBackupRun | undefined>(
initial?.status == "running" ? initial : undefined,
);
const [error, setError] = useState<string>();
useEffect(() => {
if (run?.status != "running") return;
const active = run;
const deadline = Date.now() + AppData.DataRestoreWaitMs;
let timeout: ReturnType<typeof setTimeout> | undefined;
let stopped = false;
const fail = (msg: string) => {
setError(msg);
setRun(undefined);
};
const poll = async () => {
const res = await adminFetchHandler({
url: "/api/admin/backups/data-status",
});
if (stopped) return;
const current = res.success
? (res.singleRes as DataBackupStatus | undefined)?.run
: undefined;
const restarted = Boolean(
current && current.boot_id != active.boot_id,
);
if (active.kind == "restore") {
if (restarted || Date.now() > deadline) {
window.location.reload();
return;
}
if (current?.status == "failed") {
fail(current.error || `Couldn't restore archive`);
return;
}
} else if (current) {
if (current.status == "done") {
window.location.reload();
return;
}
if (current.status == "failed") {
fail(current.error || `Couldn't create archive`);
return;
}
if (restarted) {
fail(`The app restarted before the archive was finished`);
return;
}
}
timeout = setTimeout(poll, AppData.DataBackupPollMs);
};
timeout = setTimeout(poll, AppData.DataBackupPollMs);
return () => {
stopped = true;
if (timeout) clearTimeout(timeout);
};
}, [run]);
return {
run,
error,
track: (next) => {
setError(undefined);
setRun(next);
},
};
}
src/pages/admin/settings/backups/(partials)/data-backup-list-actions.tsx
+21 -7
View File
@@ -5,15 +5,20 @@ import Row from "@moduletrace/twui/layout/Row";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import createDataBackup from "../(functions)/create-data-backup";
import UploadRestoreModal from "../(partials)/upload-restore-modal";
import { useState } from "react";
import { DataBackupRunContext } from "../(data)/data-backup-run-context";
import { useContext, useState } from "react";
/**
* The data archive list's two ways in: make a fresh archive, or restore
* one from a file.
* one from a file. Also says how the archive or restore running now is
* going, and why the last one failed.
*/
export default function DataBackupListActions() {
const { loading, setLoading, status, setStatus } = useStatus();
const { run, error, track } = useContext(DataBackupRunContext);
const [uploading, setUploading] = useState(false);
const archiving = run?.kind == "archive";
const restoring = run?.kind == "restore";
const create_backup = async () => {
setStatus(undefined);
@@ -21,13 +26,13 @@ export default function DataBackupListActions() {
const res = await createDataBackup();
if (res.success) {
window.location.reload();
setLoading(false);
if (res.run) {
track(res.run);
return;
}
setLoading(false);
if (res.msg) {
setStatus({ error: true, msg: res.msg });
}
@@ -36,11 +41,19 @@ export default function DataBackupListActions() {
return (
<Row className="gap-2! flex-nowrap">
{status?.error && <Tag color="error">{status.msg}</Tag>}
{!status?.error && error && <Tag color="error">{error}</Tag>}
{archiving && (
<Tag color="warning">Archiving, this can take a while...</Tag>
)}
{restoring && (
<Tag color="warning">Restoring, the app restarts soon...</Tag>
)}
<Button
title="Restore from an archive file"
variant="outlined"
color="gray"
beforeIcon={<Upload />}
disabled={Boolean(run)}
onClick={() => {
setUploading(true);
}}
@@ -50,7 +63,8 @@ export default function DataBackupListActions() {
<Button
title="Create Data Archive"
beforeIcon={<Archive />}
loading={loading}
loading={loading || archiving}
disabled={Boolean(run)}
onClick={() => {
create_backup();
}}
src/pages/admin/settings/backups/(partials)/data-backup-row-actions.tsx
+13 -29
View File
@@ -6,13 +6,8 @@ import restoreDataBackup from "../(functions)/restore-data-backup";
import deleteDataBackup from "../(functions)/delete-data-backup";
import RowIconButton from "@/src/components/general/row-icon-button";
import { ArchiveRestore, Download, Trash2 } from "lucide-react";
import { useEffect, useRef, useState } from "react";
/**
* How long the app is given to come back after a restore restarted it
* before the page is reloaded anyway.
*/
const RELOAD_DELAY_MS = 8000;
import { useContext } from "react";
import { DataBackupRunContext } from "../(data)/data-backup-run-context";
type Props = {
backup: DataBackup;
@@ -21,18 +16,13 @@ type Props = {
/**
* One data archive's icon actions: restore (replaces the whole data
* volume, then the app restarts), download or delete, with their own
* one-line status.
* one-line status. A restore runs in the background and the section
* follows it, so every row waits while one is going.
*/
export default function DataBackupRowActions({ backup }: Props) {
const { loading, setLoading, status, setStatus } = useStatus();
const [restarting, setRestarting] = useState(false);
const timeout = useRef<ReturnType<typeof setTimeout>>(undefined);
useEffect(() => {
return () => {
if (timeout.current) clearTimeout(timeout.current);
};
}, []);
const { run, track } = useContext(DataBackupRunContext);
const busy = Boolean(run);
const remove = async () => {
setStatus(undefined);
@@ -58,16 +48,13 @@ export default function DataBackupRowActions({ backup }: Props) {
const res = await restoreDataBackup({ backup });
if (res.success) {
setRestarting(true);
timeout.current = setTimeout(() => {
window.location.reload();
}, RELOAD_DELAY_MS);
setLoading(false);
if (res.run) {
track(res.run);
return;
}
setLoading(false);
if (res.msg) {
setStatus({ error: true, msg: res.msg });
}
@@ -76,14 +63,11 @@ export default function DataBackupRowActions({ backup }: Props) {
return (
<Row className="gap-1! flex-nowrap justify-end">
{status?.error && <Tag color="error">{status.msg}</Tag>}
{restarting && (
<Tag color="warning">Restoring, the app restarts soon...</Tag>
)}
<RowIconButton
icon={ArchiveRestore}
title="Restore this archive"
loading={loading && !restarting}
disabled={restarting}
loading={loading}
disabled={busy}
onClick={() => {
restore();
}}
@@ -104,7 +88,7 @@ export default function DataBackupRowActions({ backup }: Props) {
<RowIconButton
icon={Trash2}
title="Delete this archive"
disabled={loading || restarting}
disabled={loading || busy}
onClick={() => {
remove();
}}
src/pages/admin/settings/backups/(partials)/upload-restore-fields.tsx
+15 -5
View File
@@ -6,6 +6,16 @@ import Stack from "@moduletrace/twui/layout/Stack";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import { useState } from "react";
import { Upload } from "lucide-react";
import type { DataBackupRun } from "@/src/types";
type UploadResult = {
success: boolean;
msg?: string;
/**
* The background restore a data archive upload started.
*/
run?: DataBackupRun;
};
type Props = {
/**
@@ -23,12 +33,12 @@ type Props = {
/**
* Sends the file and reports the outcome.
*/
upload: (params: { file: File }) => Promise<{ success: boolean; msg?: string }>;
upload: (params: { file: File }) => Promise<UploadResult>;
/**
* Reload the page once it worked (the app restarts after an archive
* restore, so the caller waits for it first).
* Called once it worked: a database restore reloads the page, a data
* archive restore hands its background run to the page to follow.
*/
onDone: () => void;
onDone: (res: UploadResult) => void;
};
/**
@@ -54,7 +64,7 @@ export default function UploadRestoreFields({
const res = await upload({ file });
if (res.success) {
onDone();
onDone(res);
return;
}
src/pages/admin/settings/backups/(partials)/upload-restore-modal.tsx
+12 -13
View File
@@ -2,12 +2,9 @@ import AddModalShell from "@/src/layouts/bots/(partials)/add-modal-shell";
import UploadRestoreFields from "./upload-restore-fields";
import uploadRestoreBackup from "../(functions)/upload-restore-backup";
import uploadRestoreDataBackup from "../(functions)/upload-restore-data-backup";
/**
* How long the app is given to come back after an archive restore
* restarted it before the page is reloaded anyway.
*/
const RELOAD_DELAY_MS = 8000;
import { DataBackupRunContext } from "../(data)/data-backup-run-context";
import type { DataBackupRun } from "@/src/types";
import { useContext } from "react";
type Props = {
open: boolean;
@@ -21,18 +18,20 @@ type Props = {
/**
* Uploading a file and restoring it, in a modal over the list it came
* from. A database file is restored in place; a data archive replaces the
* whole data volume, so the page waits for the app to restart.
* whole data volume in the background, so the modal closes and the
* section follows the run until the app is back from its restart.
*/
export default function UploadRestoreModal({ open, onClose, is_data }: Props) {
const reload = () => {
if (!is_data) {
const { track } = useContext(DataBackupRunContext);
const done = ({ run }: { run?: DataBackupRun }) => {
if (!is_data || !run) {
window.location.reload();
return;
}
setTimeout(() => {
window.location.reload();
}, RELOAD_DELAY_MS);
track(run);
onClose();
};
return (
@@ -64,7 +63,7 @@ export default function UploadRestoreModal({ open, onClose, is_data }: Props) {
? uploadRestoreDataBackup
: uploadRestoreBackup
}
onDone={reload}
onDone={done}
/>
) : null}
</AddModalShell>
src/pages/admin/settings/backups/(sections)/data-backups-section.tsx
+26 -19
View File
@@ -5,35 +5,42 @@ import SettingsGroup from "../../(partials)/settings-group";
import { AppContext } from "@/src/pages/__root";
import DataBackupListActions from "../(partials)/data-backup-list-actions";
import DataBackupsTable from "../(partials)/data-backups-table";
import { DataBackupRunContext } from "../(data)/data-backup-run-context";
import useDataBackupRun from "../(hooks)/use-data-backup-run";
/**
* The tar.gz archives of the whole data volume, newest first. Each one
* restores (replacing every folder and the database snapshots with the
* archive's own, then the app restarts), downloads or deletes; the newest
* `AppData.DataBackupsKept` are kept and older ones pruned.
* `AppData.DataBackupsKept` are kept and older ones pruned. Archives and
* restores run in the background on the server; the section follows the
* one running (`useDataBackupRun`) for every control in it.
*/
export default function DataBackupsSection() {
const { pageProps } = useContext(AppContext);
const backups = pageProps?.data_backups || [];
const data_backup_run = useDataBackupRun();
return (
<SettingsGroup
title="Data archives"
description="One tar.gz of the whole data volume, under a single folder named after the archive: the database, media, computer homes, every bot and workspace file, and the database snapshots. They live outside the data volume, and the newest few are kept."
>
<Row className="w-full items-center justify-between gap-2! flex-wrap">
<Span size="small" variant="faded">
{`${backups.length} archive${backups.length == 1 ? "" : "s"}`}
</Span>
<DataBackupListActions />
</Row>
{backups[0] ? (
<DataBackupsTable backups={backups} />
) : (
<Span size="small" variant="faded">
No archives yet.
</Span>
)}
</SettingsGroup>
<DataBackupRunContext.Provider value={data_backup_run}>
<SettingsGroup
title="Data archives"
description="One tar.gz of the whole data volume, under a single folder named after the archive: the database, media, computer homes, every bot and workspace file, and the database snapshots. They live outside the data volume, and the newest few are kept."
>
<Row className="w-full items-center justify-between gap-2! flex-wrap">
<Span size="small" variant="faded">
{`${backups.length} archive${backups.length == 1 ? "" : "s"}`}
</Span>
<DataBackupListActions />
</Row>
{backups[0] ? (
<DataBackupsTable backups={backups} />
) : (
<Span size="small" variant="faded">
No archives yet.
</Span>
)}
</SettingsGroup>
</DataBackupRunContext.Provider>
);
}