Moduletrace/cobble
2
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 9 Wiki Activity

Archive the whole data volume in Backups, not just the database

Browse Source 
Settings > Backups gains a whole-data archive beside the database
snapshots, and both lists now fully work from the page.

- Data archives: `write-data-backup-archive.ts` tars all of DATA_DIR into
  BACKUP_ARCHIVE_DIR as cobble-data-<epoch-ms>.tar.gz, every entry under one
  root folder named after the archive, writing a .part name and renaming on
  success so a cut-off run is never listed. The newest
  AppData.DataBackupsKept are kept; db/.backups is left as it is.
- Restore: `restore-data-backup.ts` extracts to a temp dir, requires one
  root folder holding a db folder, takes a safety archive first, then
  replaces DATA_DIR (the database snapshots included) and `restart-app.ts`
  restarts the process through the docker socket, since the open SQLite
  handle points at the replaced file.
- Upload and restore: a modal over each list (database file or data
  archive), replacing the old /admin/settings/backups/restore page.
- Delete for both kinds, icon-only row actions shared through
  RowIconButton, and restore on every row.
- One backup or restore of either kind at a time (with-backup-run-lock).
- BACKUP_ARCHIVE_DIR is a start-up path (ProcessOnlyEnvKeys); the installer
  mounts $COBBLE_HOME/backups-archives at /backups-archives and recreates a
  container that lacks it. The image installs tar and gzip.
This commit is contained in:
tben
2026-10-22 11:00:37 +01:00
parent fd75111bb1
commit a88898d169
56 changed files with 1686 additions and 261 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+1 -1
View File
@@ -209,7 +209,7 @@ Live routes under `/admin` (see `AdminSections`): `/admin/bots`, `/admin/workspa
- `/admin/logs` (live) - Activity, the audit log: super admins only (others are redirected; the popover's "Activity" item shows to super admins only). Rendered in `SettingsShell` with `wide`
- `/admin/help` (live) - familiar problems and their fixes, one `SettingsGroup` per topic (`(data)/help-topics.ts`, markdown rendered with the chat's `MessageMarkdownComponents`, the topic id as the group's anchor). First topic: signing in to Google and other strict sites with Cookie-Editor over the Desktop view (`HelpSignInTopicId` in `src/dict/help-dict.ts`; `ask_user_to_take_over` points bots to it)
- Routines, Files, Activity and Help are the `RailPages` (`src/dict/rail-pages-dict.ts`), opened from the rail's user popover (which lists them, then Settings and Sign out). On them the rail shows its own aside (`(partials)/rail-pages-list.tsx`): a back link to the bots, then those pages, like settings
- `/admin/settings/backups` - super admin only (listed in `SettingsPages`): create/list/download database snapshots in `db/.backups`, plus `restore/` (restore a listed backup, or upload a SQLite database file which is saved to `.backups` then restored). Shells out to the `bun-sqlite` CLI (`backup`, and `import <file>` since `restore` is interactive) via `src/functions/backend/backups/`; API routes in `api/admin/backups/`
- `/admin/settings/backups` - super admin only (listed in `SettingsPages`), two groups. Database snapshots: create, list, restore, download and delete in `db/.backups` (`bun-sqlite backup`); "Restore from file" beside "Create backup" opens `upload-restore-modal.tsx` (upload a SQLite database file, which is saved to `.backups` then restored with `bun-sqlite import <file>`, since `restore` is interactive). Whole-data archives: `write-data-backup-archive.ts` tars all of `DATA_DIR` (database, media, computer homes, bot and workspace files, MCP installs) into `BACKUP_ARCHIVE_DIR` as `cobble-data-<epoch-ms>.tar.gz` (via `tar -czf` into a `.part` name, renamed on success so a cut-off run is never listed, with a `--transform` putting every member under one root folder named after the archive), keeping the newest `AppData.DataBackupsKept` and never touching `db/.backups`; its own "Restore from file" uploads a tar.gz (`upload-restore-data.ts`, gzip header checked first). `restore-data-backup.ts` extracts an archive to a temp dir, requires exactly one root folder holding a `db` folder (`findArchiveRoot`, since an upload is saved under a fresh timestamp and its inner name no longer matches the file name), takes a safety archive before touching anything (`write-data-backup-archive.ts`, not the locked `create-data-backup.ts`), empties `DATA_DIR` (the snapshots included) and copies the archive's in, then `restart-app.ts` restarts the process through the docker socket after the response lands (`process.exit`, the container's restart policy brings it back), because the open SQLite handle points at the replaced file. Row actions are icon-only (`RowIconButton`, `src/components/general/row-icon-button.tsx`): restore, download, delete. `BACKUP_ARCHIVE_DIR` is a `ProcessOnlyEnvKeys` start-up path (the installer mounts `$COBBLE_HOME/backups-archives` there; outside docker it falls back to `~/.cobble/backups`). Every backup and restore, of either kind, runs one at a time via `with-backup-run-lock.ts`. Helpers in `src/functions/backend/backups/`, API routes in `api/admin/backups/`
- `/admin/settings` (live) - redirects to `SettingsDefaultPath` (`/admin/settings/general`). Sub pages: `general/` (theme: System/Light/Dark per device via `useThemePreference`, stored in `localStorage[AppData.ThemeStorageKey]`, the same key `ThemeInitScript` and twui's `ColorSchemeSelector` read; applies instantly, no save bar; and, for `canManageCrew`, the bot file quota: `storage-section.tsx`, `storage-quota-form.tsx` saving `api/admin/settings/storage-quota` into `app_settings`), `profile/` (current user profile), `notifications/` (this device's Web Push switch, a test send, and every device the user subscribed), `change-password/` (set/change own password) `models/` (super admin only, `super_admin_only` in the dict hides it from the rail: model providers and which model each harness uses) and `environment/` (super admin only: app settings and the envs bots and computers use, per scope), each wrapped in `SettingsShell` (`(partials)/settings-shell.tsx`: 64px pane header like the conversation header, then one centered column). Content is built from `SettingsGroup` blocks (label + description left, controls right, hairline-separated, no cards) and ends with the sticky `SettingsSaveBar` (Save enabled only when dirty, Discard, one-line status). Saves don't reload the page. Keep save notices in component state: `useFormInit` clears `status` whenever the form changes. `bots/` (the crew: chat, edit with `BotForm`, models, delete a worker via `api/admin/bots/delete-bot`), `workspaces/` (open, create through the Add flow, delete), `computers/` (state and resources, live; super admins create, edit with `AddComputerForm`, retry, delete), `mcps/` and `tools/` (see MCPs and custom tools: super admins add, edit, refresh or try, delete; admins only set Access, `extension-access-modal.tsx`; add, edit, access and try open in modals (`AddModalShell`)), each row with a status line under it (`extension-status-line.tsx`: connected or not, last run worked or failed, when, and why), those five for `canManageCrew` only (`grab-crew-settings-server-props.ts`, `can_view` in the dict) and `users/` (for `canListUsers`: super admins and admins; the dict's `can_view` gates it in the rail; add with a password (`canCreateUser`), edit (`canUpdateUser`), both with the one `UserForm` (`admin/settings/users/(partials)/user-form/`, `useFormInit`, photo included, saved in one request), revoke (adds `revoked`, `canRevokeUser`), delete (`canRevokeUser`'s rules, never yourself), pick their user types from `UserTypes` (`user-types-select.tsx`: only the ones `filterAssignableUserTypes` allows, never `UneditableUserTypes`; at least one, since login needs a type; `saveUser` checks each grant and removal with `userTypeAssignment`; the super admin's types are never shown or changed there, and they can't be revoked; the password is entered twice, `SaveUserRequestSchema` checks they match), delete with everything theirs; `api/admin/users/save-user`, `delete-user`). The lists share `SettingsList`, `SettingsListRow`, `SettingsRowAction` and `use-settings-list`. Sub pages are declared once in `src/dict/settings-pages-dict.ts`; on every `/admin/settings/*` page the rail swaps the crew for a back link plus that list. Add a sub page by adding its folder and a dict entry
Every `/admin/*` page renders in the bots layout (`src/layouts/bots/`); there is no separate admin layout or aside. The rail header's Add icon (super admins and admins only, like the rail's "+" buttons) opens the Add flow (`(partials)/add-flow.tsx`, `AddFlowContext`): a bot, a workspace or (super admins) a computer, each form in a modal stacked over the choice, so closing it goes back there (the Bots and Workspaces labels have a "+" that opens the form directly; in a workspace's rail, "Workspace bots" has a "+" that opens `link-bots-modal.tsx` instead: the crew's workers not in it yet (`api/admin/workspaces/linkable-bots`), each added with `add-bot`, which pushes `server:bot-created`, and "New bot" stacking the bot form over it); on a workspace's pages new bots and computers join it. The rail shows the crew (`(partials)/rail-crew-list.tsx`, needs `bots` and `workspaces` in page props): the coordinator, the latest `AppData.RailRecentBots` workers (plus any open, unread or needing you, `pick-rail-bots`), "See all bots" with a few stones opening every bot in a modal, then Workspaces with a "+". On `/admin/workspaces/*` it shows the workspace's crew instead (`(partials)/rail-workspace-list.tsx`: back link, the workspace, then only the bots attached to it, "Workspace bots"; the coordinator stays in the main crew rail, and on a workspace bot's page the back link returns to the workspace page), except on `/admin/settings/*` where it shows "Back to bots" plus the settings sub pages (`(partials)/rail-settings-list.tsx`). Page heroes use the shared `AdminHero` component (`src/components/general/admin-hero.tsx`).
Dockerfile
+1 -1
View File
@@ -7,7 +7,7 @@ LABEL org.opencontainers.image.title="Cobble" \
# The app serves plain HTTP; HTTPS is the reverse proxy's job.
RUN apt-get update \
&& apt-get install -y --no-install-recommends curl bash nano wget zip unzip \
&& apt-get install -y --no-install-recommends curl bash nano wget zip unzip tar gzip \
&& rm -rf /var/lib/apt/lists/*
ENV NODE_ENV=production
README.md
+11
View File
@@ -195,6 +195,7 @@ Everything that must survive an update lives under the install directory:
.env # generated once, mode 600
install.conf # installer state
backups/ # pre-update database snapshots
backups-archives/ # whole-data archives, from Settings > Backups
data/
db/cobble
db/.backups/
@@ -213,6 +214,16 @@ Edit `.env` and run `docker restart cobble` to apply changes. The installer
only appends missing values and keeps what is already there, apart from the
port it manages.
Settings > Backups makes two kinds of copy. Database snapshots (bun-sqlite's
`backup` command) stay in `data/db/.backups` and can be restored from the
app. Whole-data archives are a `tar.gz` of the entire `data/` directory,
written to `backups-archives/` (mounted at `/backups-archives`, the
`BACKUP_ARCHIVE_DIR` setting); every entry sits under one folder named after
the archive, so extracting one by hand drops a single folder. Each group
can create a copy, restore from one (a listed row, or a file uploaded from
the button beside Create), download it and delete it; the newest five
archives are kept.
The app container is started with the Docker socket mounted and
`MAIN_SERVER=1`, so it can create computer containers and run cron, the
queue and Web Push.
RELEASING.md
+7 -3
View File
@@ -70,6 +70,10 @@ Before the first public release:
- Images are tagged with the release version, so an older tag can be
installed with `--version vX.Y.Z`.
- The installer keeps the last five database backups in
`$COBBLE_HOME/backups`.
- Restore a backup from **Settings > Backups** in the app, or restore the
files with the app stopped.
`$COBBLE_HOME/backups`. Whole-data archives from Settings > Backups live
in `$COBBLE_HOME/backups-archives`; they are kept, not pruned by the
installer.
- Restore a backup from **Settings > Backups** in the app: a database
snapshot, or a whole-data archive (which replaces every folder, the
snapshots included, takes a safety archive first and restarts the app).
A snapshot can also be restored with the app stopped.
install.sh
+13 -2
View File
@@ -295,6 +295,9 @@ ENV_FILE="$HOME_DIR/.env"
STATE_FILE="$HOME_DIR/install.conf"
DATA_DIR="${DATA_DIR%/}"
BACKUP_DIR="$HOME_DIR/backups"
# Whole-data archives made in the app (Settings > Backups). They live
# outside DATA_DIR, so an archive never contains another one.
BACKUP_ARCHIVE_DIR_HOST="$HOME_DIR/backups-archives"
# ---------------------------------------------------------------- platform
@@ -325,9 +328,11 @@ fi
ENV_MOUNT="$ENV_FILE:/app/.env:ro"
DATA_MOUNT="$DATA_DIR:/data"
BACKUP_ARCHIVE_MOUNT="$BACKUP_ARCHIVE_DIR_HOST:/backups-archives"
if [ -n "$SELINUX_SUFFIX" ]; then
ENV_MOUNT="$ENV_FILE:/app/.env:ro,z"
DATA_MOUNT="$DATA_DIR:/data,z"
BACKUP_ARCHIVE_MOUNT="$BACKUP_ARCHIVE_DIR_HOST:/backups-archives,z"
fi
# ---------------------------------------------------------------- helpers
@@ -418,12 +423,14 @@ container_matches() {
[ "$(container_image_id)" = "$TARGET_IMAGE_ID" ] || return 1
[ "$(docker inspect -f '{{range .Mounts}}{{if eq .Destination "/data"}}{{.Source}}{{end}}{{end}}' "$CONTAINER")" = "$DATA_DIR" ] || return 1
[ "$(docker inspect -f '{{range .Mounts}}{{if eq .Destination "/app/.env"}}{{.Source}}{{end}}{{end}}' "$CONTAINER")" = "$ENV_FILE" ] || return 1
[ "$(docker inspect -f '{{range .Mounts}}{{if eq .Destination "/backups-archives"}}{{.Source}}{{end}}{{end}}' "$CONTAINER")" = "$BACKUP_ARCHIVE_DIR_HOST" ] || return 1
[ "$(docker_socket_mounted)" = "$DOCKER_SOCKET" ] || return 1
[ "$(docker inspect -f '{{with index .HostConfig.PortBindings "'"$PORT"'/tcp"}}{{range .}}{{.HostIp}}:{{.HostPort}}{{end}}{{end}}' "$CONTAINER")" = "$BIND:$PORT" ] || return 1
[ "$(docker inspect -f '{{.HostConfig.RestartPolicy.Name}}' "$CONTAINER")" = "unless-stopped" ] || return 1
[ "$(docker inspect -f '{{.HostConfig.Init}}' "$CONTAINER")" = "true" ] || return 1
docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$CONTAINER" | grep -qx "MAIN_SERVER=1" || return 1
docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$CONTAINER" | grep -qx "PORT=$PORT" || return 1
docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$CONTAINER" | grep -qx "BACKUP_ARCHIVE_DIR=/backups-archives" || return 1
docker inspect -f '{{range $name, $net := .NetworkSettings.Networks}}{{println $name}}{{end}}' "$CONTAINER" | grep -qx "$NETWORK" || return 1
if [ -n "$EXTRA_NETWORK" ]; then
docker inspect -f '{{range $name, $net := .NetworkSettings.Networks}}{{println $name}}{{end}}' "$CONTAINER" | grep -qx "$EXTRA_NETWORK" || return 1
@@ -562,11 +569,13 @@ run_container() {
local mounts=(
-v "$ENV_MOUNT"
-v "$DATA_MOUNT"
-v "$BACKUP_ARCHIVE_MOUNT"
-v "$DOCKER_SOCKET:/var/run/docker.sock"
)
local envs=(
-e MAIN_SERVER=1
-e "PORT=$PORT"
-e BACKUP_ARCHIVE_DIR=/backups-archives
)
docker run -d \
@@ -667,8 +676,8 @@ if [ "$ACTION" = "uninstall" ]; then
uninstall
fi
mkdir -p "$HOME_DIR" "$DATA_DIR" "$BACKUP_DIR"
chmod 755 "$HOME_DIR" "$DATA_DIR" "$BACKUP_DIR"
mkdir -p "$HOME_DIR" "$DATA_DIR" "$BACKUP_DIR" "$BACKUP_ARCHIVE_DIR_HOST"
chmod 755 "$HOME_DIR" "$DATA_DIR" "$BACKUP_DIR" "$BACKUP_ARCHIVE_DIR_HOST"
if ! docker network inspect "$NETWORK" >/dev/null 2>&1; then
say "Creating the $NETWORK network"
@@ -707,6 +716,7 @@ unset_env HTTP_REDIRECT_PORT
{
printf 'COBBLE_DATA_DIR=%q\n' "$DATA_DIR"
printf 'COBBLE_BACKUP_ARCHIVE_DIR=%q\n' "$BACKUP_ARCHIVE_DIR_HOST"
printf 'COBBLE_PORT=%q\n' "$PORT"
printf 'COBBLE_BIND=%q\n' "$BIND"
printf 'COBBLE_IMAGE=%q\n' "$IMAGE"
@@ -774,6 +784,7 @@ printf ' First visit: create the super admin account.\n'
printf ' Data: %s\n' "$DATA_DIR"
printf ' Settings: %s\n' "$ENV_FILE"
printf ' Backups: %s\n' "$BACKUP_DIR"
printf ' Archives: %s\n' "$BACKUP_ARCHIVE_DIR_HOST"
printf ' Logs: docker logs -f %s\n' "$CONTAINER"
printf ' Update: curl -fsSL %s | sudo bash\n' "$INSTALL_URL"
printf ' Uninstall: curl -fsSL %s | sudo bash -s -- --uninstall\n' "$INSTALL_URL"
src/components/general/row-icon-button.tsx
+54
View File
@@ -0,0 +1,54 @@
import type { AnchorHTMLAttributes, DetailedHTMLProps } from "react";
import Button from "@moduletrace/twui/layout/Button";
import type { LucideIcon } from "lucide-react";
type Props = {
icon: LucideIcon;
/**
* Says what the button does; the button shows only the icon.
*/
title: string;
disabled?: boolean;
loading?: boolean;
/**
* For a link action (a download); leaves out `onClick`.
*/
href?: string;
linkProps?: DetailedHTMLProps<
AnchorHTMLAttributes<HTMLAnchorElement>,
HTMLAnchorElement
>;
onClick?: () => void;
};
/**
* One icon-only action on a list row (download, restore, delete), with its
* label kept for the tooltip and screen readers.
*/
export default function RowIconButton({
icon: Icon,
title,
disabled,
loading,
href,
linkProps,
onClick,
}: Props) {
return (
<Button
title={title}
aria-label={title}
type="button"
variant="ghost"
color="gray"
size="small"
href={href}
linkProps={linkProps}
disabled={disabled}
loading={loading}
onClick={onClick}
>
<Icon size={16} aria-hidden />
</Button>
);
}
src/data/app-data.ts
+8
View File
@@ -482,6 +482,14 @@ export const AppData = {
* is offline.
*/
PushTtlSeconds: 60 * 60,
/**
* Whole-data archives (Settings > Backups): the file name prefix and
* how many of them are kept, newest first. They live outside
* `DATA_DIR` (`BACKUP_ARCHIVE_DIR`), never in the database's own
* `db/.backups` directory, which keeps its bun-sqlite snapshots.
*/
DataBackupNamePrefix: "cobble-data-",
DataBackupsKept: 5,
/**
* Responses smaller than this go out uncompressed: the encoding
* costs more than it saves.
src/dict/env-dict.ts
+4 -3
View File
@@ -41,9 +41,9 @@ export type KnownEnv = {
* their own keys beside these for bots and computers to use.
*
* Not here, and never in the `envs` table: `ENCRYPTION_KEY` and
* `ENCRYPTION_SALT` (they encrypt the table), and `PORT`, `DATA_DIR` and
* `MAIN_SERVER` (the container's own start-up settings). Those stay in the
* process environment.
* `ENCRYPTION_SALT` (they encrypt the table), and `PORT`, `DATA_DIR`,
* `BACKUP_ARCHIVE_DIR` and `MAIN_SERVER` (the container's own start-up
* settings). Those stay in the process environment.
*/
export const EnvDict: readonly KnownEnv[] = [
{
@@ -95,6 +95,7 @@ export const ProcessOnlyEnvKeys = [
"PORT",
"DATA_DIR",
"DB_DIR",
"BACKUP_ARCHIVE_DIR",
"MAIN_SERVER",
] as const;
src/dict/settings-pages-dict.ts
-1
View File
@@ -34,7 +34,6 @@ export type SettingsPage = {
export const SettingsComputersPath = "/admin/settings/computers";
export const SettingsBackupsPath = "/admin/settings/backups";
export const SettingsBackupsRestorePath = `${SettingsBackupsPath}/restore`;
/**
* The `/admin/settings/*` sub pages, listed in the rail on every settings
src/functions/backend/backups/create-data-backup.ts
+21
View File
@@ -0,0 +1,21 @@
import withBackupRunLock from "./with-backup-run-lock";
import writeDataBackupArchive from "./write-data-backup-archive";
/**
* Archives the whole `DATA_DIR` (database, media, computer homes, bot and
* workspace files, MCP installs) into `BACKUP_ARCHIVE_DIR` as
* `cobble-data-{timestamp}.tar.gz`, then keeps only the newest
* `AppData.DataBackupsKept`. Every entry sits under one root folder named
* after the archive (`cobble-data-{timestamp}/`), so extracting it lands
* one folder, never a heap of files in the current directory. The
* database's own snapshots in `db/.backups` are inside `DATA_DIR`, so they
* ride along and are left alone on disk.
*
* The archive is a filesystem copy of a live volume, so the SQLite file
* inside it is only as consistent as the moment tar read it; a consistent
* database snapshot is what the `db/.backups` list is for. Restoring a
* whole-data archive is `restoreDataBackup` (`restore-data-backup.ts`).
*/
export default async function createDataBackup() {
return await withBackupRunLock({ task: writeDataBackupArchive });
}
src/functions/backend/backups/delete-data-backup.ts
+20
View File
@@ -0,0 +1,20 @@
import fs from "fs";
import path from "path";
import grabBackupArchiveDir from "@/src/utils/grab-backup-archive-dir";
import resolveDataBackupPath from "./resolve-data-backup-path";
type Params = {
backup_name?: string;
};
/**
* Deletes one whole-data archive. Only an existing archive's own file is
* removed, never a path from the request.
*/
export default function deleteDataBackup({ backup_name }: Params) {
const backup_path = resolveDataBackupPath({ backup_name });
fs.rmSync(path.join(grabBackupArchiveDir(), path.basename(backup_path)), {
force: true,
});
}
src/functions/backend/backups/delete-db-backup.ts
+14
View File
@@ -0,0 +1,14 @@
import fs from "fs";
import resolveDbBackupPath from "./resolve-db-backup-path";
type Params = {
backup_name?: string;
};
/**
* Deletes one database snapshot from `db/.backups`. Only an existing
* snapshot's own file is removed, never a path from the request.
*/
export default function deleteDbBackup({ backup_name }: Params) {
fs.rmSync(resolveDbBackupPath({ backup_name }), { force: true });
}
src/functions/backend/backups/grab-data-backups.ts
+39
View File
@@ -0,0 +1,39 @@
import fs from "fs";
import path from "path";
import { AppData } from "@/src/data/app-data";
import type { DataBackup } from "@/src/types";
import grabBackupArchiveDir from "@/src/utils/grab-backup-archive-dir";
const DATA_BACKUP_NAME_REGEX = new RegExp(
`^${AppData.DataBackupNamePrefix}(\\d+)\\.tar\\.gz$`,
);
/**
* Lists the `cobble-data-{timestamp}.tar.gz` archives of the whole
* `DATA_DIR`, newest first. Only finished files match: a run in progress
* writes a `.part` name, so a half-written archive never shows up. The
* database snapshots bun-sqlite writes into `db/.backups` never match
* either, so the two lists stay apart.
*/
export default function grabDataBackups(): DataBackup[] {
const archive_dir = grabBackupArchiveDir();
return fs
.readdirSync(archive_dir)
.filter((name) => DATA_BACKUP_NAME_REGEX.test(name))
.map((name) => {
const stat = fs.statSync(path.join(archive_dir, name));
const timestamp = Number(
DATA_BACKUP_NAME_REGEX.exec(name)?.[1],
);
return {
name,
created_at: Number.isFinite(timestamp)
? timestamp
: stat.mtimeMs,
size: stat.size,
};
})
.sort((a, b) => b.created_at - a.created_at);
}
src/functions/backend/backups/resolve-data-backup-path.ts
+22
View File
@@ -0,0 +1,22 @@
import path from "path";
import grabBackupArchiveDir from "@/src/utils/grab-backup-archive-dir";
import grabDataBackups from "./grab-data-backups";
type Params = {
backup_name?: string;
};
/**
* Resolves a data archive name from the request to its absolute path.
* Only names of existing archives are accepted, so arbitrary paths can't
* be read or written.
*/
export default function resolveDataBackupPath({ backup_name }: Params) {
const backup = grabDataBackups().find((b) => b.name === backup_name);
if (!backup) {
throw new Error(`Archive not found`);
}
return path.join(grabBackupArchiveDir(), backup.name);
}
src/functions/backend/backups/restart-app.ts
+36
View File
@@ -0,0 +1,36 @@
import dockerRequest from "@/src/functions/backend/docker/docker-request";
import grabAppContainer from "@/src/functions/backend/docker/grab-app-container";
/**
* How long the process waits before it restarts itself, so the response
* that "the restore worked" reaches the browser first.
*/
const RESTART_DELAY_MS = 1500;
/**
* Restarts the app after the data volume was replaced: the open SQLite
* handle still points at the old file, so the process must come back up
* on the restored one. In docker the container is restarted through the
* socket (its `unless-stopped` policy and the entrypoint do the rest);
* outside docker the process simply exits and is the dev server's to
* start again.
*/
export default function restartApp() {
setTimeout(async () => {
try {
const container = await grabAppContainer();
if (container) {
await dockerRequest({
method: "POST",
path: `/containers/${encodeURIComponent(container.id)}/restart`,
});
}
} catch {
// The exit below still brings the container back through its
// restart policy; a failed API call must not block it.
}
process.exit(0);
}, RESTART_DELAY_MS);
}
src/functions/backend/backups/restore-data-backup.ts
+149
View File
@@ -0,0 +1,149 @@
import fs from "fs";
import os from "os";
import path from "path";
import { AppData } from "@/src/data/app-data";
import grabDirNames from "@/src/utils/grab-dir-names";
import writeDataBackupArchive from "./write-data-backup-archive";
import resolveDataBackupPath from "./resolve-data-backup-path";
import withBackupRunLock from "./with-backup-run-lock";
export type DataBackupRestoreResult = {
/**
* The archive taken of the data as it was before the restore, so the
* restore itself can be undone.
*/
safety_name: string;
restored_name: string;
};
type Params = {
backup_name?: string;
};
type RootParams = {
/**
* Where the archive was extracted to.
*/
staging_dir: string;
};
/**
* The single folder an archive holds its data in (its root folder, named
* after the archive when it was made). An archive with no single root
* folder, or with loose files beside it, isn't one of ours.
*/
function findArchiveRoot({ staging_dir }: RootParams) {
const entries = fs.readdirSync(staging_dir);
const folders = entries.filter((entry) =>
fs.statSync(path.join(staging_dir, entry)).isDirectory(),
);
const [folder] = folders;
if (entries.length != 1 || !folder) {
throw new Error(
`The archive has no single root folder: it isn't a Cobble data archive`,
);
}
return path.join(staging_dir, folder);
}
/**
* Replaces the whole `DATA_DIR` with the contents of a data archive:
* the database, media, computer homes, bot and workspace files, and the
* database snapshots in `db/.backups` (which the archive's own overwrite,
* so the restore is one consistent set).
*
* The archive is read into a temporary directory on the same volume
* first, must hold one root folder with a database in it, and nothing on
* disk is touched before that passes. A fresh archive of the current data
* is taken as a safety copy, then the data directory is emptied (the
* database snapshots included) and filled from the archive.
*
* The caller restarts the process afterwards: the open SQLite handle still
* points at the replaced file, so nothing may keep reading it.
*/
export default async function restoreDataBackup({
backup_name,
}: Params): Promise<DataBackupRestoreResult> {
return await withBackupRunLock({
task: async () => {
const { DATA_DIR } = grabDirNames();
const archive_path = resolveDataBackupPath({ backup_name });
const archive_name = path.basename(archive_path);
const staging_dir = fs.mkdtempSync(
path.join(os.tmpdir(), "cobble-restore-"),
);
try {
const extract = Bun.spawn(
["tar", "-xzf", archive_path, "-C", staging_dir],
{
cwd: process.cwd(),
env: process.env,
stdin: "ignore",
stdout: "pipe",
stderr: "pipe",
},
);
const [stdout, stderr, exit_code] = await Promise.all([
new Response(extract.stdout).text(),
new Response(extract.stderr).text(),
extract.exited,
]);
if (exit_code !== 0) {
throw new Error(
stderr.trim() ||
stdout.trim() ||
`tar failed (exit ${exit_code})`,
);
}
// The archive's own name may differ from its file name: an
// upload is saved under a fresh timestamp. Its one root
// folder is what holds the data, so find it rather than
// assume it matches.
const source_dir = findArchiveRoot({ staging_dir });
if (!fs.existsSync(path.join(source_dir, "db"))) {
throw new Error(
`The archive has no database, so nothing was restored`,
);
}
// The archive's snapshots would otherwise vanish with the
// current ones, so the safety copy is taken before them.
// The run lock is already held here, hence the unlocked
// writer rather than `createDataBackup`.
const safety = await writeDataBackupArchive();
// Everything goes, the snapshots in `db/.backups` included:
// the archive's own set replaces them.
for (const entry of fs.readdirSync(DATA_DIR)) {
fs.rmSync(path.join(DATA_DIR, entry), {
recursive: true,
force: true,
});
}
for (const entry of fs.readdirSync(source_dir)) {
fs.cpSync(
path.join(source_dir, entry),
path.join(DATA_DIR, entry),
{ recursive: true, force: true },
);
}
return {
safety_name: safety.name,
restored_name: archive_name,
};
} finally {
fs.rmSync(staging_dir, { recursive: true, force: true });
}
},
});
}
src/functions/backend/backups/run-bun-sqlite-command.ts
+13 -15
View File
@@ -1,28 +1,27 @@
import withBackupRunLock from "./with-backup-run-lock";
type Params = {
args: string[];
};
let is_running = false;
/**
* Runs a `bun-sqlite` CLI command (same as `bunx bun-sqlite ...`). Only one
* command runs at a time so a backup and a restore never overlap.
* backup or restore runs at a time so a backup, a restore and a whole-data
* archive never overlap.
*/
export default async function runBunSQLiteCommand({ args }: Params) {
if (is_running) {
throw new Error(`Another backup or restore is already running`);
}
is_running = true;
try {
const proc = Bun.spawn([process.execPath, "x", "bun-sqlite", ...args], {
return await withBackupRunLock({
task: async () => {
const proc = Bun.spawn(
[process.execPath, "x", "bun-sqlite", ...args],
{
cwd: process.cwd(),
env: process.env,
stdin: "ignore",
stdout: "pipe",
stderr: "pipe",
});
},
);
const [stdout, stderr, exit_code] = await Promise.all([
new Response(proc.stdout).text(),
@@ -39,7 +38,6 @@ export default async function runBunSQLiteCommand({ args }: Params) {
}
return { stdout };
} finally {
is_running = false;
}
},
});
}
src/functions/backend/backups/save-uploaded-data-backup.ts
+36
View File
@@ -0,0 +1,36 @@
import fs from "fs";
import path from "path";
import { AppData } from "@/src/data/app-data";
import grabBackupArchiveDir from "@/src/utils/grab-backup-archive-dir";
type Params = {
file: File;
};
const GZIP_MAGIC = [0x1f, 0x8b];
/**
* Saves an uploaded data archive into the archive directory under the
* same `{prefix}{timestamp}.tar.gz` name an archive made here uses, so it
* is listed, kept and restored like any other. A gzip header is required,
* so an arbitrary upload is refused before it reaches the restore.
* Returns the saved file name.
*/
export default async function saveUploadedDataBackup({ file }: Params) {
if (!file.size) {
throw new Error(`The uploaded archive is empty`);
}
const head = new Uint8Array(await file.slice(0, 2).arrayBuffer());
if (head[0] != GZIP_MAGIC[0] || head[1] != GZIP_MAGIC[1]) {
throw new Error(`Only a gzipped tar archive can be restored`);
}
const archive_dir = grabBackupArchiveDir();
const archive_name = `${AppData.DataBackupNamePrefix}${Date.now()}.tar.gz`;
await Bun.write(path.join(archive_dir, archive_name), file);
return archive_name;
}
src/functions/backend/backups/with-backup-run-lock.ts
+25
View File
@@ -0,0 +1,25 @@
let is_running = false;
type Params<T> = {
task: () => Promise<T>;
};
/**
* Runs one backup or restore at a time, whichever kind it is, so a
* database snapshot, a database restore and a whole-data archive never
* overlap and never fight over the same files. Hands back what the task
* returned.
*/
export default async function withBackupRunLock<T>({ task }: Params<T>) {
if (is_running) {
throw new Error(`Another backup or restore is already running`);
}
is_running = true;
try {
return await task();
} finally {
is_running = false;
}
}
src/functions/backend/backups/write-data-backup-archive.ts
+86
View File
@@ -0,0 +1,86 @@
import fs from "fs";
import path from "path";
import { AppData } from "@/src/data/app-data";
import grabDirNames from "@/src/utils/grab-dir-names";
import grabBackupArchiveDir from "@/src/utils/grab-backup-archive-dir";
import grabDataBackups from "./grab-data-backups";
export type CreatedDataBackup = {
name: string;
path: string;
};
/**
* Writes one archive of the whole `DATA_DIR` and prunes the old ones.
* Callers already hold the backup run lock (`withBackupRunLock`), so a
* restore can take its safety copy without deadlocking on it.
*/
export default async function writeDataBackupArchive(): Promise<CreatedDataBackup> {
const { DATA_DIR } = grabDirNames();
const archive_dir = grabBackupArchiveDir();
const archive_name = `${AppData.DataBackupNamePrefix}${Date.now()}.tar.gz`;
const archive_root = archive_name.replace(/\.tar\.gz$/, "");
const archive_path = path.join(archive_dir, archive_name);
const part_path = `${archive_path}.part`;
try {
const proc = Bun.spawn(
[
"tar",
"-czf",
part_path,
// Every member (`.`, `./db`, `./media/x`, ...) is moved
// under one folder named after the archive, so an extract
// drops a single folder rather than a heap of files.
// Two expressions: the root `.` itself and every `./`
// member, so the whole archive sits under one folder named
// after it and an extract drops a single folder.
"--transform",
`s,^\\.$,${archive_root},;s,^\\./,${archive_root}/,`,
"-C",
DATA_DIR,
".",
],
{
cwd: process.cwd(),
env: process.env,
stdin: "ignore",
stdout: "pipe",
stderr: "pipe",
},
);
const [stdout, stderr, exit_code] = await Promise.all([
new Response(proc.stdout).text(),
new Response(proc.stderr).text(),
proc.exited,
]);
if (exit_code !== 0) {
throw new Error(
stderr.trim() ||
stdout.trim() ||
`tar failed (exit ${exit_code})`,
);
}
// Only a finished archive takes its real name, so a failed or
// cut-off run is never listed or downloaded.
fs.renameSync(part_path, archive_path);
} finally {
if (fs.existsSync(part_path)) {
fs.rmSync(part_path, { force: true });
}
}
const old = grabDataBackups().slice(AppData.DataBackupsKept);
for (const backup of old) {
fs.rmSync(path.join(archive_dir, backup.name), { force: true });
}
return {
name: archive_name,
path: archive_path,
};
}
src/pages/admin/settings/backups/(data)/data-backups-table-columns.ts
+12
View File
@@ -0,0 +1,12 @@
export type DataBackupsTableColumn = {
title: string;
className?: string;
hide_title?: boolean;
};
export const DataBackupsTableColumns: DataBackupsTableColumn[] = [
{ title: "Archive" },
{ title: "Created" },
{ title: "Size", className: "hidden md:table-cell" },
{ title: "Actions", className: "text-right", hide_title: true },
];
src/pages/admin/settings/backups/(functions)/create-data-backup.ts
+21
View File
@@ -0,0 +1,21 @@
import adminFetchHandler from "@/src/functions/frontend/admin-fetch-handler";
export default async function createDataBackup() {
if (
!window.confirm(
`Archive all of the data volume now? It can take a while on a busy install.`,
)
) {
return { success: false, cancelled: true };
}
const res = await adminFetchHandler({
url: "/api/admin/backups/create-data",
method: "POST",
});
return {
success: Boolean(res.success),
msg: res.success ? undefined : res.msg || `Couldn't create archive`,
};
}
src/pages/admin/settings/backups/(functions)/delete-data-backup.ts
+28
View File
@@ -0,0 +1,28 @@
import adminFetchHandler from "@/src/functions/frontend/admin-fetch-handler";
import type { DataBackup } from "@/src/types";
import dayjs from "dayjs";
type Params = {
backup: DataBackup;
};
export default async function deleteDataBackup({ backup }: Params) {
const confirm_msg = `Delete the data archive of ${dayjs(backup.created_at).format("MMM D, YYYY HH:mm:ss")}?\n\nThis removes the file for good.`;
if (!window.confirm(confirm_msg)) {
return { success: false, cancelled: true };
}
const res = await adminFetchHandler({
url: "/api/admin/backups/delete-data",
method: "POST",
body: {
backup_name: backup.name,
},
});
return {
success: Boolean(res.success),
msg: res.success ? undefined : res.msg || `Couldn't delete archive`,
};
}
src/pages/admin/settings/backups/(functions)/delete-db-backup.ts
+28
View File
@@ -0,0 +1,28 @@
import adminFetchHandler from "@/src/functions/frontend/admin-fetch-handler";
import type { DbBackup } from "@/src/types";
import dayjs from "dayjs";
type Params = {
backup: DbBackup;
};
export default async function deleteDbBackup({ backup }: Params) {
const confirm_msg = `Delete the database snapshot of ${dayjs(backup.created_at).format("MMM D, YYYY HH:mm:ss")}?\n\nThis removes the file for good.`;
if (!window.confirm(confirm_msg)) {
return { success: false, cancelled: true };
}
const res = await adminFetchHandler({
url: "/api/admin/backups/delete-db",
method: "POST",
body: {
backup_name: backup.name,
},
});
return {
success: Boolean(res.success),
msg: res.success ? undefined : res.msg || `Couldn't delete snapshot`,
};
}
src/pages/admin/settings/backups/(functions)/grab-backups-server-props.ts
+4 -2
View File
@@ -1,5 +1,6 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import grabDbBackups from "@/src/functions/backend/backups/grab-db-backups";
import grabDataBackups from "@/src/functions/backend/backups/grab-data-backups";
import { SettingsDefaultPath } from "@/src/dict/settings-pages-dict";
import type { PagePropsType } from "@/src/types";
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
@@ -9,8 +10,8 @@ type Params = Parameters<BunextPageServerFn<PagePropsType>>[0];
/**
* Shared server props for the `/admin/settings/backups` sub-pages: the
* saved snapshots. Super admins only; everyone else lands on the general
* settings.
* saved database snapshots and the whole-data archives. Super admins
* only; everyone else lands on the general settings.
*/
export default async function grabBackupsServerProps(params: Params) {
const settings = await grabSettingsServerProps(params);
@@ -30,6 +31,7 @@ export default async function grabBackupsServerProps(params: Params) {
props: {
...settings.props,
backups: grabDbBackups(),
data_backups: grabDataBackups(),
},
};
}
src/pages/admin/settings/backups/(functions)/restore-data-backup.ts
+33
View File
@@ -0,0 +1,33 @@
import adminFetchHandler from "@/src/functions/frontend/admin-fetch-handler";
import type { DataBackup } from "@/src/types";
import dayjs from "dayjs";
type Params = {
backup: DataBackup;
};
/**
* Replaces the whole data volume with the archive. The server takes a
* safety archive first and restarts afterwards, so this only sends the
* request and leaves the reload to the button.
*/
export default async function restoreDataBackup({ backup }: Params) {
const confirm_msg = `Restore the data volume from the archive of ${dayjs(backup.created_at).format("MMM D, YYYY HH:mm:ss")}?\n\nEvery bot, workspace, computer home, file, media item and database snapshot is replaced by the archive's, and the app restarts. A safety archive of the current data is made first.`;
if (!window.confirm(confirm_msg)) {
return { success: false, cancelled: true };
}
const res = await adminFetchHandler({
url: "/api/admin/backups/restore-data",
method: "POST",
body: {
backup_name: backup.name,
},
});
return {
success: Boolean(res.success),
msg: res.success ? undefined : res.msg || `Couldn't restore archive`,
};
}
src/pages/admin/settings/backups/restore/(functions)/upload-restore-backup.ts → src/pages/admin/settings/backups/(functions)/upload-restore-backup.ts
+2 -1
View File
@@ -4,7 +4,8 @@ type Params = {
/**
* Sends the file as multipart form data (not base64 JSON) so large dumps
* aren't inflated.
* aren't inflated. The server saves it into the snapshots list, then
* restores it.
*/
export default async function uploadRestoreBackup({ file }: Params) {
const confirm_msg = `Restore the database from "${file.name}"?\n\nAll current data will be replaced with the data in this file. This cannot be undone. Consider creating a fresh backup first.`;
src/pages/admin/settings/backups/(functions)/upload-restore-data-backup.ts
+38
View File
@@ -0,0 +1,38 @@
type Params = {
file: File;
};
/**
* Sends the file as multipart form data (not base64 JSON) so large archives
* aren't inflated. The app replaces the data volume and restarts, so the
* caller reloads the page once it is back.
*/
export default async function uploadRestoreDataBackup({ file }: Params) {
const confirm_msg = `Restore the data volume from "${file.name}"?\n\nEvery bot, workspace, computer home, file, media item and database snapshot is replaced by the archive's, and the app restarts. A safety archive of the current data is made first.`;
if (!window.confirm(confirm_msg)) {
return { success: false, cancelled: true };
}
try {
const form_data = new FormData();
form_data.append("file", file);
const res = await fetch("/api/admin/backups/upload-restore-data", {
method: "POST",
body: form_data,
});
const json = await res.json();
return {
success: Boolean(json.success),
msg: json.success ? undefined : json.msg || `Couldn't restore archive`,
};
} catch (error: any) {
return {
success: false,
msg: error.message || `Couldn't restore archive`,
};
}
}
src/pages/admin/settings/backups/(partials)/create-backup-button.tsx → src/pages/admin/settings/backups/(partials)/backup-list-actions.tsx
+24 -2
View File
@@ -1,12 +1,19 @@
import { DatabaseBackup } from "lucide-react";
import { DatabaseBackup, Upload } from "lucide-react";
import Button from "@moduletrace/twui/layout/Button";
import Tag from "@moduletrace/twui/elements/Tag";
import Row from "@moduletrace/twui/layout/Row";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import createBackup from "../(functions)/create-backup";
import UploadRestoreModal from "../(partials)/upload-restore-modal";
import { useState } from "react";
export default function CreateBackupButton() {
/**
* The database snapshot list's two ways in: make a fresh snapshot, or
* restore one from a file.
*/
export default function BackupListActions() {
const { loading, setLoading, status, setStatus } = useStatus();
const [uploading, setUploading] = useState(false);
const create_backup = async () => {
setStatus(undefined);
@@ -29,6 +36,17 @@ export default function CreateBackupButton() {
return (
<Row className="gap-2! flex-nowrap">
{status?.error && <Tag color="error">{status.msg}</Tag>}
<Button
title="Restore from a file"
variant="outlined"
color="gray"
beforeIcon={<Upload />}
onClick={() => {
setUploading(true);
}}
>
Restore from file
</Button>
<Button
title="Create Backup"
beforeIcon={<DatabaseBackup />}
@@ -39,6 +57,10 @@ export default function CreateBackupButton() {
>
Create backup
</Button>
<UploadRestoreModal
open={uploading}
onClose={() => setUploading(false)}
/>
</Row>
);
}
src/pages/admin/settings/backups/(partials)/backup-table-row.tsx
+3 -10
View File
@@ -2,16 +2,13 @@ import Span from "@moduletrace/twui/layout/Span";
import formatBytes from "@/src/utils/format-bytes";
import type { DbBackup } from "@/src/types";
import dayjs from "dayjs";
import type { BackupsTableMode } from "./backups-table";
import DownloadBackupButton from "./download-backup-button";
import RestoreBackupButton from "./restore-backup-button";
import DbBackupRowActions from "./db-backup-row-actions";
type Props = {
backup: DbBackup;
mode: BackupsTableMode;
};
export default function BackupTableRow({ backup, mode }: Props) {
export default function BackupTableRow({ backup }: Props) {
return (
<tr className="border-t border-slate-200 dark:border-white/10">
<td className="px-4 py-2.5 max-w-0 w-full">
@@ -34,11 +31,7 @@ export default function BackupTableRow({ backup, mode }: Props) {
</Span>
</td>
<td className="px-4 py-2 text-right whitespace-nowrap">
{mode == "restore" ? (
<RestoreBackupButton backup={backup} />
) : (
<DownloadBackupButton backup={backup} />
)}
<DbBackupRowActions backup={backup} />
</td>
</tr>
);
src/pages/admin/settings/backups/(partials)/backups-table.tsx
+2 -9
View File
@@ -4,14 +4,11 @@ import { BackupsTableColumns } from "../(data)/backups-table-columns";
import BackupTableRow from "./backup-table-row";
import BackupsTableHeadCell from "./backups-table-head-cell";
export type BackupsTableMode = "view" | "restore";
type Props = {
backups: DbBackup[];
mode: BackupsTableMode;
};
export default function BackupsTable({ backups, mode }: Props) {
export default function BackupsTable({ backups }: Props) {
return (
<Paper className="p-0 gap-0 overflow-x-auto">
<table className="w-full text-sm">
@@ -24,11 +21,7 @@ export default function BackupsTable({ backups, mode }: Props) {
</thead>
<tbody>
{backups.map((backup) => (
<BackupTableRow
backup={backup}
mode={mode}
key={backup.name}
/>
<BackupTableRow backup={backup} key={backup.name} />
))}
</tbody>
</table>
src/pages/admin/settings/backups/(partials)/data-backup-list-actions.tsx
+67
View File
@@ -0,0 +1,67 @@
import { Archive, Upload } from "lucide-react";
import Button from "@moduletrace/twui/layout/Button";
import Tag from "@moduletrace/twui/elements/Tag";
import Row from "@moduletrace/twui/layout/Row";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import createDataBackup from "../(functions)/create-data-backup";
import UploadRestoreModal from "../(partials)/upload-restore-modal";
import { useState } from "react";
/**
* The data archive list's two ways in: make a fresh archive, or restore
* one from a file.
*/
export default function DataBackupListActions() {
const { loading, setLoading, status, setStatus } = useStatus();
const [uploading, setUploading] = useState(false);
const create_backup = async () => {
setStatus(undefined);
setLoading(true);
const res = await createDataBackup();
if (res.success) {
window.location.reload();
return;
}
setLoading(false);
if (res.msg) {
setStatus({ error: true, msg: res.msg });
}
};
return (
<Row className="gap-2! flex-nowrap">
{status?.error && <Tag color="error">{status.msg}</Tag>}
<Button
title="Restore from an archive file"
variant="outlined"
color="gray"
beforeIcon={<Upload />}
onClick={() => {
setUploading(true);
}}
>
Restore from file
</Button>
<Button
title="Create Data Archive"
beforeIcon={<Archive />}
loading={loading}
onClick={() => {
create_backup();
}}
>
Create archive
</Button>
<UploadRestoreModal
open={uploading}
onClose={() => setUploading(false)}
is_data
/>
</Row>
);
}
src/pages/admin/settings/backups/(partials)/data-backup-row-actions.tsx
+114
View File
@@ -0,0 +1,114 @@
import Tag from "@moduletrace/twui/elements/Tag";
import Row from "@moduletrace/twui/layout/Row";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import type { DataBackup } from "@/src/types";
import restoreDataBackup from "../(functions)/restore-data-backup";
import deleteDataBackup from "../(functions)/delete-data-backup";
import RowIconButton from "@/src/components/general/row-icon-button";
import { ArchiveRestore, Download, Trash2 } from "lucide-react";
import { useEffect, useRef, useState } from "react";
/**
* How long the app is given to come back after a restore restarted it
* before the page is reloaded anyway.
*/
const RELOAD_DELAY_MS = 8000;
type Props = {
backup: DataBackup;
};
/**
* One data archive's icon actions: restore (replaces the whole data
* volume, then the app restarts), download or delete, with their own
* one-line status.
*/
export default function DataBackupRowActions({ backup }: Props) {
const { loading, setLoading, status, setStatus } = useStatus();
const [restarting, setRestarting] = useState(false);
const timeout = useRef<ReturnType<typeof setTimeout>>(undefined);
useEffect(() => {
return () => {
if (timeout.current) clearTimeout(timeout.current);
};
}, []);
const remove = async () => {
setStatus(undefined);
setLoading(true);
const res = await deleteDataBackup({ backup });
if (res.success) {
window.location.reload();
return;
}
setLoading(false);
if (res.msg) {
setStatus({ error: true, msg: res.msg });
}
};
const restore = async () => {
setStatus(undefined);
setLoading(true);
const res = await restoreDataBackup({ backup });
if (res.success) {
setRestarting(true);
timeout.current = setTimeout(() => {
window.location.reload();
}, RELOAD_DELAY_MS);
return;
}
setLoading(false);
if (res.msg) {
setStatus({ error: true, msg: res.msg });
}
};
return (
<Row className="gap-1! flex-nowrap justify-end">
{status?.error && <Tag color="error">{status.msg}</Tag>}
{restarting && (
<Tag color="warning">Restoring, the app restarts soon...</Tag>
)}
<RowIconButton
icon={ArchiveRestore}
title="Restore this archive"
loading={loading && !restarting}
disabled={restarting}
onClick={() => {
restore();
}}
/>
<RowIconButton
icon={Download}
title="Download this archive"
href={`/api/admin/backups/download-data?backup_name=${encodeURIComponent(backup.name)}`}
linkProps={{
download: backup.name,
onClick: (e) => {
if (!window.confirm(`Download archive ${backup.name}?`)) {
e.preventDefault();
}
},
}}
/>
<RowIconButton
icon={Trash2}
title="Delete this archive"
disabled={loading || restarting}
onClick={() => {
remove();
}}
/>
</Row>
);
}
src/pages/admin/settings/backups/(partials)/data-backup-table-row.tsx
+38
View File
@@ -0,0 +1,38 @@
import Span from "@moduletrace/twui/layout/Span";
import formatBytes from "@/src/utils/format-bytes";
import type { DataBackup } from "@/src/types";
import dayjs from "dayjs";
import DataBackupRowActions from "./data-backup-row-actions";
type Props = {
backup: DataBackup;
};
export default function DataBackupTableRow({ backup }: Props) {
return (
<tr className="border-t border-slate-200 dark:border-white/10">
<td className="px-4 py-2.5 max-w-0 w-full">
<Span
size="small"
className="block truncate font-mono"
title={backup.name}
>
{backup.name}
</Span>
</td>
<td className="px-4 py-2.5 whitespace-nowrap">
<Span size="small">
{dayjs(backup.created_at).format("MMM D, YYYY HH:mm:ss")}
</Span>
</td>
<td className="hidden md:table-cell px-4 py-2.5 whitespace-nowrap">
<Span size="small" variant="faded">
{formatBytes({ bytes: backup.size })}
</Span>
</td>
<td className="px-4 py-2 text-right whitespace-nowrap">
<DataBackupRowActions backup={backup} />
</td>
</tr>
);
}
src/pages/admin/settings/backups/(partials)/data-backups-table-head-cell.tsx
+21
View File
@@ -0,0 +1,21 @@
import { twMerge } from "tailwind-merge";
import Span from "@moduletrace/twui/layout/Span";
import type { DataBackupsTableColumn } from "../(data)/data-backups-table-columns";
export default function DataBackupsTableHeadCell({
title,
className,
hide_title,
}: DataBackupsTableColumn) {
return (
<th className={twMerge("px-4 py-2.5 text-left font-medium", className)}>
<Span
size="smaller"
variant="faded"
className={twMerge(hide_title && "sr-only")}
>
{title}
</Span>
</th>
);
}
src/pages/admin/settings/backups/(partials)/data-backups-table.tsx
+33
View File
@@ -0,0 +1,33 @@
import Paper from "@moduletrace/twui/elements/Paper";
import type { DataBackup } from "@/src/types";
import { DataBackupsTableColumns } from "../(data)/data-backups-table-columns";
import DataBackupTableRow from "./data-backup-table-row";
import DataBackupsTableHeadCell from "./data-backups-table-head-cell";
type Props = {
backups: DataBackup[];
};
export default function DataBackupsTable({ backups }: Props) {
return (
<Paper className="p-0 gap-0 overflow-x-auto">
<table className="w-full text-sm">
<thead className="bg-slate-50 dark:bg-white/5">
<tr>
{DataBackupsTableColumns.map((column) => (
<DataBackupsTableHeadCell
{...column}
key={column.title}
/>
))}
</tr>
</thead>
<tbody>
{backups.map((backup) => (
<DataBackupTableRow backup={backup} key={backup.name} />
))}
</tbody>
</table>
</Paper>
);
}
src/pages/admin/settings/backups/(partials)/db-backup-row-actions.tsx
+74
View File
@@ -0,0 +1,74 @@
import Button from "@moduletrace/twui/layout/Button";
import Tag from "@moduletrace/twui/elements/Tag";
import Row from "@moduletrace/twui/layout/Row";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import type { DbBackup } from "@/src/types";
import restoreBackup from "../(functions)/restore-backup";
import deleteDbBackup from "../(functions)/delete-db-backup";
import RowIconButton from "@/src/components/general/row-icon-button";
import { ArchiveRestore, Download, Trash2 } from "lucide-react";
type Props = {
backup: DbBackup;
};
/**
* One database snapshot's icon actions: restore, download or delete, with
* their own one-line status.
*/
export default function DbBackupRowActions({ backup }: Props) {
const { loading, setLoading, status, setStatus } = useStatus();
const run = async (task: () => Promise<{ success: boolean; msg?: string }>) => {
setStatus(undefined);
setLoading(true);
const res = await task();
if (res.success) {
window.location.reload();
return;
}
setLoading(false);
if (res.msg) {
setStatus({ error: true, msg: res.msg });
}
};
return (
<Row className="gap-1! flex-nowrap justify-end">
{status?.error && <Tag color="error">{status.msg}</Tag>}
<RowIconButton
icon={ArchiveRestore}
title="Restore this snapshot"
disabled={loading}
onClick={() => {
run(() => restoreBackup({ backup }));
}}
/>
<RowIconButton
icon={Download}
title="Download this snapshot"
href={`/api/admin/backups/download?backup_name=${encodeURIComponent(backup.name)}`}
linkProps={{
download: backup.name,
onClick: (e) => {
if (!window.confirm(`Download backup ${backup.name}?`)) {
e.preventDefault();
}
},
}}
/>
<RowIconButton
icon={Trash2}
title="Delete this snapshot"
disabled={loading}
onClick={() => {
run(() => deleteDbBackup({ backup }));
}}
/>
</Row>
);
}
src/pages/admin/settings/backups/(partials)/download-backup-button.tsx
-30
View File
@@ -1,30 +0,0 @@
import Button from "@moduletrace/twui/layout/Button";
import type { DbBackup } from "@/src/types";
import { Download } from "lucide-react";
type Props = {
backup: DbBackup;
};
export default function DownloadBackupButton({ backup }: Props) {
return (
<Button
title="Download Backup"
variant="outlined"
color="gray"
size="small"
beforeIcon={<Download size={20} />}
href={`/api/admin/backups/download?backup_name=${encodeURIComponent(backup.name)}`}
linkProps={{
download: backup.name,
onClick: (e) => {
if (!window.confirm(`Download backup ${backup.name}?`)) {
e.preventDefault();
}
},
}}
>
Download
</Button>
);
}
src/pages/admin/settings/backups/(partials)/restore-backup-button.tsx
-52
View File
@@ -1,52 +0,0 @@
import Button from "@moduletrace/twui/layout/Button";
import Tag from "@moduletrace/twui/elements/Tag";
import Row from "@moduletrace/twui/layout/Row";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import type { DbBackup } from "@/src/types";
import restoreBackup from "../(functions)/restore-backup";
import { ArchiveRestore } from "lucide-react";
type Props = {
backup: DbBackup;
};
export default function RestoreBackupButton({ backup }: Props) {
const { loading, setLoading, status, setStatus } = useStatus();
const restore_backup = async () => {
setStatus(undefined);
setLoading(true);
const res = await restoreBackup({ backup });
if (res.success) {
window.location.reload();
return;
}
setLoading(false);
if (res.msg) {
setStatus({ error: true, msg: res.msg });
}
};
return (
<Row className="gap-2! flex-nowrap justify-end">
{status?.error && <Tag color="error">{status.msg}</Tag>}
<Button
title="Restore Backup"
variant="outlined"
color="warning"
size="small"
beforeIcon={<ArchiveRestore size={20} />}
loading={loading}
onClick={() => {
restore_backup();
}}
>
Restore
</Button>
</Row>
);
}
src/pages/admin/settings/backups/restore/(partials)/upload-restore-form.tsx → src/pages/admin/settings/backups/(partials)/upload-restore-fields.tsx
+44 -11
View File
@@ -1,4 +1,3 @@
import { Upload } from "lucide-react";
import FileUpload from "@moduletrace/twui/form/FileUpload";
import Button from "@moduletrace/twui/layout/Button";
import Tag from "@moduletrace/twui/elements/Tag";
@@ -6,22 +5,56 @@ import Row from "@moduletrace/twui/layout/Row";
import Stack from "@moduletrace/twui/layout/Stack";
import useStatus from "@moduletrace/twui/hooks/useStatus";
import { useState } from "react";
import uploadRestoreBackup from "../(functions)/upload-restore-backup";
import { Upload } from "lucide-react";
export default function UploadRestoreForm() {
type Props = {
/**
* The file picker's hint, e.g. "Click or drop a SQLite database file".
*/
label: string;
/**
* What the picker accepts, e.g. `.db,.sqlite,.sqlite3`.
*/
accept?: string;
/**
* The button's label.
*/
action_label: string;
/**
* Sends the file and reports the outcome.
*/
upload: (params: { file: File }) => Promise<{ success: boolean; msg?: string }>;
/**
* Reload the page once it worked (the app restarts after an archive
* restore, so the caller waits for it first).
*/
onDone: () => void;
};
/**
* Picks one file and sends it for a restore: shared by the database upload
* and the data archive upload, which differ only in what they accept.
*/
export default function UploadRestoreFields({
label,
accept,
action_label,
upload,
onDone,
}: Props) {
const { loading, setLoading, status, setStatus } = useStatus();
const [file, setFile] = useState<File>();
const upload_restore = async () => {
const send = async () => {
if (!file) return;
setStatus(undefined);
setLoading(true);
const res = await uploadRestoreBackup({ file });
const res = await upload({ file });
if (res.success) {
window.location.reload();
onDone();
return;
}
@@ -35,9 +68,9 @@ export default function UploadRestoreForm() {
return (
<Stack className="w-full items-stretch gap-4">
<FileUpload
label="Click or drop a SQLite database file"
label={label}
className="h-[220px]"
fileInputProps={{ accept: ".db,.sqlite,.sqlite3" }}
fileInputProps={accept ? { accept } : undefined}
changeHandler={(file_data) => {
setStatus(undefined);
setFile(file_data?.file);
@@ -46,16 +79,16 @@ export default function UploadRestoreForm() {
<Row className="w-full justify-end gap-2! flex-nowrap">
{status?.error && <Tag color="error">{status.msg}</Tag>}
<Button
title="Upload And Restore Backup"
title={action_label}
color="warning"
beforeIcon={<Upload />}
loading={loading}
disabled={!file || loading}
onClick={() => {
upload_restore();
send();
}}
>
Upload & restore
{action_label}
</Button>
</Row>
</Stack>
src/pages/admin/settings/backups/(partials)/upload-restore-modal.tsx
+72
View File
@@ -0,0 +1,72 @@
import AddModalShell from "@/src/layouts/bots/(partials)/add-modal-shell";
import UploadRestoreFields from "./upload-restore-fields";
import uploadRestoreBackup from "../(functions)/upload-restore-backup";
import uploadRestoreDataBackup from "../(functions)/upload-restore-data-backup";
/**
* How long the app is given to come back after an archive restore
* restarted it before the page is reloaded anyway.
*/
const RELOAD_DELAY_MS = 8000;
type Props = {
open: boolean;
onClose: () => void;
/**
* True for an upload listing a data archive, which restarts the app.
*/
is_data?: boolean;
};
/**
* Uploading a file and restoring it, in a modal over the list it came
* from. A database file is restored in place; a data archive replaces the
* whole data volume, so the page waits for the app to restart.
*/
export default function UploadRestoreModal({ open, onClose, is_data }: Props) {
const reload = () => {
if (!is_data) {
window.location.reload();
return;
}
setTimeout(() => {
window.location.reload();
}, RELOAD_DELAY_MS);
};
return (
<AddModalShell
open={open}
setOpen={(next) => !next && onClose()}
title={is_data ? "Restore from an archive" : "Restore from a file"}
description={
is_data
? "Upload a whole-data tar.gz archive. It replaces every folder and the database snapshots, a safety archive is made first, and the app restarts."
: "Upload a SQLite database file. It is saved to the snapshots list, then restored."
}
>
{open ? (
<UploadRestoreFields
label={
is_data
? "Click or drop a data archive"
: "Click or drop a SQLite database file"
}
accept={
is_data
? ".tar.gz,.tgz,application/gzip"
: ".db,.sqlite,.sqlite3"
}
action_label={is_data ? "Upload & restore archive" : "Upload & restore"}
upload={
is_data
? uploadRestoreDataBackup
: uploadRestoreBackup
}
onDone={reload}
/>
) : null}
</AddModalShell>
);
}
src/pages/admin/settings/backups/(sections)/backups-list-section.tsx
+10 -18
View File
@@ -3,39 +3,31 @@ import Row from "@moduletrace/twui/layout/Row";
import Span from "@moduletrace/twui/layout/Span";
import SettingsGroup from "../../(partials)/settings-group";
import { AppContext } from "@/src/pages/__root";
import CreateBackupButton from "../(partials)/create-backup-button";
import BackupsTable, { type BackupsTableMode } from "../(partials)/backups-table";
type Props = {
mode: BackupsTableMode;
};
import BackupListActions from "../(partials)/backup-list-actions";
import BackupsTable from "../(partials)/backups-table";
/**
* The database snapshots, newest first. On the restore page each row
* restores; otherwise each one downloads, and a new backup can be made.
* The database snapshots, newest first. Each row restores, downloads or
* deletes, and a fresh one can be made or uploaded. These are the
* bun-sqlite snapshots in `db/.backups`, not the whole-data archives.
*/
export default function BackupsListSection({ mode }: Props) {
export default function BackupsListSection() {
const { pageProps } = useContext(AppContext);
const backups = pageProps?.backups || [];
const is_restore = mode == "restore";
return (
<SettingsGroup
title={is_restore ? "Saved snapshots" : "Snapshots"}
description={
is_restore
? "Restore any of these, or upload a file below. Create a fresh backup first if you might need the current data."
: "Older snapshots are pruned automatically. Download one to keep it."
}
title="Database snapshots"
description="Consistent snapshots of the database only, kept beside it. Older ones are pruned automatically. Download one to keep it."
>
<Row className="w-full items-center justify-between gap-2! flex-wrap">
<Span size="small" variant="faded">
{`${backups.length} backup${backups.length == 1 ? "" : "s"}`}
</Span>
{is_restore ? null : <CreateBackupButton />}
<BackupListActions />
</Row>
{backups[0] ? (
<BackupsTable backups={backups} mode={mode} />
<BackupsTable backups={backups} />
) : (
<Span size="small" variant="faded">
No backups yet.
src/pages/admin/settings/backups/(sections)/data-backups-section.tsx
+39
View File
@@ -0,0 +1,39 @@
import { useContext } from "react";
import Row from "@moduletrace/twui/layout/Row";
import Span from "@moduletrace/twui/layout/Span";
import SettingsGroup from "../../(partials)/settings-group";
import { AppContext } from "@/src/pages/__root";
import DataBackupListActions from "../(partials)/data-backup-list-actions";
import DataBackupsTable from "../(partials)/data-backups-table";
/**
* The tar.gz archives of the whole data volume, newest first. Each one
* restores (replacing every folder and the database snapshots with the
* archive's own, then the app restarts), downloads or deletes; the newest
* `AppData.DataBackupsKept` are kept and older ones pruned.
*/
export default function DataBackupsSection() {
const { pageProps } = useContext(AppContext);
const backups = pageProps?.data_backups || [];
return (
<SettingsGroup
title="Data archives"
description="One tar.gz of the whole data volume, under a single folder named after the archive: the database, media, computer homes, every bot and workspace file, and the database snapshots. They live outside the data volume, and the newest few are kept."
>
<Row className="w-full items-center justify-between gap-2! flex-wrap">
<Span size="small" variant="faded">
{`${backups.length} archive${backups.length == 1 ? "" : "s"}`}
</Span>
<DataBackupListActions />
</Row>
{backups[0] ? (
<DataBackupsTable backups={backups} />
) : (
<Span size="small" variant="faded">
No archives yet.
</Span>
)}
</SettingsGroup>
);
}
src/pages/admin/settings/backups/(sections)/restore-backups-section.tsx
-28
View File
@@ -1,28 +0,0 @@
import Button from "@moduletrace/twui/layout/Button";
import { ArchiveRestore } from "lucide-react";
import SettingsGroup from "../../(partials)/settings-group";
import { SettingsBackupsRestorePath } from "@/src/dict/settings-pages-dict";
/**
* The way into the restore page from the backups list.
*/
export default function RestoreBackupsSection() {
return (
<SettingsGroup
title="Restore"
description="Replace all current data with the contents of a saved or uploaded backup. This cannot be undone."
>
<Button
title="Restore backup"
href={SettingsBackupsRestorePath}
variant="outlined"
color="warning"
size="small"
beforeIcon={<ArchiveRestore size={20} />}
className="self-start"
>
Restore a backup
</Button>
</SettingsGroup>
);
}
src/pages/admin/settings/backups/index.tsx
+4 -4
View File
@@ -1,5 +1,5 @@
import BackupsListSection from "./(sections)/backups-list-section";
import RestoreBackupsSection from "./(sections)/restore-backups-section";
import DataBackupsSection from "./(sections)/data-backups-section";
import SettingsShell from "../(partials)/settings-shell";
import settingsPageMeta from "../(functions)/settings-page-meta";
@@ -7,11 +7,11 @@ export default function AdminSettingsBackupsPage() {
return (
<SettingsShell
title="Backups"
description="Snapshots of the database, and restoring one."
description="Database snapshots and whole-data archives: make them, restore from one, and delete them."
wide
>
<BackupsListSection mode="view" />
<RestoreBackupsSection />
<BackupsListSection />
<DataBackupsSection />
</SettingsShell>
);
}
src/pages/admin/settings/backups/restore/(sections)/upload-restore-section.tsx
-16
View File
@@ -1,16 +0,0 @@
import SettingsGroup from "../../../(partials)/settings-group";
import UploadRestoreForm from "../(partials)/upload-restore-form";
/**
* Restoring from a file on this machine, for a backup kept elsewhere.
*/
export default function UploadRestoreSection() {
return (
<SettingsGroup
title="From a file"
description="Upload a SQLite database file. It is saved to the backups list, then restored."
>
<UploadRestoreForm />
</SettingsGroup>
);
}
src/pages/admin/settings/backups/restore/index.server.ts
-7
View File
@@ -1,7 +0,0 @@
import type { PagePropsType } from "@/src/types";
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
import grabBackupsServerProps from "../(functions)/grab-backups-server-props";
export const server: BunextPageServerFn<PagePropsType> = async (params) => {
return await grabBackupsServerProps(params);
};
src/pages/admin/settings/backups/restore/index.tsx
-26
View File
@@ -1,26 +0,0 @@
import UploadRestoreSection from "./(sections)/upload-restore-section";
import BackupsListSection from "../(sections)/backups-list-section";
import SettingsShell from "../../(partials)/settings-shell";
import type { BunextPageModuleMetaFn } from "@moduletrace/bunext/types";
import { SiteData } from "@/src/data/site-data";
export default function AdminSettingsRestoreBackupPage() {
return (
<SettingsShell
title="Restore backup"
description="Replace all current data with a saved or uploaded backup."
breadcrumb_title="Restore backup"
wide
>
<UploadRestoreSection />
<BackupsListSection mode="restore" />
</SettingsShell>
);
}
export const meta: BunextPageModuleMetaFn = async () => {
return {
title: `Restore backup | ${SiteData["SiteName"]}`,
description: `Restore a database backup on ${SiteData["SiteName"]}.`,
};
};
src/pages/api/admin/backups/create-data.ts
+52
View File
@@ -0,0 +1,52 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import createDataBackup from "@/src/functions/backend/backups/create-data-backup";
import logEvent, { grabRequestIP, logError } from "@/src/functions/backend/logger";
import type {
APIResponseObject,
BunextAPIRouteHandler,
} from "@moduletrace/bunext/types";
/**
* Archives the whole `DATA_DIR` (Settings > Backups > Data archives).
*/
export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
req,
}) => {
const user = await grabSuperAdminUser({ req });
if (!user?.id) {
return {
success: false,
msg: `Unauthorized`,
};
}
try {
const backup = await createDataBackup();
await logEvent({
title: "Data backup created",
category: "system",
user_id: user.id,
plain_text: `User #${user.id} created a data backup (${backup.name}).`,
metadata: { backup_name: backup.name },
ip_address: grabRequestIP(req),
});
return {
success: true,
};
} catch (error: any) {
await logError({
title: "Data backup failed",
error,
user_id: user.id,
ip_address: grabRequestIP(req),
});
return {
success: false,
msg: error.message,
};
}
};
src/pages/api/admin/backups/delete-data.ts
+57
View File
@@ -0,0 +1,57 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import deleteDataBackup from "@/src/functions/backend/backups/delete-data-backup";
import logEvent, { grabRequestIP, logError } from "@/src/functions/backend/logger";
import type { ApiReqParams } from "@/src/types";
import type {
APIResponseObject,
BunextAPIRouteHandler,
} from "@moduletrace/bunext/types";
/**
* Deletes one whole-data archive.
*/
export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
req,
body,
}) => {
const user = await grabSuperAdminUser({ req });
if (!user?.id) {
return {
success: false,
msg: `Unauthorized`,
};
}
const backup_name = (body as ApiReqParams | undefined)?.backup_name;
try {
deleteDataBackup({ backup_name });
await logEvent({
title: "Data backup deleted",
category: "system",
user_id: user.id,
plain_text: `User #${user.id} deleted data archive ${backup_name}.`,
metadata: { backup_name },
ip_address: grabRequestIP(req),
});
return {
success: true,
};
} catch (error: any) {
await logError({
title: "Data backup deletion failed",
error,
user_id: user.id,
metadata: { backup_name },
ip_address: grabRequestIP(req),
});
return {
success: false,
msg: error.message,
};
}
};
src/pages/api/admin/backups/delete-db.ts
+57
View File
@@ -0,0 +1,57 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import deleteDbBackup from "@/src/functions/backend/backups/delete-db-backup";
import logEvent, { grabRequestIP, logError } from "@/src/functions/backend/logger";
import type { ApiReqParams } from "@/src/types";
import type {
APIResponseObject,
BunextAPIRouteHandler,
} from "@moduletrace/bunext/types";
/**
* Deletes one database snapshot from `db/.backups`.
*/
export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
req,
body,
}) => {
const user = await grabSuperAdminUser({ req });
if (!user?.id) {
return {
success: false,
msg: `Unauthorized`,
};
}
const backup_name = (body as ApiReqParams | undefined)?.backup_name;
try {
deleteDbBackup({ backup_name });
await logEvent({
title: "Database backup deleted",
category: "system",
user_id: user.id,
plain_text: `User #${user.id} deleted database backup ${backup_name}.`,
metadata: { backup_name },
ip_address: grabRequestIP(req),
});
return {
success: true,
};
} catch (error: any) {
await logError({
title: "Database backup deletion failed",
error,
user_id: user.id,
metadata: { backup_name },
ip_address: grabRequestIP(req),
});
return {
success: false,
msg: error.message,
};
}
};
src/pages/api/admin/backups/download-data.ts
+43
View File
@@ -0,0 +1,43 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import resolveDataBackupPath from "@/src/functions/backend/backups/resolve-data-backup-path";
import type { ApiReqParams } from "@/src/types";
import type {
APIResponseObject,
BunextAPIRouteHandler,
} from "@moduletrace/bunext/types";
import path from "path";
/**
* Streams a whole-data archive back, by name.
*/
export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
req,
query,
}) => {
const user = await grabSuperAdminUser({ req });
if (!user?.id) {
return {
success: false,
msg: `Unauthorized`,
};
}
try {
const backup_path = resolveDataBackupPath({
backup_name: (query as ApiReqParams | undefined)?.backup_name,
});
return new Response(Bun.file(backup_path), {
headers: {
"Content-Type": "application/gzip",
"Content-Disposition": `attachment; filename="${path.basename(backup_path)}"`,
},
});
} catch (error: any) {
return {
success: false,
msg: error.message,
};
}
};
src/pages/api/admin/backups/restore-data.ts
+66
View File
@@ -0,0 +1,66 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import restoreDataBackup from "@/src/functions/backend/backups/restore-data-backup";
import restartApp from "@/src/functions/backend/backups/restart-app";
import logEvent, { grabRequestIP, logError } from "@/src/functions/backend/logger";
import type { ApiReqParams } from "@/src/types";
import type {
APIResponseObject,
BunextAPIRouteHandler,
} from "@moduletrace/bunext/types";
/**
* Replaces the whole data volume with a saved archive, then restarts the
* app so it opens the restored database.
*/
export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
req,
body,
}) => {
const user = await grabSuperAdminUser({ req });
if (!user?.id) {
return {
success: false,
msg: `Unauthorized`,
};
}
const backup_name = (body as ApiReqParams | undefined)?.backup_name;
try {
const result = await restoreDataBackup({ backup_name });
await logEvent({
title: "Data restored",
category: "system",
user_id: user.id,
plain_text: `User #${user.id} restored the data volume from archive ${result.restored_name} (safety copy ${result.safety_name}). The app restarted.`,
metadata: {
backup_name: result.restored_name,
safety_name: result.safety_name,
},
ip_address: grabRequestIP(req),
});
restartApp();
return {
success: true,
};
} catch (error: any) {
await logError({
title: "Data restore failed",
error,
user_id: user.id,
metadata: {
backup_name,
},
ip_address: grabRequestIP(req),
});
return {
success: false,
msg: error.message,
};
}
};
src/pages/api/admin/backups/upload-restore-data.ts
+82
View File
@@ -0,0 +1,82 @@
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
import restoreDataBackup from "@/src/functions/backend/backups/restore-data-backup";
import restartApp from "@/src/functions/backend/backups/restart-app";
import saveUploadedDataBackup from "@/src/functions/backend/backups/save-uploaded-data-backup";
import logEvent, { grabRequestIP, logError } from "@/src/functions/backend/logger";
import type {
APIResponseObject,
BunextAPIRouteHandler,
BunextServerRouteConfig,
} from "@moduletrace/bunext/types";
export const config: BunextServerRouteConfig = {
max_request_body_mb: 500,
};
/**
* Multipart upload (`file` field) of a whole-data archive. The file is
* saved into the archive directory, then restored, and the app restarts.
*/
export const handler: BunextAPIRouteHandler<APIResponseObject> = async ({
req,
}) => {
const user = await grabSuperAdminUser({ req });
if (!user?.id) {
return {
success: false,
msg: `Unauthorized`,
};
}
let file_name: string | undefined;
let backup_name: string | undefined;
try {
const file = (await req.formData()).get("file");
if (!(file instanceof File)) {
throw new Error(`No archive uploaded`);
}
file_name = file.name;
backup_name = await saveUploadedDataBackup({ file });
const result = await restoreDataBackup({ backup_name });
await logEvent({
title: "Data restored from upload",
category: "system",
user_id: user.id,
plain_text: `User #${user.id} restored the data volume from uploaded archive ${file_name} (saved as ${result.restored_name}, safety copy ${result.safety_name}). The app restarted.`,
metadata: {
file_name,
backup_name: result.restored_name,
safety_name: result.safety_name,
},
ip_address: grabRequestIP(req),
});
restartApp();
return {
success: true,
};
} catch (error: any) {
await logError({
title: "Data restore from upload failed",
error,
user_id: user.id,
metadata: {
file_name,
backup_name,
},
ip_address: grabRequestIP(req),
});
return {
success: false,
msg: error.message,
};
}
};
src/types/index.ts
+12
View File
@@ -865,6 +865,17 @@ export type DbBackup = {
size: number;
};
/**
* A tar.gz archive of the whole `DATA_DIR` in `BACKUP_ARCHIVE_DIR`
* (outside the data volume), newest first. Restoring one is an offline
* operation: the app only creates, lists and downloads them.
*/
export type DataBackup = {
name: string;
created_at: number;
size: number;
};
export type BotVoicePayload = {
transcription: boolean;
speech: boolean;
@@ -967,6 +978,7 @@ export type PagePropsType = {
environment?: string;
has_password?: boolean;
backups?: DbBackup[] | null;
data_backups?: DataBackup[] | null;
harness_overview?: HarnessOverviewPayload | null;
bot_harness_overview?: BotHarnessOverviewPayload | null;
envs_overview?: EnvsOverviewPayload | null;
src/utils/grab-backup-archive-dir.ts
+27
View File
@@ -0,0 +1,27 @@
import fs from "fs";
import os from "os";
import path from "path";
/**
* Absolute path of the directory the whole-data archives are written to.
* It is deliberately outside `DATA_DIR` (an archive must never contain
* other archives) and outside the database's own `db/.backups`
* directory, which keeps its bun-sqlite snapshots.
*
* `BACKUP_ARCHIVE_DIR` is the container's or install's setting (the
* installer mounts `/backups-archives` from `$COBBLE_HOME/backups-archives`);
* outside docker it falls back to `~/.cobble/backups`. It is a start-up
* path like `DATA_DIR`, so it stays in the process environment and is
* never an `envs` row.
*/
export default function grabBackupArchiveDir() {
const dir =
process.env.BACKUP_ARCHIVE_DIR ||
path.join(os.homedir(), ".cobble", "backups");
if (!fs.existsSync(dir)) {
fs.mkdirSync(dir, { recursive: true });
}
return path.resolve(dir);
}