Let a super admin read the database from the app
A new super admin settings page, /admin/settings/database, lists every table with its record count and size, and one page of the open table's records newest key first. Values are cut inside SQL (400 characters, with a flag saying so) and a record opens whole in a modal, so a table of long texts reads without shipping it all. A search matches every column's text, word by word or as a phrase, which is the one thing that also works on a virtual table. Nothing in it writes: the catalog, the records and the single record are all reads, GET only.
This commit is contained in:
@@ -216,6 +216,7 @@ Live routes under `/admin` (see `AdminSections`): `/admin/bots`, `/admin/workspa
|
||||
- `/admin/help` (live) - familiar problems and their fixes, one `SettingsGroup` per topic (`(data)/help-topics.ts`, markdown rendered with the chat's `MessageMarkdownComponents`, the topic id as the group's anchor). First topic: signing in to Google and other strict sites with Cookie-Editor over the Desktop view (`HelpSignInTopicId` in `src/dict/help-dict.ts`; `ask_user_to_take_over` points bots to it)
|
||||
- Routines, Files, Activity and Help are the `RailPages` (`src/dict/rail-pages-dict.ts`), opened from the rail's user popover (which lists them, then Settings and Sign out). On them the rail shows its own aside (`(partials)/rail-pages-list.tsx`): a back link to the bots, then those pages, like settings
|
||||
- `/admin/settings/backups` - super admin only (listed in `SettingsPages`), two groups. Database snapshots: create, list, restore, download and delete in `db/.backups` (`bun-sqlite backup`); "Restore from file" beside "Create backup" opens `upload-restore-modal.tsx` (upload a SQLite database file, which is saved to `.backups` then restored with `bun-sqlite import <file>`, since `restore` is interactive). Whole-data archives: `write-data-backup-archive.ts` tars all of `DATA_DIR` (database, media, computer homes, bot and workspace files, MCP installs) into `BACKUP_ARCHIVE_DIR` as `cobble-data-<epoch-ms>.tar.gz` (via `tar -czf` into a `.part` name, renamed on success so a cut-off run is never listed, with a `--transform` putting every member under one root folder named after the archive), keeping the newest `AppData.DataBackupsKept` and never touching `db/.backups`; its own "Restore from file" uploads a tar.gz (`upload-restore-data.ts`, gzip header checked first). `restore-data-backup.ts` extracts an archive to a temp dir, requires exactly one root folder holding a `db` folder (`findArchiveRoot`, since an upload is saved under a fresh timestamp and its inner name no longer matches the file name), takes a safety archive before touching anything (`write-data-backup-archive.ts`, not the locked `create-data-backup.ts`), empties `DATA_DIR` (the snapshots included) and copies the archive's in, then `restart-app.ts` restarts the process through the docker socket after the response lands (`process.exit`, the container's restart policy brings it back), because the open SQLite handle points at the replaced file. Row actions are icon-only (`RowIconButton`, `src/components/general/row-icon-button.tsx`): restore, download, delete. `BACKUP_ARCHIVE_DIR` is a `ProcessOnlyEnvKeys` start-up path (the installer mounts `$COBBLE_HOME/backups-archives` there; outside docker it falls back to `~/.cobble/backups`). Every backup and restore, of either kind, runs one at a time via `with-backup-run-lock.ts`. Helpers in `src/functions/backend/backups/`, API routes in `api/admin/backups/`
|
||||
- `/admin/settings/database` - super admin only (listed in `SettingsPages`), a read-only browser of the install's SQLite database, no writes anywhere in it. `grabDatabaseTables` reads the catalog once (every table and its columns, SQLite's own `sqlite_*` tables and sqlite-vec's `bot_file_vectors_*` shadow tables left out) and adds each table's `COUNT(*)` and its size from `dbstat` (index pages folded back onto their table). `grabTableRecords` then returns one page (`AppData.DatabaseBrowserPageSize`, 50) of the open table's records newest key first, every value already cut in SQL at `AppData.DatabaseBrowserCellMaxChars` (400) so a table of long texts never ships a megabyte of JSON, and a `truncated` flag per cell; the search matches over the concatenation of every column's text (`buildSearchClause`: every word, or the whole phrase), which is what also works on a virtual table, where `MATCH` and `LIKE` over a vector blob do not. A record's key is its single primary key column, else its rowid, else nothing (a virtual table with no primary key reads as plain cells); clicking a cell's row opens `grabTableRecord` for that key, which returns the record whole up to `AppData.DatabaseBrowserValueMaxChars` (20000), the answer to a cell cut at three lines. `clampPage` puts a page past the end on the last one. The page's state (table, search, mode, page) lives in the address through `databaseHref`, so a table or a search can be linked to; the open table's first page comes with the page props and anything else loads through the API. Helpers in `src/functions/backend/database/` (`build-table-catalog-sql`, `build-table-sizes-sql`, `build-records-sql`, `build-search-clause`, `resolve-table-key`, `build-record-row`, `quote-sql-identifier`), API `api/admin/database/records` and `record` (GET only, `grabSuperAdminUser`), zod in `src/schemas/database-schema.ts`, page under `src/pages/admin/settings/database/`
|
||||
- `/admin/settings` (live) - redirects to `SettingsDefaultPath` (`/admin/settings/general`). Sub pages: `general/` (theme: System/Light/Dark per device via `useThemePreference`, stored in `localStorage[AppData.ThemeStorageKey]`, the same key `ThemeInitScript` and twui's `ColorSchemeSelector` read; applies instantly, no save bar; for super admins the prompt appendix (`prompt-section.tsx`, `system-prompt-append-form.tsx`) saving `api/admin/settings/system-prompt-append` into the `bot_system_prompt_append` app setting; and, for `canManageCrew`, the bot file quota: `storage-section.tsx`, `storage-quota-form.tsx` saving `api/admin/settings/storage-quota` into `app_settings`; and, for everyone, the running version: `version-section.tsx`, `version-line.tsx` showing `pageProps.running_version` from `runningVersion` (`src/utils/running-version.ts`: `COBBLE_VERSION`, the release tag the image was built with, else `package.json`'s version, in development too, with `(development)` after it and no such suffix in production), `profile/` (current user profile), `notifications/` (this device's Web Push switch, a test send, and every device the user subscribed), `change-password/` (set/change own password) `models/` (super admin only, `super_admin_only` in the dict hides it from the rail: model providers and which model each harness uses) and `environment/` (super admin only: app settings and the envs bots and computers use, per scope), each wrapped in `SettingsShell` (`(partials)/settings-shell.tsx`: 64px pane header like the conversation header, then one centered column). Content is built from `SettingsGroup` blocks (label + description left, controls right, hairline-separated, no cards) and ends with the sticky `SettingsSaveBar` (Save enabled only when dirty, Discard, one-line status). Saves don't reload the page. Keep save notices in component state: `useFormInit` clears `status` whenever the form changes. `bots/` (the crew: chat, edit with `BotForm`, models, delete a worker via `api/admin/bots/delete-bot`), `workspaces/` (open, create through the Add flow, delete), `computers/` (state and resources, live; super admins create, edit with `AddComputerForm`, retry, delete), `mcps/`, `tools/` and `skills/` (see MCPs, custom tools and skills: super admins add, edit, refresh or try, delete; admins only set Access, `extension-access-modal.tsx`; add, edit, access and try open in modals (`AddModalShell`). A skill has no status: nothing connects or runs, so `skill-row.tsx` puts its slug, who gets it (`extension-access-label.ts`) and its description where a server's or tool's status line goes), each MCP server or tool row with a status line under it (`extension-status-line.tsx`: connected or not, last run worked or failed, when, and why), those five for `canManageCrew` only (`grab-crew-settings-server-props.ts`, `can_view` in the dict) and `users/` (for `canListUsers`: super admins and admins; the dict's `can_view` gates it in the rail; add with a password (`canCreateUser`), edit (`canUpdateUser`), both with the one `UserForm` (`admin/settings/users/(partials)/user-form/`, `useFormInit`, photo included, saved in one request), revoke (adds `revoked`, `canRevokeUser`), delete (`canRevokeUser`'s rules, never yourself), pick their user types from `UserTypes` (`user-types-select.tsx`: only the ones `filterAssignableUserTypes` allows, never `UneditableUserTypes`; at least one, since login needs a type; `saveUser` checks each grant and removal with `userTypeAssignment`; the super admin's types are never shown or changed there, and they can't be revoked; the password is entered twice, `SaveUserRequestSchema` checks they match), delete with everything theirs; `api/admin/users/save-user`, `delete-user`). The lists share `SettingsList`, `SettingsListRow`, `SettingsRowAction` and `use-settings-list`. Sub pages are declared once in `src/dict/settings-pages-dict.ts`; on every `/admin/settings/*` page the rail swaps the crew for a back link plus that list. Add a sub page by adding its folder and a dict entry
|
||||
|
||||
Every `/admin/*` page renders in the bots layout (`src/layouts/bots/`); there is no separate admin layout or aside. The rail header's Add icon (super admins and admins only, like the rail's "+" buttons) opens the Add flow (`(partials)/add-flow.tsx`, `AddFlowContext`): a bot, a workspace or (super admins) a computer, each form in a modal stacked over the choice, so closing it goes back there (the Bots and Workspaces labels have a "+" that opens the form directly; in a workspace's rail, "Workspace bots" has a "+" that opens `link-bots-modal.tsx` instead: the crew's workers not in it yet (`api/admin/workspaces/linkable-bots`), each added with `add-bot`, which pushes `server:bot-created`, and "New bot" stacking the bot form over it); on a workspace's pages new bots and computers join it. The rail shows the crew (`(partials)/rail-crew-list.tsx`, needs `bots` and `workspaces` in page props): the coordinator, the latest `AppData.RailRecentBots` workers (plus any open, unread or needing you, `pick-rail-bots`), "See all bots" with a few stones opening every bot in a modal, then Workspaces with a "+". On `/admin/workspaces/*` it shows the workspace's crew instead (`(partials)/rail-workspace-list.tsx`: back link, the workspace, then only the bots attached to it, "Workspace bots"; the coordinator stays in the main crew rail, and on a workspace bot's page the back link returns to the workspace page), except on `/admin/settings/*` where it shows "Back to bots" plus the settings sub pages (`(partials)/rail-settings-list.tsx`). Page heroes use the shared `AdminHero` component (`src/components/general/admin-hero.tsx`).
|
||||
|
||||
@@ -451,6 +451,20 @@ export const AppData = {
|
||||
*/
|
||||
StorageBrowserPageSize: 200,
|
||||
StorageBrowserReadMaxBytes: 50 * 1024 * 1024,
|
||||
/**
|
||||
* The database browser at `/admin/settings/database`: how many records
|
||||
* of a table one page shows, how much of a value a cell carries before
|
||||
* the rest waits behind its record's modal, and the most of one value
|
||||
* that modal shows (a whole media row or job checkpoint is not meant
|
||||
* to be rendered).
|
||||
*/
|
||||
DatabaseBrowserPageSize: 50,
|
||||
DatabaseBrowserCellMaxChars: 400,
|
||||
DatabaseBrowserValueMaxChars: 20000,
|
||||
/**
|
||||
* The most words one search matches in the database browser.
|
||||
*/
|
||||
DatabaseBrowserMaxSearchWords: 8,
|
||||
/**
|
||||
* The most a bot copies off a computer into its own files in one
|
||||
* call, so a folder of raw evidence can't fill the data volume.
|
||||
|
||||
@@ -2,6 +2,7 @@ import {
|
||||
BookOpen,
|
||||
Bot,
|
||||
CpuAi,
|
||||
Database,
|
||||
DatabaseBackup,
|
||||
KeyRound,
|
||||
Layers,
|
||||
@@ -34,6 +35,7 @@ export type SettingsPage = {
|
||||
|
||||
export const SettingsComputersPath = "/admin/settings/computers";
|
||||
export const SettingsBackupsPath = "/admin/settings/backups";
|
||||
export const SettingsDatabasePath = "/admin/settings/database";
|
||||
|
||||
/**
|
||||
* The `/admin/settings/*` sub pages, listed in the rail on every settings
|
||||
@@ -81,6 +83,12 @@ export const SettingsPages: SettingsPage[] = [
|
||||
Icon: DatabaseBackup,
|
||||
super_admin_only: true,
|
||||
},
|
||||
{
|
||||
title: "Database",
|
||||
url: SettingsDatabasePath,
|
||||
Icon: Database,
|
||||
super_admin_only: true,
|
||||
},
|
||||
];
|
||||
|
||||
export const SettingsDefaultPath = "/admin/settings/general";
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import quoteSqlIdentifier from "./quote-sql-identifier";
|
||||
import resolveTableKey from "./resolve-table-key";
|
||||
import type { DatabaseTableSummary } from "@/src/types";
|
||||
import type { TableCatalogEntry } from "./table-catalog-types";
|
||||
|
||||
type Params = {
|
||||
tables: TableCatalogEntry[];
|
||||
/**
|
||||
* How many bytes each table takes on disk, by name; empty when SQLite
|
||||
* can't measure it, which leaves the size column blank.
|
||||
*/
|
||||
sizes: Map<string, number>;
|
||||
};
|
||||
|
||||
type CountRow = { rows: number };
|
||||
|
||||
/**
|
||||
* The catalog as the browser lists it: every table with its record count,
|
||||
* its size on disk, its columns and what identifies its records.
|
||||
*
|
||||
* Each count is its own tiny statement rather than one aggregate, because
|
||||
* a virtual table has no rowid to build one from, and the whole read stays
|
||||
* in the low milliseconds for the install's fifty-odd tables.
|
||||
*/
|
||||
export default async function buildDatabaseTables({ tables, sizes }: Params): Promise<DatabaseTableSummary[]> {
|
||||
const summaries: DatabaseTableSummary[] = [];
|
||||
|
||||
for (const table of tables) {
|
||||
const size = sizes.get(table.name);
|
||||
|
||||
summaries.push({
|
||||
name: table.name,
|
||||
type: table.type,
|
||||
rows: await countRows({ table: table.name }),
|
||||
size_bytes: size === undefined ? null : size,
|
||||
columns: table.columns,
|
||||
key: resolveTableKey({ columns: table.columns, has_rowid: table.has_rowid }),
|
||||
});
|
||||
}
|
||||
|
||||
return summaries;
|
||||
}
|
||||
|
||||
async function countRows({ table }: { table: string }): Promise<number | null> {
|
||||
const res = await BunSQLite.sql<CountRow>({
|
||||
sql: `SELECT COUNT(*) AS rows FROM ${quoteSqlIdentifier({ name: table })}`,
|
||||
});
|
||||
|
||||
if (!res.success || !res.payload?.[0]) {
|
||||
/**
|
||||
* A table the module won't count (an index over a table whose
|
||||
* module is missing) is still a table; it just has no count.
|
||||
*/
|
||||
return null;
|
||||
}
|
||||
|
||||
return Number(res.payload[0].rows) || 0;
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import { metaColumnAlias, RecordKeyAlias } from "./build-records-sql";
|
||||
import toDatabaseCell from "./to-database-cell";
|
||||
import type { DatabaseRecord, DatabaseTableColumn } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The row as SQLite returned it, its meta columns included.
|
||||
*/
|
||||
row: Record<string, unknown>;
|
||||
columns: DatabaseTableColumn[];
|
||||
max_chars: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* One row of a records query as a record the browser can draw: its key,
|
||||
* and every column's value as a cell. The meta columns the query added
|
||||
* (`typeof` and `length` per column) are read and dropped here, so they
|
||||
* never reach the client.
|
||||
*/
|
||||
export default function buildRecordRow({ row, columns, max_chars }: Params): DatabaseRecord {
|
||||
const cells: Record<string, DatabaseRecord["cells"][string]> = {};
|
||||
|
||||
for (const column of columns) {
|
||||
cells[column.name] = toDatabaseCell({
|
||||
value: row[column.name],
|
||||
type: String(row[metaColumnAlias({ name: column.name, kind: "type" })] ?? "null"),
|
||||
length: Number(row[metaColumnAlias({ name: column.name, kind: "length" })] ?? 0),
|
||||
max_chars,
|
||||
});
|
||||
}
|
||||
|
||||
const key = row[RecordKeyAlias];
|
||||
|
||||
return {
|
||||
key:
|
||||
typeof key == "string" || typeof key == "number"
|
||||
? key
|
||||
: key === null || key === undefined
|
||||
? null
|
||||
: String(key),
|
||||
cells,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,337 @@
|
||||
import { Database } from "bun:sqlite";
|
||||
import { beforeAll, describe, expect, test } from "bun:test";
|
||||
import buildRecordsSql, {
|
||||
buildCountSql,
|
||||
buildRecordSql,
|
||||
buildRecordSelect,
|
||||
metaColumnAlias,
|
||||
parseRecordKey,
|
||||
RecordKeyAlias,
|
||||
} from "./build-records-sql";
|
||||
import buildSearchClause, { searchWords } from "./build-search-clause";
|
||||
import buildRecordRow from "./build-record-row";
|
||||
import buildTableCatalogSql from "./build-table-catalog-sql";
|
||||
import groupTableCatalog from "./group-table-catalog";
|
||||
import resolveTableKey from "./resolve-table-key";
|
||||
import quoteSqlIdentifier from "./quote-sql-identifier";
|
||||
import type { TableCatalogRow } from "./table-catalog-types";
|
||||
import type { DatabaseTableColumn } from "@/src/types";
|
||||
|
||||
const db = new Database(":memory:");
|
||||
|
||||
const messages_columns: DatabaseTableColumn[] = [
|
||||
{ name: "id", type: "INTEGER", pk: true },
|
||||
{ name: "role", type: "TEXT", pk: false },
|
||||
{ name: "text", type: "TEXT", pk: false },
|
||||
];
|
||||
|
||||
beforeAll(() => {
|
||||
db.run(`CREATE TABLE "messages" ("id" INTEGER PRIMARY KEY, "role" TEXT, "text" TEXT)`);
|
||||
db.run(`INSERT INTO "messages" VALUES
|
||||
(1, 'user', 'hello there'),
|
||||
(2, 'bot', 'Hello there, how can I help?'),
|
||||
(3, 'user', 'a "quoted" table name and a long body'),
|
||||
(4, 'bot', NULL)`);
|
||||
db.run(`CREATE TABLE "no_key" ("a" TEXT, "b" TEXT)`);
|
||||
db.run(`INSERT INTO "no_key" VALUES ('one', 'two')`);
|
||||
});
|
||||
|
||||
function rows({ sql, values, max_chars = 400 }: { sql: string; values: string[]; max_chars?: number }) {
|
||||
return db
|
||||
.query(sql)
|
||||
.all(...values)
|
||||
.map((row) =>
|
||||
buildRecordRow({ row: row as Record<string, unknown>, columns: messages_columns, max_chars }),
|
||||
);
|
||||
}
|
||||
|
||||
describe("quoteSqlIdentifier", () => {
|
||||
test("wraps a plain name", () => {
|
||||
expect(quoteSqlIdentifier({ name: "bots" })).toBe(`"bots"`);
|
||||
});
|
||||
|
||||
test("doubles a quote instead of letting it out", () => {
|
||||
expect(quoteSqlIdentifier({ name: `a"b` })).toBe(`"a""b"`);
|
||||
});
|
||||
});
|
||||
|
||||
describe("resolveTableKey", () => {
|
||||
test("a single primary key column names the records", () => {
|
||||
expect(resolveTableKey({ columns: messages_columns, has_rowid: true })).toEqual({ kind: "column", name: "id" });
|
||||
});
|
||||
|
||||
test("no primary key falls back to the rowid", () => {
|
||||
expect(resolveTableKey({ columns: [{ name: "a", type: "TEXT", pk: false }], has_rowid: true })).toEqual({
|
||||
kind: "rowid",
|
||||
});
|
||||
});
|
||||
|
||||
test("a virtual table with neither has no key", () => {
|
||||
expect(resolveTableKey({ columns: [{ name: "a", type: "TEXT", pk: false }], has_rowid: false })).toEqual({
|
||||
kind: "none",
|
||||
});
|
||||
});
|
||||
|
||||
test("a composite primary key falls back to the rowid", () => {
|
||||
expect(
|
||||
resolveTableKey({
|
||||
columns: [
|
||||
{ name: "a", type: "TEXT", pk: true },
|
||||
{ name: "b", type: "TEXT", pk: true },
|
||||
],
|
||||
has_rowid: true,
|
||||
}),
|
||||
).toEqual({ kind: "rowid" });
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildRecordsSql", () => {
|
||||
test("returns a page of records newest key first", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns: messages_columns.map((column) => column.name),
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
limit: 2,
|
||||
offset: 0,
|
||||
search: null,
|
||||
});
|
||||
const records = rows(query);
|
||||
|
||||
expect(records.map((record) => record.key)).toEqual([4, 3]);
|
||||
expect(records[0]?.cells["text"]?.kind).toBe("null");
|
||||
expect(records[1]?.cells["role"]?.text).toBe("user");
|
||||
});
|
||||
|
||||
test("pages with the offset it is given", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns: messages_columns.map((column) => column.name),
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
limit: 2,
|
||||
offset: 2,
|
||||
search: null,
|
||||
});
|
||||
|
||||
expect(rows(query).map((record) => record.key)).toEqual([2, 1]);
|
||||
});
|
||||
|
||||
test("cuts a long value and says so", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns: messages_columns.map((column) => column.name),
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 4,
|
||||
limit: 1,
|
||||
offset: 1,
|
||||
search: null,
|
||||
});
|
||||
const [record] = rows({ ...query, max_chars: 4 });
|
||||
|
||||
expect(record?.cells["text"]?.text).toBe(`a "q`); // four characters, cut
|
||||
expect(record?.cells["text"]?.truncated).toBe(true);
|
||||
});
|
||||
|
||||
test("orders by the rowid when the table has no primary key", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "no_key",
|
||||
columns: ["a", "b"],
|
||||
key: { kind: "rowid" },
|
||||
max_chars: 400,
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
search: null,
|
||||
});
|
||||
const record = db.query(query.sql).get(...query.values) as Record<string, unknown>;
|
||||
|
||||
expect(Number(record[RecordKeyAlias])).toBe(1);
|
||||
});
|
||||
|
||||
test("a table with no key comes back without one", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "no_key",
|
||||
columns: ["a", "b"],
|
||||
key: { kind: "none" },
|
||||
max_chars: 400,
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
search: null,
|
||||
});
|
||||
const record = db.query(query.sql).get(...query.values) as Record<string, unknown>;
|
||||
|
||||
expect(query.sql).not.toContain(`ORDER BY`);
|
||||
expect(record[RecordKeyAlias]).toBeUndefined();
|
||||
});
|
||||
|
||||
test("selects a meta column per column, so a cut value can be known", () => {
|
||||
const select = buildRecordSelect({
|
||||
table: "messages",
|
||||
columns: messages_columns.map((column) => column.name),
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
});
|
||||
|
||||
expect(select.sql).toContain(quoteSqlIdentifier({ name: metaColumnAlias({ name: "text", kind: "length" }) }));
|
||||
expect(select.sql).toContain(quoteSqlIdentifier({ name: metaColumnAlias({ name: "text", kind: "type" }) }));
|
||||
});
|
||||
});
|
||||
|
||||
describe("search", () => {
|
||||
const columns = messages_columns.map((column) => column.name);
|
||||
const every_word = { text: "hello there", mode: "every_word" as const, max_words: 8 };
|
||||
|
||||
test("every word matches anywhere in the record, in any order", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns,
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
search: every_word,
|
||||
});
|
||||
|
||||
expect(rows(query).map((record) => record.key).sort()).toEqual([1, 2]);
|
||||
});
|
||||
|
||||
test("a phrase only matches where the words sit together", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns,
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
search: { text: "hello there", mode: "phrase", max_words: 8 },
|
||||
});
|
||||
|
||||
expect(rows(query).map((record) => record.key).sort()).toEqual([1, 2]);
|
||||
});
|
||||
|
||||
test("a phrase of one word that sits apart doesn't match", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns,
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
search: { text: "help hello", mode: "phrase", max_words: 8 },
|
||||
});
|
||||
|
||||
expect(rows(query)).toEqual([]);
|
||||
});
|
||||
|
||||
test("prefixes find a value, wherever it is in a word", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns,
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
search: { text: "quoted", mode: "every_word", max_words: 8 },
|
||||
});
|
||||
|
||||
expect(rows(query).map((record) => record.key)).toEqual([3]);
|
||||
});
|
||||
|
||||
test("matches an id as text", () => {
|
||||
const query = buildRecordsSql({
|
||||
table: "messages",
|
||||
columns,
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
limit: 10,
|
||||
offset: 0,
|
||||
search: { text: "3", mode: "every_word", max_words: 8 },
|
||||
});
|
||||
|
||||
expect(rows(query).map((record) => record.key)).toEqual([3]);
|
||||
});
|
||||
|
||||
test("counts only what matched", () => {
|
||||
const count = buildCountSql({ table: "messages", columns, search: every_word });
|
||||
const result = db.query(count.sql).get(...count.values) as { total: number };
|
||||
|
||||
expect(result.total).toBe(2);
|
||||
});
|
||||
|
||||
test("counts the whole table with no search", () => {
|
||||
const count = buildCountSql({ table: "messages", columns, search: null });
|
||||
const result = db.query(count.sql).get(...count.values) as { total: number };
|
||||
|
||||
expect(result.total).toBe(4);
|
||||
});
|
||||
|
||||
test("an empty search matches everything rather than failing", () => {
|
||||
const clause = buildSearchClause({ columns, search: " ", mode: "every_word", max_words: 8 });
|
||||
|
||||
expect(clause).toEqual({ sql: "1", values: [] });
|
||||
});
|
||||
|
||||
test("keeps a search to its words", () => {
|
||||
expect(searchWords({ search: " a b c ", max_words: 2 })).toEqual(["a", "b"]);
|
||||
});
|
||||
|
||||
test("a table with no columns matches nothing", () => {
|
||||
const clause = buildSearchClause({ columns: [], search: "x", mode: "every_word", max_words: 8 });
|
||||
|
||||
expect(clause.values).toEqual(["x"]);
|
||||
expect(clause.sql).toContain(`''`);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildRecordSql", () => {
|
||||
test("reads one record by its key", () => {
|
||||
const query = buildRecordSql({
|
||||
table: "messages",
|
||||
columns: messages_columns.map((column) => column.name),
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
value: 2,
|
||||
});
|
||||
const [record] = rows({ sql: query.sql, values: query.values as string[] });
|
||||
|
||||
expect(record?.key).toBe(2);
|
||||
expect(record?.cells["text"]?.text).toBe("Hello there, how can I help?");
|
||||
});
|
||||
|
||||
test("finds nothing for a key that isn't there", () => {
|
||||
const query = buildRecordSql({
|
||||
table: "messages",
|
||||
columns: messages_columns.map((column) => column.name),
|
||||
key: { kind: "column", name: "id" },
|
||||
max_chars: 400,
|
||||
value: 99,
|
||||
});
|
||||
|
||||
expect(db.query(query.sql).all(...(query.values as string[]))).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("parseRecordKey", () => {
|
||||
test("an integer key comes back as a number", () => {
|
||||
expect(parseRecordKey({ value: "42" })).toBe(42);
|
||||
});
|
||||
|
||||
test("a text key stays text", () => {
|
||||
expect(parseRecordKey({ value: "abc" })).toBe("abc");
|
||||
});
|
||||
|
||||
test("a number too big to be safe stays text", () => {
|
||||
expect(parseRecordKey({ value: "99999999999999999999" })).toBe("99999999999999999999");
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildTableCatalogSql", () => {
|
||||
test("reads every table and its columns, without SQLite's own tables", () => {
|
||||
const res = db.query(buildTableCatalogSql()).all() as TableCatalogRow[];
|
||||
const tables = groupTableCatalog({ rows: res });
|
||||
|
||||
expect(tables.map((table) => table.name)).toEqual(["messages", "no_key"]);
|
||||
expect(tables[0]?.columns.map((column) => column.name)).toEqual(["id", "role", "text"]);
|
||||
expect(tables[0]?.has_rowid).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,189 @@
|
||||
import buildSearchClause from "./build-search-clause";
|
||||
import quoteSqlIdentifier from "./quote-sql-identifier";
|
||||
import type { DatabaseSearchMode, DatabaseTableKey } from "@/src/types";
|
||||
|
||||
/**
|
||||
* The column every record's key comes back in.
|
||||
*/
|
||||
export const RecordKeyAlias = "__cobble_key";
|
||||
|
||||
type MetaKind = "type" | "length";
|
||||
|
||||
/**
|
||||
* The column SQLite's own type of a value comes back in, beside the value.
|
||||
*/
|
||||
export function metaColumnAlias({ name, kind }: { name: string; kind: MetaKind }): string {
|
||||
return `__cobble_${kind}__${name}`;
|
||||
}
|
||||
|
||||
type ColumnListParams = {
|
||||
/**
|
||||
* The table's columns, in their declared order.
|
||||
*/
|
||||
columns: string[];
|
||||
/**
|
||||
* The most characters of a value the query takes out of SQLite.
|
||||
*/
|
||||
max_chars: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* What a query selects: every column, cut to `max_chars` so no huge value
|
||||
* (a record's long text, a job's checkpoint) crosses out of SQLite, and
|
||||
* beside each one its stored type and length, the two things that say how
|
||||
* to read it and whether it was cut.
|
||||
*/
|
||||
export function buildRecordColumns({ columns, max_chars }: ColumnListParams): string[] {
|
||||
const selected: string[] = [];
|
||||
|
||||
for (const name of columns) {
|
||||
const column = quoteSqlIdentifier({ name });
|
||||
|
||||
selected.push(`substr(${column}, 1, ${max_chars}) AS ${column}`);
|
||||
selected.push(
|
||||
`typeof(${column}) AS ${quoteSqlIdentifier({ name: metaColumnAlias({ name, kind: "type" }) })}`,
|
||||
);
|
||||
selected.push(
|
||||
`length(${column}) AS ${quoteSqlIdentifier({ name: metaColumnAlias({ name, kind: "length" }) })}`,
|
||||
);
|
||||
}
|
||||
|
||||
return selected;
|
||||
}
|
||||
|
||||
/**
|
||||
* The key column, which is what identifies a record: a primary key, the
|
||||
* rowid, or nothing at all for a virtual table that has neither.
|
||||
*/
|
||||
export function buildKeyColumn({ key }: { key: DatabaseTableKey }): string[] {
|
||||
if (key.kind == "column" && key.name) {
|
||||
return [`${quoteSqlIdentifier({ name: key.name })} AS "${RecordKeyAlias}"`];
|
||||
}
|
||||
|
||||
if (key.kind == "rowid") {
|
||||
/**
|
||||
* Quoted: `rowid` is read as the ref's rowid that way, and as a
|
||||
* column of the table's own named `rowid` otherwise.
|
||||
*/
|
||||
return [`"rowid" AS "${RecordKeyAlias}"`];
|
||||
}
|
||||
|
||||
return [];
|
||||
}
|
||||
|
||||
type SelectParams = ColumnListParams & {
|
||||
table: string;
|
||||
key: DatabaseTableKey;
|
||||
/**
|
||||
* What to look for, when the browser is narrowing the table down.
|
||||
*/
|
||||
search?: DatabaseSearch | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* A search a records query narrows its table by.
|
||||
*/
|
||||
export type DatabaseSearch = {
|
||||
text: string;
|
||||
mode: DatabaseSearchMode;
|
||||
/**
|
||||
* How many words at most are matched, so a long line can't build a
|
||||
* query of a hundred conditions.
|
||||
*/
|
||||
max_words: number;
|
||||
};
|
||||
|
||||
type Query = { sql: string; values: string[] };
|
||||
|
||||
/**
|
||||
* The `SELECT` a page of records and one record both start from: the key,
|
||||
* every column with its meta columns, and, when something is searched, a
|
||||
* `WHERE` over all of the table's text.
|
||||
*/
|
||||
export function buildRecordSelect({ table, columns, key, max_chars, search }: SelectParams): Query {
|
||||
const selected = [...buildKeyColumn({ key }), ...buildRecordColumns({ columns, max_chars })];
|
||||
const from = `SELECT ${selected.join(", ")} FROM ${quoteSqlIdentifier({ name: table })}`;
|
||||
|
||||
if (!search) return { sql: from, values: [] };
|
||||
|
||||
const where = buildSearchClause({
|
||||
columns,
|
||||
search: search.text,
|
||||
mode: search.mode,
|
||||
max_words: search.max_words,
|
||||
});
|
||||
|
||||
return { sql: `${from} WHERE ${where.sql}`, values: where.values };
|
||||
}
|
||||
|
||||
type PageParams = SelectParams & {
|
||||
limit: number;
|
||||
offset: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* One page of a table's records, newest key first, so a freshly written
|
||||
* record shows at the top. A table with no key to order by comes back in
|
||||
* whatever order SQLite reads it.
|
||||
*/
|
||||
export default function buildRecordsSql({ limit, offset, ...params }: PageParams): Query {
|
||||
const select = buildRecordSelect(params);
|
||||
const order = params.key.kind == "none" ? "" : ` ORDER BY "${RecordKeyAlias}" DESC`;
|
||||
|
||||
return {
|
||||
sql: `${select.sql}${order} LIMIT ${limit} OFFSET ${offset}`,
|
||||
values: select.values,
|
||||
};
|
||||
}
|
||||
|
||||
type RecordParams = SelectParams & {
|
||||
/**
|
||||
* The record's key, as it came in the query string.
|
||||
*/
|
||||
value: string | number;
|
||||
};
|
||||
|
||||
/**
|
||||
* One record, by its key. Only asked for a table that has one, so a row
|
||||
* always comes back.
|
||||
*/
|
||||
export function buildRecordSql({ value, ...params }: RecordParams): { sql: string; values: (string | number)[] } {
|
||||
const select = buildRecordSelect(params);
|
||||
|
||||
return {
|
||||
sql: `${select.sql} WHERE "${RecordKeyAlias}" = ? LIMIT 1`,
|
||||
values: [...select.values, value],
|
||||
};
|
||||
}
|
||||
|
||||
type CountParams = Omit<SelectParams, "key" | "max_chars">;
|
||||
|
||||
/**
|
||||
* How many records the table holds, or how many the search matched, for
|
||||
* the pager under it.
|
||||
*/
|
||||
export function buildCountSql({ table, columns, search }: CountParams): Query {
|
||||
const from = `SELECT COUNT(*) AS total FROM ${quoteSqlIdentifier({ name: table })}`;
|
||||
|
||||
if (!search) return { sql: from, values: [] };
|
||||
|
||||
const where = buildSearchClause({
|
||||
columns,
|
||||
search: search.text,
|
||||
mode: search.mode,
|
||||
max_words: search.max_words,
|
||||
});
|
||||
|
||||
return { sql: `${from} WHERE ${where.sql}`, values: where.values };
|
||||
}
|
||||
|
||||
/**
|
||||
* The record SQL's parameter (a primary key or a rowid), read from a
|
||||
* query string: digits are a number, so an integer key compares as one,
|
||||
* and anything else stays a string.
|
||||
*/
|
||||
export function parseRecordKey({ value }: { value: string }): string | number {
|
||||
const text = value.trim();
|
||||
|
||||
return /^-?\d+$/.test(text) && Number.isSafeInteger(Number(text)) ? Number(text) : text;
|
||||
}
|
||||
@@ -0,0 +1,85 @@
|
||||
import quoteSqlIdentifier from "./quote-sql-identifier";
|
||||
import type { DatabaseSearchMode } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The table's columns, in their declared order.
|
||||
*/
|
||||
columns: string[];
|
||||
search: string;
|
||||
mode: DatabaseSearchMode;
|
||||
/**
|
||||
* How many words at most are matched, so a long line can't build a
|
||||
* query of a hundred conditions.
|
||||
*/
|
||||
max_words: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* Splits a search into its words, in the case they were typed (SQLite's
|
||||
* `lower` folds ASCII case).
|
||||
*/
|
||||
export function searchWords({ search, max_words }: { search: string; max_words: number }): string[] {
|
||||
return search
|
||||
.trim()
|
||||
.split(/\s+/)
|
||||
.filter(Boolean)
|
||||
.slice(0, max_words);
|
||||
}
|
||||
|
||||
type WordParams = {
|
||||
/**
|
||||
* The SQL that reads every column of a row as one text blob.
|
||||
*/
|
||||
text: string;
|
||||
search: string;
|
||||
mode: DatabaseSearchMode;
|
||||
max_words: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* A search as a `WHERE` condition, with the parameters its placeholders
|
||||
* take, in order.
|
||||
*/
|
||||
export function searchCondition({ text, search, mode, max_words }: WordParams): { sql: string; values: string[] } {
|
||||
if (mode == "phrase") {
|
||||
return {
|
||||
sql: `instr(lower(${text}), lower(?)) > 0`,
|
||||
values: [search.trim()],
|
||||
};
|
||||
}
|
||||
|
||||
const words = searchWords({ search, max_words });
|
||||
|
||||
if (!words.length) {
|
||||
return { sql: `1`, values: [] };
|
||||
}
|
||||
|
||||
return {
|
||||
sql: words.map(() => `instr(lower(${text}), lower(?)) > 0`).join(` AND `),
|
||||
values: words,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* A search over every column of a table, as a condition the records query
|
||||
* puts in its `WHERE` (and the count query puts in its own), with the
|
||||
* parameters its placeholders take, in order.
|
||||
*
|
||||
* The columns are joined into one text blob per row, so a value is found
|
||||
* wherever it lives: an id, a name, a message's words. Reading every row
|
||||
* is the only honest way to find a value in a table with no index for it;
|
||||
* the page sizes keep what comes back small, not the cost of the search.
|
||||
*/
|
||||
export default function buildSearchClause({ columns, search, mode, max_words }: Params): {
|
||||
sql: string;
|
||||
values: string[];
|
||||
} {
|
||||
const text = columns.length
|
||||
? columns
|
||||
.map((name) => `COALESCE(CAST(${quoteSqlIdentifier({ name })} AS TEXT), '')`)
|
||||
.join(` || ' ' || `)
|
||||
: `''`;
|
||||
|
||||
return searchCondition({ text, search, mode, max_words });
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
/**
|
||||
* The tables the database browser lists, each with its columns in their
|
||||
* declared order, in one statement: `pragma_table_list` finds the tables
|
||||
* and `pragma_table_info` reads each one's columns, the pragma taking the
|
||||
* table name as an argument the way a table-valued function does.
|
||||
*
|
||||
* Left out: SQLite's own `sqlite_*` tables, and the shadow tables a
|
||||
* virtual table keeps its internals in (the `bot_file_vectors_*` tables
|
||||
* behind the sqlite-vec index), which are storage, not rows anyone
|
||||
* browses.
|
||||
*/
|
||||
export default function buildTableCatalogSql(): string {
|
||||
return `SELECT
|
||||
l.name AS table_name,
|
||||
l.type AS table_type,
|
||||
l.wr AS without_rowid,
|
||||
p.cid AS column_index,
|
||||
p.name AS column_name,
|
||||
p.type AS column_type,
|
||||
p.pk AS column_pk
|
||||
FROM pragma_table_list AS l
|
||||
JOIN pragma_table_info(l.name) AS p
|
||||
WHERE l.schema = 'main'
|
||||
AND l.type IN ('table', 'virtual')
|
||||
AND l.name NOT LIKE 'sqlite\\_%' ESCAPE '\\'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1
|
||||
FROM pragma_table_list AS v
|
||||
WHERE v.type = 'virtual'
|
||||
AND l.name LIKE v.name || '\\_%' ESCAPE '\\'
|
||||
)
|
||||
ORDER BY l.name ASC, p.cid ASC`;
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
/**
|
||||
* How many bytes each table takes on disk. `dbstat` reports pages per
|
||||
* b-tree; an index's pages are named after the index (`autoindex` or the
|
||||
* schema's own), so the join puts them back under the table they belong
|
||||
* to. A table with no pages yet has no row here.
|
||||
*/
|
||||
export default function buildTableSizesSql(): string {
|
||||
return `SELECT
|
||||
COALESCE(i.tbl_name, d.name) AS table_name,
|
||||
SUM(d.pgsize) AS bytes
|
||||
FROM dbstat AS d
|
||||
LEFT JOIN sqlite_master AS i ON i.type = 'index' AND i.name = d.name
|
||||
GROUP BY COALESCE(i.tbl_name, d.name)`;
|
||||
}
|
||||
|
||||
export type TableSizeRow = {
|
||||
table_name: string;
|
||||
bytes: number;
|
||||
};
|
||||
@@ -0,0 +1,30 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import clampPage from "./clamp-page";
|
||||
|
||||
describe("clampPage", () => {
|
||||
test("keeps a page that is inside the table", () => {
|
||||
expect(clampPage({ page: 2, total: 120, page_size: 50 })).toBe(2);
|
||||
});
|
||||
|
||||
test("a page past the end lands on the last one", () => {
|
||||
expect(clampPage({ page: 99, total: 120, page_size: 50 })).toBe(3);
|
||||
});
|
||||
|
||||
test("an empty table still has a first page", () => {
|
||||
expect(clampPage({ page: 4, total: 0, page_size: 50 })).toBe(1);
|
||||
});
|
||||
|
||||
test("a page below one is the first", () => {
|
||||
expect(clampPage({ page: 0, total: 120, page_size: 50 })).toBe(1);
|
||||
expect(clampPage({ page: -3, total: 120, page_size: 50 })).toBe(1);
|
||||
});
|
||||
|
||||
test("a table that fits exactly ends on the page asked for", () => {
|
||||
expect(clampPage({ page: 2, total: 100, page_size: 50 })).toBe(2);
|
||||
expect(clampPage({ page: 3, total: 100, page_size: 50 })).toBe(2);
|
||||
});
|
||||
|
||||
test("a page size of nothing doesn't divide by zero", () => {
|
||||
expect(clampPage({ page: 1, total: 10, page_size: 0 })).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,23 @@
|
||||
type Params = {
|
||||
/**
|
||||
* The page asked for, from 1.
|
||||
*/
|
||||
page: number;
|
||||
/**
|
||||
* How many records the table holds (or how many matched a search).
|
||||
*/
|
||||
total: number;
|
||||
page_size: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* The page that actually gets read: a page past the end of the table lands
|
||||
* on its last one (so a stale link or a deleted record never shows an empty
|
||||
* page), and a table with nothing in it still has a first page.
|
||||
*/
|
||||
export default function clampPage({ page, total, page_size }: Params): number {
|
||||
const size = Math.max(1, Math.trunc(page_size) || 1);
|
||||
const last = Math.max(1, Math.ceil(total / size));
|
||||
|
||||
return Math.min(Math.max(1, Math.trunc(page) || 1), last);
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import buildDatabaseTables from "./build-database-tables";
|
||||
import buildTableCatalogSql from "./build-table-catalog-sql";
|
||||
import grabTableSizes from "./grab-table-sizes";
|
||||
import groupTableCatalog from "./group-table-catalog";
|
||||
import type { TableCatalogRow } from "./table-catalog-types";
|
||||
import type { DatabaseTableSummary } from "@/src/types";
|
||||
|
||||
/**
|
||||
* Every table the database browser lists: the catalog read in one
|
||||
* statement (`buildTableCatalogSql`), the sizes on disk, then a count each.
|
||||
* Empty when the catalog can't be read, which leaves the page saying so
|
||||
* rather than failing.
|
||||
*/
|
||||
export default async function grabDatabaseTables(): Promise<DatabaseTableSummary[]> {
|
||||
const res = await BunSQLite.sql<TableCatalogRow>({ sql: buildTableCatalogSql() });
|
||||
|
||||
if (!res.success || !res.payload) return [];
|
||||
|
||||
const tables = groupTableCatalog({ rows: res.payload });
|
||||
|
||||
if (!tables.length) return [];
|
||||
|
||||
return await buildDatabaseTables({ tables, sizes: await grabTableSizes() });
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import grabDatabaseTables from "./grab-database-tables";
|
||||
import type { DatabaseTableSummary } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The table asked for, as it came in the query string. The first table
|
||||
* is used when nothing is asked for.
|
||||
*/
|
||||
table?: string | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* One table, or the first one when nothing is asked for. Anything else is
|
||||
* refused with a message safe to show: the browser only ever reads tables
|
||||
* that are in the database, never a name it built itself.
|
||||
*/
|
||||
export default async function grabTableCatalogEntry({ table }: Params): Promise<DatabaseTableSummary> {
|
||||
const tables = await grabDatabaseTables();
|
||||
|
||||
if (!tables.length) {
|
||||
throw new AppError(`Couldn't read the database's tables.`);
|
||||
}
|
||||
|
||||
const first = tables[0];
|
||||
|
||||
if (!table && first) return first;
|
||||
|
||||
const entry = tables.find((candidate) => candidate.name == table);
|
||||
|
||||
if (!entry) {
|
||||
throw new AppError(`There's no table called "${table}".`);
|
||||
}
|
||||
|
||||
return entry;
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import buildRecordRow from "./build-record-row";
|
||||
import { buildRecordSql, parseRecordKey } from "./build-records-sql";
|
||||
import grabTableCatalogEntry from "./grab-table-catalog-entry";
|
||||
import type { DatabaseRecordPayload } from "@/src/types";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
|
||||
type Params = {
|
||||
table?: string | null;
|
||||
/**
|
||||
* The record's key, as it came in the query string.
|
||||
*/
|
||||
key: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* One record of one table, in full for its modal: every value read whole
|
||||
* (up to `AppData.DatabaseBrowserValueMaxChars`, since a megabyte of blob
|
||||
* or checkpoint is not something to render), so a cell cut short in the
|
||||
* table can still be read to its end.
|
||||
*/
|
||||
export default async function grabTableRecord({ table, key }: Params): Promise<DatabaseRecordPayload> {
|
||||
const entry = await grabTableCatalogEntry({ table });
|
||||
|
||||
if (entry.key.kind == "none") {
|
||||
throw new AppError(`"${entry.name}" has no key to open a single record by.`);
|
||||
}
|
||||
|
||||
const columns = entry.columns.map((column) => column.name);
|
||||
const query = buildRecordSql({
|
||||
table: entry.name,
|
||||
columns,
|
||||
key: entry.key,
|
||||
max_chars: AppData.DatabaseBrowserValueMaxChars,
|
||||
value: parseRecordKey({ value: key }),
|
||||
});
|
||||
const res = await BunSQLite.sql<Record<string, unknown>>(query);
|
||||
|
||||
if (!res.success) {
|
||||
throw new AppError(`Couldn't read that record of "${entry.name}".`);
|
||||
}
|
||||
|
||||
const row = res.payload?.[0];
|
||||
|
||||
if (!row) {
|
||||
throw new AppError(`That record isn't in "${entry.name}" any more.`);
|
||||
}
|
||||
|
||||
return {
|
||||
table: entry,
|
||||
record: buildRecordRow({
|
||||
row,
|
||||
columns: entry.columns,
|
||||
max_chars: AppData.DatabaseBrowserValueMaxChars,
|
||||
}),
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import AppError from "@/src/utils/app-error";
|
||||
import buildRecordRow from "./build-record-row";
|
||||
import buildRecordsSql, { buildCountSql } from "./build-records-sql";
|
||||
import clampPage from "./clamp-page";
|
||||
import grabTableCatalogEntry from "./grab-table-catalog-entry";
|
||||
import type { DatabaseRecord, DatabaseRecordsPayload, DatabaseSearchMode } from "@/src/types";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
|
||||
type Params = {
|
||||
table?: string | null;
|
||||
/**
|
||||
* Which page of records, from 1.
|
||||
*/
|
||||
page?: number;
|
||||
/**
|
||||
* What to look for in the table's records, across every column's text.
|
||||
*/
|
||||
search?: string;
|
||||
mode?: DatabaseSearchMode;
|
||||
};
|
||||
|
||||
type TotalRow = { total: number };
|
||||
|
||||
/**
|
||||
* One page of one table's records, read-only: the table asked for with all
|
||||
* of its columns, its records newest key first, and how many matched so the
|
||||
* pager knows where the end is.
|
||||
*
|
||||
* Every value crosses out of SQLite already cut to
|
||||
* `AppData.DatabaseBrowserCellMaxChars`, so a table of long texts can't
|
||||
* turn one page into a megabyte of JSON.
|
||||
*/
|
||||
export default async function grabTableRecords({
|
||||
table,
|
||||
page = 1,
|
||||
search = "",
|
||||
mode = "every_word",
|
||||
}: Params): Promise<DatabaseRecordsPayload> {
|
||||
const entry = await grabTableCatalogEntry({ table });
|
||||
const page_size = AppData.DatabaseBrowserPageSize;
|
||||
const current_page = Math.max(1, Math.trunc(page || 1));
|
||||
const columns = entry.columns.map((column) => column.name);
|
||||
const trimmed = search.trim();
|
||||
const search_params = trimmed
|
||||
? { text: trimmed, mode, max_words: AppData.DatabaseBrowserMaxSearchWords }
|
||||
: null;
|
||||
const count_res = await BunSQLite.sql<TotalRow>(
|
||||
buildCountSql({
|
||||
table: entry.name,
|
||||
columns,
|
||||
search: search_params,
|
||||
}),
|
||||
);
|
||||
const total = Number(count_res.payload?.[0]?.total) || 0;
|
||||
const final_page = clampPage({ page: current_page, total, page_size });
|
||||
const page_query = buildRecordsSql({
|
||||
table: entry.name,
|
||||
columns,
|
||||
key: entry.key,
|
||||
max_chars: AppData.DatabaseBrowserCellMaxChars,
|
||||
limit: page_size,
|
||||
offset: (final_page - 1) * page_size,
|
||||
search: search_params,
|
||||
});
|
||||
const res = await BunSQLite.sql<Record<string, unknown>>(page_query);
|
||||
|
||||
if (!res.success || !res.payload) {
|
||||
throw new AppError(`Couldn't read the records of "${entry.name}".`);
|
||||
}
|
||||
|
||||
const records: DatabaseRecord[] = res.payload.map((row) =>
|
||||
buildRecordRow({
|
||||
row,
|
||||
columns: entry.columns,
|
||||
max_chars: AppData.DatabaseBrowserCellMaxChars,
|
||||
}),
|
||||
);
|
||||
|
||||
return {
|
||||
table: entry,
|
||||
records,
|
||||
total,
|
||||
page: final_page,
|
||||
page_size,
|
||||
search: trimmed,
|
||||
mode,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
import BunSQLite from "@moduletrace/bun-sqlite";
|
||||
import buildTableSizesSql, { type TableSizeRow } from "./build-table-sizes-sql";
|
||||
|
||||
/**
|
||||
* How many bytes each table takes on disk, by table name. Empty when
|
||||
* SQLite was built without the `dbstat` vtab, which only costs the browser
|
||||
* its size column: the records still read.
|
||||
*/
|
||||
export default async function grabTableSizes(): Promise<Map<string, number>> {
|
||||
const sizes = new Map<string, number>();
|
||||
const res = await BunSQLite.sql<TableSizeRow>({ sql: buildTableSizesSql() });
|
||||
|
||||
if (!res.success || !res.payload) return sizes;
|
||||
|
||||
for (const row of res.payload) {
|
||||
sizes.set(row.table_name, Number(row.bytes) || 0);
|
||||
}
|
||||
|
||||
return sizes;
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
import type { TableCatalogEntry, TableCatalogRow } from "./table-catalog-types";
|
||||
|
||||
type Params = {
|
||||
rows: TableCatalogRow[];
|
||||
};
|
||||
|
||||
/**
|
||||
* Groups the catalog's rows per table, keeping the statement's order, so
|
||||
* the list of tables and every table's columns come back in one read.
|
||||
*/
|
||||
export default function groupTableCatalog({ rows }: Params): TableCatalogEntry[] {
|
||||
const entries: TableCatalogEntry[] = [];
|
||||
|
||||
for (const row of rows) {
|
||||
let entry = entries.at(-1);
|
||||
|
||||
if (!entry || entry.name != row.table_name) {
|
||||
entry = {
|
||||
name: row.table_name,
|
||||
type: row.table_type,
|
||||
has_rowid: row.table_type == "table" && !row.without_rowid,
|
||||
columns: [],
|
||||
};
|
||||
entries.push(entry);
|
||||
}
|
||||
|
||||
entry.columns.push({
|
||||
name: row.column_name,
|
||||
type: row.column_type || "",
|
||||
pk: Boolean(row.column_pk),
|
||||
});
|
||||
}
|
||||
|
||||
return entries;
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
/**
|
||||
* Quotes a table or column name for SQL, wrapping it in double quotes and
|
||||
* doubling any it holds. The database browser builds its statements from
|
||||
* the database's own names, and every one of them still goes through here
|
||||
* so a name with a quote or a space in it can't break out of the query.
|
||||
*/
|
||||
export default function quoteSqlIdentifier({ name }: { name: string }): string {
|
||||
return `"${name.replace(/"/g, `""`)}"`;
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
import type { DatabaseTableColumn, DatabaseTableKey } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The table's columns in their declared order.
|
||||
*/
|
||||
columns: DatabaseTableColumn[];
|
||||
/**
|
||||
* Whether the table is one SQLite gives a rowid to: every ordinary
|
||||
* table has one, a virtual table and a `WITHOUT ROWID` table do not.
|
||||
*/
|
||||
has_rowid: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* What a table's records are identified by: its primary key when that is
|
||||
* one column (a sqlite-vec index keeps its own `id`), else the rowid every
|
||||
* ordinary SQLite table has, else nothing, which leaves the browser able
|
||||
* to read a page but not to open one record.
|
||||
*
|
||||
* A composite primary key falls back to the rowid, which is the record's
|
||||
* own and shows in the key column either way.
|
||||
*/
|
||||
export default function resolveTableKey({ columns, has_rowid }: Params): DatabaseTableKey {
|
||||
const pk_columns = columns.filter((column) => column.pk);
|
||||
|
||||
const [pk_column] = pk_columns;
|
||||
|
||||
if (pk_column && pk_columns.length == 1) {
|
||||
return { kind: "column", name: pk_column.name };
|
||||
}
|
||||
|
||||
if (has_rowid) {
|
||||
return { kind: "rowid" };
|
||||
}
|
||||
|
||||
return { kind: "none" };
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
/**
|
||||
* One row of the table catalog: one column of one table.
|
||||
*/
|
||||
export type TableCatalogRow = {
|
||||
table_name: string;
|
||||
table_type: "table" | "virtual";
|
||||
/**
|
||||
* 1 when the table was created `WITHOUT ROWID`.
|
||||
*/
|
||||
without_rowid: number;
|
||||
column_index: number;
|
||||
column_name: string;
|
||||
column_type: string | null;
|
||||
/**
|
||||
* Where the column sits in the table's primary key; 0 when it isn't
|
||||
* one.
|
||||
*/
|
||||
column_pk: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* One table of the catalog.
|
||||
*/
|
||||
export type TableCatalogEntry = {
|
||||
name: string;
|
||||
type: "table" | "virtual";
|
||||
/**
|
||||
* Whether SQLite gives the table a rowid: an ordinary table does, a
|
||||
* `WITHOUT ROWID` table and a virtual table do not.
|
||||
*/
|
||||
has_rowid: boolean;
|
||||
columns: { name: string; type: string; pk: boolean }[];
|
||||
};
|
||||
@@ -0,0 +1,48 @@
|
||||
import formatBytes from "@/src/utils/format-bytes";
|
||||
import type { DatabaseCell } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The value read out of SQLite, already cut to the length asked for.
|
||||
*/
|
||||
value: unknown;
|
||||
/**
|
||||
* SQLite's own type for the stored value: `integer`, `real`, `text`,
|
||||
* `blob` or `null`.
|
||||
*/
|
||||
type: string;
|
||||
/**
|
||||
* How long the stored value is (characters for text, bytes for a
|
||||
* blob), which says whether `value` is the whole of it.
|
||||
*/
|
||||
length: number;
|
||||
max_chars: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* One stored value as a cell shows it. Text is cut at `max_chars` and
|
||||
* marked truncated, a blob is only its size (its bytes would fill the page
|
||||
* with noise), a number is printed as it is, and NULL is said, never left
|
||||
* blank.
|
||||
*/
|
||||
export default function toDatabaseCell({ value, type, length, max_chars }: Params): DatabaseCell {
|
||||
if (type == "null" || value === null || value === undefined) {
|
||||
return { text: "", kind: "null", truncated: false };
|
||||
}
|
||||
|
||||
if (type == "blob") {
|
||||
return {
|
||||
text: `<${formatBytes({ bytes: length })} of binary data>`,
|
||||
kind: "blob",
|
||||
truncated: false,
|
||||
};
|
||||
}
|
||||
|
||||
const text = String(value);
|
||||
|
||||
if (type == "integer" || type == "real") {
|
||||
return { text, kind: "number", truncated: false };
|
||||
}
|
||||
|
||||
return { text, kind: "text", truncated: length > max_chars };
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import type { DatabaseSearchMode } from "@/src/types";
|
||||
|
||||
export type DatabaseSearchModeOption = {
|
||||
value: DatabaseSearchMode;
|
||||
title: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* How a search matches, in the order the picker offers them: every word is
|
||||
* the useful default (it finds a message by two words it holds wherever
|
||||
* they sit), the phrase is for a run of words that has to stay together.
|
||||
*/
|
||||
export const DatabaseSearchModes: DatabaseSearchModeOption[] = [
|
||||
{ value: "every_word", title: "Every word" },
|
||||
{ value: "phrase", title: "The phrase" },
|
||||
];
|
||||
|
||||
/**
|
||||
* A mode read from the address, or the default when it isn't one of them.
|
||||
*/
|
||||
export function parseDatabaseSearchMode({ value }: { value?: string | null }): DatabaseSearchMode {
|
||||
const mode = DatabaseSearchModes.find((option) => option.value == value);
|
||||
|
||||
return mode?.value || "every_word";
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
import type { DatabaseSearchMode } from "@/src/types";
|
||||
import { SettingsDatabasePath } from "@/src/dict/settings-pages-dict";
|
||||
|
||||
type Params = {
|
||||
table?: string;
|
||||
search?: string;
|
||||
mode?: DatabaseSearchMode;
|
||||
page?: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* The database browser's address for what it is showing, so a table, a
|
||||
* search and a page can be linked to and the browser's own back button
|
||||
* walks back through them. What is at its default is left out.
|
||||
*/
|
||||
export default function databaseHref({ table, search, mode, page }: Params): string {
|
||||
const params = new URLSearchParams();
|
||||
|
||||
if (table) params.set("table", table);
|
||||
if (search) params.set("search", search);
|
||||
if (mode && mode != "every_word") params.set("mode", mode);
|
||||
if (page && page > 1) params.set("page", String(page));
|
||||
|
||||
const query = params.toString();
|
||||
|
||||
return query ? `${SettingsDatabasePath}?${query}` : SettingsDatabasePath;
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
|
||||
import grabDatabaseTables from "@/src/functions/backend/database/grab-database-tables";
|
||||
import grabTableRecords from "@/src/functions/backend/database/grab-table-records";
|
||||
import { SettingsDefaultPath } from "@/src/dict/settings-pages-dict";
|
||||
import type { PagePropsType } from "@/src/types";
|
||||
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
|
||||
import grabSettingsServerProps from "../../(functions)/grab-settings-server-props";
|
||||
|
||||
type Params = Parameters<BunextPageServerFn<PagePropsType>>[0];
|
||||
|
||||
/**
|
||||
* The database browser's page data: every table with its size and record
|
||||
* count, and the first page of the table asked for. Super admins only;
|
||||
* everyone else lands on the general settings.
|
||||
*/
|
||||
export default async function grabDatabaseServerProps(params: Params) {
|
||||
const settings = await grabSettingsServerProps(params);
|
||||
|
||||
if (!settings.props) return settings;
|
||||
|
||||
if (!(await grabSuperAdminUser({ req: params.req }))) {
|
||||
return {
|
||||
redirect: {
|
||||
destination: SettingsDefaultPath,
|
||||
permanent: false,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const tables = await grabDatabaseTables().catch(() => []);
|
||||
|
||||
if (!tables.length) {
|
||||
return {
|
||||
props: {
|
||||
...settings.props,
|
||||
database_tables: [],
|
||||
database_records: null,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
const asked = params.query?.table;
|
||||
const first = tables[0];
|
||||
const table = typeof asked == "string" && asked ? asked : first?.name;
|
||||
const records = await grabTableRecords({ table }).catch(() => null);
|
||||
|
||||
return {
|
||||
props: {
|
||||
...settings.props,
|
||||
database_tables: tables,
|
||||
database_records: records,
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import formatBytes from "@/src/utils/format-bytes";
|
||||
import type { DatabaseRecordsPayload, DatabaseTableSummary } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
table: DatabaseTableSummary;
|
||||
/**
|
||||
* The page of records under the line, for the matched count of a
|
||||
* search.
|
||||
*/
|
||||
records: DatabaseRecordsPayload | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* What the open table is, above its records: how many it holds, how many
|
||||
* a search matched, what it takes on disk and how wide it is.
|
||||
*/
|
||||
export default function tableSummaryText({ table, records }: Params): string {
|
||||
const rows = table.rows === null ? "-" : table.rows.toLocaleString();
|
||||
const size = table.size_bytes === null ? "" : `, ${formatBytes({ bytes: table.size_bytes })} on disk`;
|
||||
const matched = records?.search ? `, ${records.total.toLocaleString()} matching` : "";
|
||||
const columns = `${table.columns.length} column${table.columns.length == 1 ? "" : "s"}`;
|
||||
|
||||
return `${rows} record${table.rows == 1 ? "" : "s"}${matched}${size} - ${columns}`;
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
import fetchApi from "@moduletrace/twui/utils/fetch/fetchApi";
|
||||
import type { APIResponseObject } from "@moduletrace/bunext/types";
|
||||
import { useEffect, useState } from "react";
|
||||
import type { DatabaseRecordPayload } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
/**
|
||||
* The table and key of the record the modal is showing; nothing while
|
||||
* no record is open.
|
||||
*/
|
||||
table?: string;
|
||||
key?: string | number;
|
||||
/**
|
||||
* Whether the modal is open at all, so nothing loads for a closed one.
|
||||
*/
|
||||
open: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* One record, in full, for the modal. Loaded when the modal opens (the
|
||||
* page's cells are cut short), and loaded again whenever another record is
|
||||
* opened, so a click is answered with the whole value.
|
||||
*/
|
||||
export default function useDatabaseRecord({ table, key, open }: Params) {
|
||||
const [record, set_record] = useState<DatabaseRecordPayload | null>(null);
|
||||
const [loading, set_loading] = useState(false);
|
||||
const [error, set_error] = useState<string>();
|
||||
|
||||
useEffect(() => {
|
||||
if (!open || !table || key === undefined || key === null) return;
|
||||
|
||||
const controller = new AbortController();
|
||||
|
||||
const load = async () => {
|
||||
set_loading(true);
|
||||
set_error(undefined);
|
||||
set_record(null);
|
||||
|
||||
const query = new URLSearchParams({
|
||||
table,
|
||||
key: String(key),
|
||||
});
|
||||
|
||||
const res = await fetchApi<{}, APIResponseObject<DatabaseRecordPayload>>(
|
||||
`/api/admin/database/record?${query.toString()}`,
|
||||
{ method: "GET" },
|
||||
).catch(() => null);
|
||||
|
||||
if (controller.signal.aborted) return;
|
||||
|
||||
set_loading(false);
|
||||
|
||||
if (!res?.success || !res.singleRes) {
|
||||
set_error(res?.msg || `Couldn't read that record.`);
|
||||
return;
|
||||
}
|
||||
|
||||
set_record(res.singleRes);
|
||||
};
|
||||
|
||||
load().catch(() => {});
|
||||
|
||||
return () => controller.abort();
|
||||
}, [table, key, open]);
|
||||
|
||||
return { record, loading, error };
|
||||
}
|
||||
@@ -0,0 +1,81 @@
|
||||
import fetchApi from "@moduletrace/twui/utils/fetch/fetchApi";
|
||||
import type { APIResponseObject } from "@moduletrace/bunext/types";
|
||||
import { useEffect, useState } from "react";
|
||||
import type { DatabaseRecordsPayload, DatabaseSearchMode } from "@/src/types";
|
||||
|
||||
type Params = {
|
||||
table: string;
|
||||
search: string;
|
||||
mode: DatabaseSearchMode;
|
||||
page: number;
|
||||
/**
|
||||
* The records the page already carries, when this is exactly what it
|
||||
* is asking for.
|
||||
*/
|
||||
page_records?: DatabaseRecordsPayload | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* One page of one table's records. The page's own copy is used when it is
|
||||
* what is asked for (no search, first page); anything else loads from the
|
||||
* API, so a page never carries a search it may not need.
|
||||
*/
|
||||
export default function useDatabaseRecords({ table, search, mode, page, page_records }: Params) {
|
||||
const from_page = Boolean(
|
||||
page_records &&
|
||||
page_records.table.name == table &&
|
||||
page_records.page == page &&
|
||||
page_records.search == search &&
|
||||
page_records.mode == mode,
|
||||
);
|
||||
const [records, set_records] = useState<DatabaseRecordsPayload | null>(from_page ? page_records || null : null);
|
||||
const [loading, set_loading] = useState(!from_page);
|
||||
const [error, set_error] = useState<string>();
|
||||
|
||||
useEffect(() => {
|
||||
if (from_page) {
|
||||
set_records(page_records || null);
|
||||
set_loading(false);
|
||||
set_error(undefined);
|
||||
return;
|
||||
}
|
||||
|
||||
const controller = new AbortController();
|
||||
|
||||
const load = async () => {
|
||||
set_loading(true);
|
||||
set_error(undefined);
|
||||
|
||||
const query = new URLSearchParams({
|
||||
table,
|
||||
page: String(page),
|
||||
mode,
|
||||
});
|
||||
|
||||
if (search) query.set("search", search);
|
||||
|
||||
const res = await fetchApi<{}, APIResponseObject<DatabaseRecordsPayload>>(
|
||||
`/api/admin/database/records?${query.toString()}`,
|
||||
{ method: "GET" },
|
||||
).catch(() => null);
|
||||
|
||||
if (controller.signal.aborted) return;
|
||||
|
||||
set_loading(false);
|
||||
|
||||
if (!res?.success || !res.singleRes) {
|
||||
set_records(null);
|
||||
set_error(res?.msg || `Couldn't read those records.`);
|
||||
return;
|
||||
}
|
||||
|
||||
set_records(res.singleRes);
|
||||
};
|
||||
|
||||
load().catch(() => {});
|
||||
|
||||
return () => controller.abort();
|
||||
}, [table, search, mode, page, from_page, page_records]);
|
||||
|
||||
return { records, loading, error };
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import type { DatabaseCell as DatabaseCellValue } from "@/src/types";
|
||||
|
||||
type Props = {
|
||||
cell?: DatabaseCellValue;
|
||||
/**
|
||||
* Whether the cell's record can be opened to read it whole.
|
||||
*/
|
||||
openable: boolean;
|
||||
onOpen: () => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* One value in the records table, cut to three lines so a long column
|
||||
* can't push the rest of the row out of sight. Clicking it opens the
|
||||
* record in full.
|
||||
*/
|
||||
export default function DatabaseCell({ cell, openable, onOpen }: Props) {
|
||||
if (!cell || cell.kind == "null") {
|
||||
return (
|
||||
<td className="px-3 py-2.5 align-top">
|
||||
<Span size="smaller" variant="faded" className="font-mono">
|
||||
NULL
|
||||
</Span>
|
||||
</td>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<td className="px-3 py-2.5 align-top max-w-[280px]">
|
||||
<Span
|
||||
size="small"
|
||||
className={twMerge(
|
||||
"block line-clamp-3 whitespace-pre-wrap break-words font-mono",
|
||||
cell.kind == "number" ? "text-ink-secondary" : "text-ink",
|
||||
cell.kind == "blob" && "text-ink-secondary",
|
||||
openable && "cursor-pointer",
|
||||
)}
|
||||
onClick={openable ? onOpen : undefined}
|
||||
title={
|
||||
openable
|
||||
? cell.truncated
|
||||
? "Cut short here: open the record to read the rest"
|
||||
: "Open the record to read it whole"
|
||||
: undefined
|
||||
}
|
||||
>
|
||||
{cell.text}
|
||||
</Span>
|
||||
</td>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
|
||||
type Props = {
|
||||
title: string;
|
||||
/**
|
||||
* What the column is, its declared type, shown on hover so the head
|
||||
* stays one line of names.
|
||||
*/
|
||||
hint?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* One column name over the records table. Column names are read as code:
|
||||
* the database wrote them, nobody typed them.
|
||||
*/
|
||||
export default function DatabaseHeadCell({ title, hint }: Props) {
|
||||
return (
|
||||
<th className="px-3 py-2.5 text-left font-medium max-w-[280px]">
|
||||
<Span size="smaller" variant="faded" className="block truncate font-mono" title={hint || title}>
|
||||
{title}
|
||||
</Span>
|
||||
</th>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
import Loading from "@moduletrace/twui/elements/Loading";
|
||||
import Paper from "@moduletrace/twui/elements/Paper";
|
||||
import Center from "@moduletrace/twui/layout/Center";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import type { DatabaseRecordPayload, DatabaseTableSummary } from "@/src/types";
|
||||
import DatabaseRecordField from "./database-record-field";
|
||||
|
||||
type Props = {
|
||||
table: DatabaseTableSummary;
|
||||
record: DatabaseRecordPayload | null;
|
||||
loading: boolean;
|
||||
error?: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* What the record modal says before a record lands, and the record itself
|
||||
* once it does: every column's whole value, in the order the table
|
||||
* declares them.
|
||||
*/
|
||||
export default function DatabaseRecordBody({ table, record, loading, error }: Props) {
|
||||
if (error) {
|
||||
return (
|
||||
<Paper>
|
||||
<Center className="p-10">
|
||||
<Span size="small">{error}</Span>
|
||||
</Center>
|
||||
</Paper>
|
||||
);
|
||||
}
|
||||
|
||||
const full = record?.record;
|
||||
|
||||
if (loading || !full) {
|
||||
return (
|
||||
<Paper>
|
||||
<Center className="p-16">
|
||||
<Loading />
|
||||
</Center>
|
||||
</Paper>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<Stack className="w-full items-stretch gap-0 max-h-[70vh] overflow-y-auto">
|
||||
{table.columns.map((column) => (
|
||||
<DatabaseRecordField key={column.name} column={column} cell={full.cells[column.name]} />
|
||||
))}
|
||||
</Stack>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
import Divider from "@moduletrace/twui/layout/Divider";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import type { DatabaseCell, DatabaseTableColumn } from "@/src/types";
|
||||
|
||||
type Props = {
|
||||
column: DatabaseTableColumn;
|
||||
cell?: DatabaseCell;
|
||||
};
|
||||
|
||||
/**
|
||||
* One column of an open record: its name and declared type, then its whole
|
||||
* value. Text keeps its own line breaks and wraps, so a stored JSON body or
|
||||
* a bot's message reads as it was written.
|
||||
*/
|
||||
export default function DatabaseRecordField({ column, cell }: Props) {
|
||||
const is_null = !cell || cell.kind == "null";
|
||||
|
||||
return (
|
||||
<Stack className="w-full items-stretch gap-1 py-3">
|
||||
<Span size="smaller" variant="faded" className="font-mono">
|
||||
{`${column.name}${column.type ? ` (${column.type})` : ""}`}
|
||||
</Span>
|
||||
<Span
|
||||
size="small"
|
||||
className={twMerge(
|
||||
"block font-mono break-words",
|
||||
is_null ? "text-ink-disabled" : "text-ink whitespace-pre-wrap",
|
||||
)}
|
||||
>
|
||||
{is_null ? "NULL" : cell.text}
|
||||
</Span>
|
||||
<Divider />
|
||||
</Stack>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
import Modal from "@moduletrace/twui/elements/Modal";
|
||||
import H3 from "@moduletrace/twui/layout/H3";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import type { DatabaseTableSummary } from "@/src/types";
|
||||
import useDatabaseRecord from "../(hooks)/use-database-record";
|
||||
import DatabaseRecordBody from "./database-record-body";
|
||||
|
||||
type Props = {
|
||||
table: DatabaseTableSummary;
|
||||
/**
|
||||
* The record being read, by its key; nothing while no record is open.
|
||||
*/
|
||||
record_key: string | number | null;
|
||||
open: boolean;
|
||||
setOpen: React.Dispatch<React.SetStateAction<boolean>>;
|
||||
};
|
||||
|
||||
/**
|
||||
* One record, whole: every column's value as its own block, in the order
|
||||
* the table declares them, so the text a cell cut at three lines can be
|
||||
* read to its end.
|
||||
*/
|
||||
export default function DatabaseRecordModal({ table, record_key, open, setOpen }: Props) {
|
||||
const { record, loading, error } = useDatabaseRecord({
|
||||
table: table.name,
|
||||
key: record_key ?? undefined,
|
||||
open,
|
||||
});
|
||||
|
||||
return (
|
||||
<Modal open={open} setOpen={setOpen}>
|
||||
<Stack className="w-full items-stretch gap-5">
|
||||
<Stack className="gap-1 pr-10">
|
||||
<Row className="gap-2! items-baseline">
|
||||
<H3 className="font-mono">{table.name}</H3>
|
||||
<Span size="smaller" variant="faded" className="font-mono">
|
||||
{record_key === null ? "" : `key ${record_key}`}
|
||||
</Span>
|
||||
</Row>
|
||||
<Span size="smaller" variant="faded">
|
||||
{table.columns.length} column{table.columns.length == 1 ? "" : "s"}, read-only
|
||||
</Span>
|
||||
</Stack>
|
||||
<DatabaseRecordBody table={table} record={record} loading={loading} error={error} />
|
||||
</Stack>
|
||||
</Modal>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,28 @@
|
||||
import type { DatabaseRecord } from "@/src/types";
|
||||
import DatabaseCell from "./database-cell";
|
||||
|
||||
type Props = {
|
||||
record: DatabaseRecord;
|
||||
columns: string[];
|
||||
onOpen: () => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* One record as a table row: every column's value cut to its cell, none of
|
||||
* them spanning the whole value. A cell is the way into the record, so a
|
||||
* long text can be read to its end.
|
||||
*/
|
||||
export default function DatabaseRecordRow({ record, columns, onOpen }: Props) {
|
||||
return (
|
||||
<tr className="border-t border-slate-200 dark:border-white/10">
|
||||
{columns.map((column) => (
|
||||
<DatabaseCell
|
||||
key={column}
|
||||
cell={record.cells[column]}
|
||||
openable={record.key !== null}
|
||||
onOpen={onOpen}
|
||||
/>
|
||||
))}
|
||||
</tr>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,149 @@
|
||||
import Loading from "@moduletrace/twui/elements/Loading";
|
||||
import Pagination from "@moduletrace/twui/elements/Pagination";
|
||||
import Center from "@moduletrace/twui/layout/Center";
|
||||
import H2 from "@moduletrace/twui/layout/H2";
|
||||
import Paper from "@moduletrace/twui/elements/Paper";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import { useState } from "react";
|
||||
import type { DatabaseRecordsPayload, DatabaseSearchMode, DatabaseTableSummary } from "@/src/types";
|
||||
import tableSummaryText from "../(functions)/table-summary-text";
|
||||
import DatabaseRecordsTable from "./database-records-table";
|
||||
import DatabaseRecordModal from "./database-record-modal";
|
||||
import DatabaseSearchFields from "./database-search-fields";
|
||||
|
||||
type Props = {
|
||||
table: DatabaseTableSummary;
|
||||
records: DatabaseRecordsPayload | null;
|
||||
search: string;
|
||||
mode: DatabaseSearchMode;
|
||||
loading?: boolean;
|
||||
error?: string;
|
||||
onSubmitSearch: (search: string) => void;
|
||||
onChangeMode: (mode: DatabaseSearchMode) => void;
|
||||
onChangePage: (page: number) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* The open table's records: what it holds, the search over it, the page of
|
||||
* records and the pager under them, each record opening whole in a modal.
|
||||
*/
|
||||
export default function DatabaseRecordsPane({
|
||||
table,
|
||||
records,
|
||||
search,
|
||||
mode,
|
||||
loading,
|
||||
error,
|
||||
onSubmitSearch,
|
||||
onChangeMode,
|
||||
onChangePage,
|
||||
}: Props) {
|
||||
const [record_key, set_record_key] = useState<string | number | null>(null);
|
||||
const [modal_open, set_modal_open] = useState(false);
|
||||
|
||||
const open_record = ({ key }: { key: string | number }) => {
|
||||
set_record_key(key);
|
||||
set_modal_open(true);
|
||||
};
|
||||
|
||||
return (
|
||||
<Paper className="w-full xl:p-8 grow">
|
||||
<Stack className="grow min-w-0 w-full items-stretch">
|
||||
<Stack className="gap-1! items-stretch">
|
||||
<H2 className="text-[18px]! leading-6 font-bold tracking-normal font-mono text-ink">{table.name}</H2>
|
||||
<Span size="smaller" variant="faded">
|
||||
{tableSummaryText({ table, records })}
|
||||
</Span>
|
||||
</Stack>
|
||||
<DatabaseSearchFields
|
||||
search={search}
|
||||
mode={mode}
|
||||
loading={loading}
|
||||
onSubmit={onSubmitSearch}
|
||||
onChangeMode={onChangeMode}
|
||||
/>
|
||||
<Stack className="w-full mt-3 items-stretch">
|
||||
<DatabaseRecordsBody
|
||||
table={table}
|
||||
records={records}
|
||||
loading={loading}
|
||||
error={error}
|
||||
onOpenRecord={(key) => open_record({ key })}
|
||||
/>
|
||||
</Stack>
|
||||
{records && records.total > records.page_size ? (
|
||||
<Pagination
|
||||
count={records.total}
|
||||
page={records.page}
|
||||
setPage={(next) => {
|
||||
const page = typeof next == "function" ? next(records.page) : next;
|
||||
onChangePage(page);
|
||||
}}
|
||||
limit={records.page_size}
|
||||
className="mt-4 w-full justify-center"
|
||||
/>
|
||||
) : null}
|
||||
</Stack>
|
||||
<DatabaseRecordModal
|
||||
table={table}
|
||||
record_key={record_key}
|
||||
open={modal_open}
|
||||
setOpen={set_modal_open}
|
||||
/>
|
||||
</Paper>
|
||||
);
|
||||
}
|
||||
|
||||
type BodyProps = {
|
||||
table: DatabaseTableSummary;
|
||||
records: DatabaseRecordsPayload | null;
|
||||
loading?: boolean;
|
||||
error?: string;
|
||||
onOpenRecord: (key: string | number) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* The page of records, or what stands in for it: loading, a refusal, or a
|
||||
* table with nothing (or nothing matching) in it.
|
||||
*/
|
||||
function DatabaseRecordsBody({ table, records, loading, error, onOpenRecord }: BodyProps) {
|
||||
if (error) {
|
||||
return (
|
||||
<Paper>
|
||||
<Center className="p-16">
|
||||
<Span size="small">{error}</Span>
|
||||
</Center>
|
||||
</Paper>
|
||||
);
|
||||
}
|
||||
|
||||
if (loading || !records) {
|
||||
return (
|
||||
<Paper>
|
||||
<Center className="p-16">
|
||||
<Loading />
|
||||
</Center>
|
||||
</Paper>
|
||||
);
|
||||
}
|
||||
|
||||
if (!records.records.length) {
|
||||
return (
|
||||
<Paper>
|
||||
<Center className="p-16">
|
||||
<Stack className="items-center gap-1">
|
||||
<Span size="small">
|
||||
{records.search ? "Nothing in this table matches." : "This table is empty."}
|
||||
</Span>
|
||||
<Span size="smaller" variant="faded">
|
||||
{records.search ? "Clear the search to see its records." : table.name}
|
||||
</Span>
|
||||
</Stack>
|
||||
</Center>
|
||||
</Paper>
|
||||
);
|
||||
}
|
||||
|
||||
return <DatabaseRecordsTable records={records} onOpenRecord={onOpenRecord} />;
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { renderToString } from "react-dom/server";
|
||||
import type { DatabaseRecordsPayload, DatabaseTableSummary } from "@/src/types";
|
||||
import DatabaseRecordsTable from "./database-records-table";
|
||||
|
||||
const table: DatabaseTableSummary = {
|
||||
name: "bot_messages",
|
||||
type: "table",
|
||||
rows: 295,
|
||||
size_bytes: 123456,
|
||||
columns: [
|
||||
{ name: "id", type: "INTEGER", pk: true },
|
||||
{ name: "text", type: "TEXT", pk: false },
|
||||
{ name: "meta", type: "TEXT", pk: false },
|
||||
],
|
||||
key: { kind: "column", name: "id" },
|
||||
};
|
||||
|
||||
const records: DatabaseRecordsPayload = {
|
||||
table,
|
||||
records: [
|
||||
{
|
||||
key: 295,
|
||||
cells: {
|
||||
id: { text: "295", kind: "number", truncated: false },
|
||||
text: {
|
||||
text: "line one\nline two\nline three\nline four",
|
||||
kind: "text",
|
||||
truncated: true,
|
||||
},
|
||||
meta: { text: "", kind: "null", truncated: false },
|
||||
},
|
||||
},
|
||||
],
|
||||
total: 1,
|
||||
page: 1,
|
||||
page_size: 50,
|
||||
search: "",
|
||||
mode: "every_word",
|
||||
};
|
||||
|
||||
describe("DatabaseRecordsTable", () => {
|
||||
test("draws every column of the table", () => {
|
||||
const html = renderToString(<DatabaseRecordsTable records={records} onOpenRecord={() => {}} />);
|
||||
|
||||
expect(html).toContain(">id<");
|
||||
expect(html).toContain(">text<");
|
||||
expect(html).toContain(">meta<");
|
||||
expect(html).toContain("295");
|
||||
});
|
||||
|
||||
test("cuts a long value to three lines rather than letting it run", () => {
|
||||
const html = renderToString(<DatabaseRecordsTable records={records} onOpenRecord={() => {}} />);
|
||||
|
||||
expect(html).toContain("line-clamp-3");
|
||||
expect(html).toContain("line one");
|
||||
});
|
||||
|
||||
test("says NULL instead of leaving a cell blank", () => {
|
||||
const html = renderToString(<DatabaseRecordsTable records={records} onOpenRecord={() => {}} />);
|
||||
|
||||
expect(html).toContain("NULL");
|
||||
});
|
||||
|
||||
test("a table with no key still reads, without a way into a record", () => {
|
||||
const html = renderToString(
|
||||
<DatabaseRecordsTable
|
||||
records={{
|
||||
...records,
|
||||
table: { ...table, key: { kind: "none" } },
|
||||
records: [{ key: null, cells: records.records[0]?.cells || {} }],
|
||||
}}
|
||||
onOpenRecord={() => {}}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(html).toContain("line one");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,55 @@
|
||||
import Paper from "@moduletrace/twui/elements/Paper";
|
||||
import type { DatabaseRecordsPayload } from "@/src/types";
|
||||
import DatabaseHeadCell from "./database-head-cell";
|
||||
import DatabaseRecordRow from "./database-record-row";
|
||||
|
||||
type Props = {
|
||||
records: DatabaseRecordsPayload;
|
||||
onOpenRecord: (key: string | number) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* One page of a table's records: every column in its declared order, with
|
||||
* its declared type on hover. A record with a key opens in full when any of
|
||||
* its cells is clicked; a table with no key at all (a virtual table with no
|
||||
* primary key) reads as plain cells, since nothing can open a record it
|
||||
* can't name.
|
||||
*/
|
||||
export default function DatabaseRecordsTable({ records, onOpenRecord }: Props) {
|
||||
const { table } = records;
|
||||
|
||||
return (
|
||||
<Paper className="p-0 gap-0 overflow-x-auto">
|
||||
<table className="w-full text-sm">
|
||||
<thead className="bg-slate-50 dark:bg-white/5 sticky top-0">
|
||||
<tr>
|
||||
{table.columns.map((column) => (
|
||||
<DatabaseHeadCell
|
||||
key={column.name}
|
||||
title={column.name}
|
||||
hint={column.type || undefined}
|
||||
/>
|
||||
))}
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{records.records.map((record, index) => (
|
||||
<DatabaseRecordRow
|
||||
key={
|
||||
record.key === null
|
||||
? `row-${index}`
|
||||
: String(record.key)
|
||||
}
|
||||
record={record}
|
||||
columns={table.columns.map((column) => column.name)}
|
||||
onOpen={() => {
|
||||
if (record.key !== null)
|
||||
onOpenRecord(record.key);
|
||||
}}
|
||||
/>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</Paper>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,59 @@
|
||||
import Input from "@moduletrace/twui/form/Input";
|
||||
import Select from "@moduletrace/twui/form/Select";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import { useEffect, useState } from "react";
|
||||
import type { DatabaseSearchMode } from "@/src/types";
|
||||
import { DatabaseSearchModes } from "../(data)/database-search-modes";
|
||||
|
||||
type Props = {
|
||||
search: string;
|
||||
mode: DatabaseSearchMode;
|
||||
loading?: boolean;
|
||||
onSubmit: (search: string) => void;
|
||||
onChangeMode: (mode: DatabaseSearchMode) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* Narrowing the open table down: a search over every column's text and how
|
||||
* it matches. Enter (or the clear button, which runs it empty) is what asks
|
||||
* for it, so typing doesn't fire a query a keystroke.
|
||||
*/
|
||||
export default function DatabaseSearchFields({ search, mode, loading, onSubmit, onChangeMode }: Props) {
|
||||
const [value, set_value] = useState(search);
|
||||
|
||||
// The search the browser is showing can change without this input: the
|
||||
// browser's back button, or a link that carries its own.
|
||||
useEffect(() => set_value(search), [search]);
|
||||
|
||||
return (
|
||||
<Row className="w-full mt-3 gap-2! flex-wrap items-start md:flex-nowrap">
|
||||
<Input
|
||||
placeholder="Search every column of this table"
|
||||
value={value}
|
||||
disabled={loading}
|
||||
onChange={(event) => set_value(event.target.value)}
|
||||
onKeyDown={(event) => {
|
||||
if (event.key == "Enter") onSubmit(event.currentTarget.value);
|
||||
}}
|
||||
clearInputProps={{
|
||||
onClick: () => {
|
||||
set_value("");
|
||||
onSubmit("");
|
||||
},
|
||||
}}
|
||||
wrapperWrapperProps={{ className: "w-full grow" }}
|
||||
/>
|
||||
<Select
|
||||
title="How the search matches"
|
||||
aria-label="How the search matches"
|
||||
options={DatabaseSearchModes.map((option) => ({
|
||||
title: option.title,
|
||||
value: option.value,
|
||||
default: option.value == mode,
|
||||
}))}
|
||||
changeHandler={(next) => onChangeMode(next)}
|
||||
wrapperWrapperProps={{ className: "w-[150px] shrink-0" }}
|
||||
/>
|
||||
</Row>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
import Divider from "@moduletrace/twui/layout/Divider";
|
||||
import H2 from "@moduletrace/twui/layout/H2";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import type { DatabaseTableSummary } from "@/src/types";
|
||||
import DatabaseTableRow from "./database-table-row";
|
||||
|
||||
type Props = {
|
||||
tables: DatabaseTableSummary[];
|
||||
/**
|
||||
* The table the browser is showing.
|
||||
*/
|
||||
table: string;
|
||||
onOpenTable: (table: string) => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* Every table of the database, in one list beside the open one's records.
|
||||
* The record counts are the browser's first read, so the list already
|
||||
* says where the data sits before anything is clicked.
|
||||
*/
|
||||
export default function DatabaseTableList({ tables, table, onOpenTable }: Props) {
|
||||
return (
|
||||
<Stack className="w-full lg:w-[240px] shrink-0 items-stretch">
|
||||
<H2 className="mb-2 text-[18px]! leading-6 font-bold tracking-normal text-ink">Tables</H2>
|
||||
<Span className="mb-2 text-[13px] leading-[18px] text-ink-secondary">
|
||||
{tables.length} table{tables.length == 1 ? "" : "s"}
|
||||
</Span>
|
||||
<Divider />
|
||||
<Stack className="gap-0! items-stretch">
|
||||
{tables.map((entry) => (
|
||||
<DatabaseTableRow
|
||||
key={entry.name}
|
||||
table={entry}
|
||||
selected={entry.name == table}
|
||||
onOpen={() => onOpenTable(entry.name)}
|
||||
/>
|
||||
))}
|
||||
</Stack>
|
||||
</Stack>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
import { describe, expect, test } from "bun:test";
|
||||
import { renderToString } from "react-dom/server";
|
||||
import type { DatabaseTableSummary } from "@/src/types";
|
||||
import DatabaseTableRow from "./database-table-row";
|
||||
|
||||
const table: DatabaseTableSummary = {
|
||||
name: "bot_file_vectors",
|
||||
type: "virtual",
|
||||
rows: 12,
|
||||
size_bytes: 2048,
|
||||
columns: [],
|
||||
key: { kind: "none" },
|
||||
};
|
||||
|
||||
describe("DatabaseTableRow", () => {
|
||||
test("names the table, its record count and its size", () => {
|
||||
const html = renderToString(<DatabaseTableRow table={table} selected={false} onOpen={() => {}} />);
|
||||
|
||||
expect(html).toContain("bot_file_vectors");
|
||||
expect(html).toContain("12");
|
||||
expect(html).toContain("2.0 KB");
|
||||
});
|
||||
|
||||
test("links to the table's own address", () => {
|
||||
const html = renderToString(<DatabaseTableRow table={table} selected={false} onOpen={() => {}} />);
|
||||
|
||||
expect(html).toContain(`href="/admin/settings/database?table=bot_file_vectors"`);
|
||||
});
|
||||
|
||||
test("a table with no count yet shows a dash", () => {
|
||||
const html = renderToString(
|
||||
<DatabaseTableRow
|
||||
table={{ ...table, rows: null, size_bytes: null }}
|
||||
selected={false}
|
||||
onOpen={() => {}}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(html).toContain("-");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,50 @@
|
||||
import Link from "@moduletrace/twui/layout/Link";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import { Layers, Table2 } from "lucide-react";
|
||||
import { twMerge } from "tailwind-merge";
|
||||
import formatBytes from "@/src/utils/format-bytes";
|
||||
import type { DatabaseTableSummary } from "@/src/types";
|
||||
import databaseHref from "../(functions)/database-href";
|
||||
|
||||
type Props = {
|
||||
table: DatabaseTableSummary;
|
||||
selected: boolean;
|
||||
onOpen: () => void;
|
||||
};
|
||||
|
||||
/**
|
||||
* One table in the database browser's picker: its name, what it holds and
|
||||
* what it takes on disk, with a virtual table's icon telling it apart from
|
||||
* the ordinary ones.
|
||||
*/
|
||||
export default function DatabaseTableRow({ table, selected, onOpen }: Props) {
|
||||
const Icon = table.type == "virtual" ? Layers : Table2;
|
||||
const rows = table.rows === null ? "-" : table.rows.toLocaleString();
|
||||
const size = table.size_bytes === null ? null : formatBytes({ bytes: table.size_bytes });
|
||||
|
||||
return (
|
||||
<Link
|
||||
href={databaseHref({ table: table.name })}
|
||||
title={`${table.name}: ${rows} record${table.rows == 1 ? "" : "s"}${size ? `, ${size}` : ""}`}
|
||||
aria-current={selected ? "page" : undefined}
|
||||
onClick={(event) => {
|
||||
event.preventDefault();
|
||||
onOpen();
|
||||
}}
|
||||
className={twMerge(
|
||||
"flex w-full min-w-0 items-center gap-2 rounded-default px-2 py-1.5 border-0 text-ink",
|
||||
"hover:opacity-100 hover:bg-sunken transition-colors duration-[var(--m-fast)]",
|
||||
selected && "bg-ink/5 hover:bg-ink/5",
|
||||
)}
|
||||
>
|
||||
<Span
|
||||
title={table.type == "virtual" ? "Virtual table" : "Table"}
|
||||
className="size-6 shrink-0 flex items-center justify-center rounded-default-sm bg-sunken text-ink-secondary"
|
||||
>
|
||||
<Icon size={14} aria-hidden />
|
||||
</Span>
|
||||
<Span className="min-w-0 flex-1 truncate font-mono text-[13px] leading-5">{table.name}</Span>
|
||||
<Span className="shrink-0 text-[11px] leading-4 text-ink-secondary">{rows}</Span>
|
||||
</Link>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,109 @@
|
||||
import Center from "@moduletrace/twui/layout/Center";
|
||||
import Row from "@moduletrace/twui/layout/Row";
|
||||
import Span from "@moduletrace/twui/layout/Span";
|
||||
import Stack from "@moduletrace/twui/layout/Stack";
|
||||
import { useContext, useState } from "react";
|
||||
import { AppContext } from "@/src/pages/__root";
|
||||
import type { DatabaseSearchMode } from "@/src/types";
|
||||
import databaseHref from "../(functions)/database-href";
|
||||
import useDatabaseRecords from "../(hooks)/use-database-records";
|
||||
import DatabaseRecordsPane from "../(partials)/database-records-pane";
|
||||
import DatabaseTableList from "../(partials)/database-table-list";
|
||||
|
||||
/**
|
||||
* The database browser's state: which table is open, what is searched in
|
||||
* it and which page of its records, kept in the address so a table or a
|
||||
* search can be linked to and the browser's own back button walks back
|
||||
* through them. The open table's first page comes with the page; anything
|
||||
* else loads through the API.
|
||||
*/
|
||||
export default function DatabaseBrowser() {
|
||||
const { pageProps } = useContext(AppContext);
|
||||
const tables = pageProps?.database_tables || [];
|
||||
const page_records = pageProps?.database_records;
|
||||
const [table, set_table] = useState(page_records?.table.name || tables[0]?.name || "");
|
||||
const [search, set_search] = useState("");
|
||||
const [mode, set_mode] = useState<DatabaseSearchMode>("every_word");
|
||||
const [page, set_page] = useState(1);
|
||||
const { records, loading, error } = useDatabaseRecords({
|
||||
table,
|
||||
search,
|
||||
mode,
|
||||
page,
|
||||
page_records,
|
||||
});
|
||||
const open = records?.table || tables.find((entry) => entry.name == table) || null;
|
||||
|
||||
const navigate = ({
|
||||
next_table,
|
||||
next_search,
|
||||
next_mode,
|
||||
next_page,
|
||||
}: {
|
||||
next_table: string;
|
||||
next_search: string;
|
||||
next_mode: DatabaseSearchMode;
|
||||
next_page: number;
|
||||
}) => {
|
||||
set_table(next_table);
|
||||
set_search(next_search);
|
||||
set_mode(next_mode);
|
||||
set_page(next_page);
|
||||
|
||||
const url = new URL(databaseHref({
|
||||
table: next_table,
|
||||
search: next_search,
|
||||
mode: next_mode,
|
||||
page: next_page,
|
||||
}), window.location.origin);
|
||||
|
||||
history.pushState(null, "", url);
|
||||
};
|
||||
|
||||
if (!tables.length) {
|
||||
return (
|
||||
<Center className="p-16">
|
||||
<Span size="small">
|
||||
The database has no tables yet, or they couldn't be read.
|
||||
</Span>
|
||||
</Center>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<Row className="w-full mt-6 gap-6! flex-wrap items-start lg:flex-nowrap">
|
||||
<DatabaseTableList
|
||||
tables={tables}
|
||||
table={table}
|
||||
onOpenTable={(next_table) =>
|
||||
navigate({ next_table, next_search: "", next_mode: mode, next_page: 1 })
|
||||
}
|
||||
/>
|
||||
{open ? (
|
||||
<DatabaseRecordsPane
|
||||
table={open}
|
||||
records={records}
|
||||
search={search}
|
||||
mode={mode}
|
||||
loading={loading}
|
||||
error={error}
|
||||
onSubmitSearch={(next_search) =>
|
||||
navigate({ next_table: table, next_search, next_mode: mode, next_page: 1 })
|
||||
}
|
||||
onChangeMode={(next_mode) =>
|
||||
navigate({ next_table: table, next_search: search, next_mode, next_page: 1 })
|
||||
}
|
||||
onChangePage={(next_page) =>
|
||||
navigate({ next_table: table, next_search: search, next_mode: mode, next_page })
|
||||
}
|
||||
/>
|
||||
) : (
|
||||
<Stack className="grow items-stretch">
|
||||
<Center className="p-16">
|
||||
<Span size="small">Pick a table to read its records.</Span>
|
||||
</Center>
|
||||
</Stack>
|
||||
)}
|
||||
</Row>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
import type { PagePropsType } from "@/src/types";
|
||||
import type { BunextPageServerFn } from "@moduletrace/bunext/types";
|
||||
import grabDatabaseServerProps from "./(functions)/grab-database-server-props";
|
||||
|
||||
export const server: BunextPageServerFn<PagePropsType> = async (params) => {
|
||||
return await grabDatabaseServerProps(params);
|
||||
};
|
||||
@@ -0,0 +1,17 @@
|
||||
import SettingsShell from "../(partials)/settings-shell";
|
||||
import settingsPageMeta from "../(functions)/settings-page-meta";
|
||||
import DatabaseBrowser from "./(sections)/database-browser";
|
||||
|
||||
export default function AdminSettingsDatabasePage() {
|
||||
return (
|
||||
<SettingsShell
|
||||
title="Database"
|
||||
description="Every table and every record, read-only: open a record to read a value whole, search a table across all of its columns."
|
||||
wide
|
||||
>
|
||||
<DatabaseBrowser />
|
||||
</SettingsShell>
|
||||
);
|
||||
}
|
||||
|
||||
export const meta = settingsPageMeta({ label: "Database" });
|
||||
@@ -0,0 +1,39 @@
|
||||
import userAuth from "@/src/functions/backend/auth/user-auth";
|
||||
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
|
||||
import grabTableRecord from "@/src/functions/backend/database/grab-table-record";
|
||||
import { DatabaseRecordRequestSchema } from "@/src/schemas/database-schema";
|
||||
import type { DatabaseRecordPayload } from "@/src/types";
|
||||
import parseWithSchema from "@/src/utils/parse-with-schema";
|
||||
import safeErrorMessage from "@/src/utils/safe-error-message";
|
||||
import type { APIResponseObject, BunextAPIRouteHandler } from "@moduletrace/bunext/types";
|
||||
|
||||
/**
|
||||
* Super admin only: one record of one table, in full for its modal,
|
||||
* read-only.
|
||||
*/
|
||||
export const handler: BunextAPIRouteHandler<APIResponseObject<DatabaseRecordPayload>> = async ({ req, query }) => {
|
||||
try {
|
||||
if (req.method != "GET") {
|
||||
return { success: false, msg: `Method not allowed` };
|
||||
}
|
||||
|
||||
const { user } = await userAuth({ req });
|
||||
|
||||
if (!user?.logged_in_status || !user.id) {
|
||||
return { success: false, msg: `Unauthorized`, logoutUser: true };
|
||||
}
|
||||
|
||||
if (!(await grabSuperAdminUser({ req }))) {
|
||||
return { success: false, msg: `Only a super admin can read the database.` };
|
||||
}
|
||||
|
||||
const request = parseWithSchema({
|
||||
schema: DatabaseRecordRequestSchema,
|
||||
data: query,
|
||||
});
|
||||
|
||||
return { success: true, singleRes: await grabTableRecord(request) };
|
||||
} catch (error: unknown) {
|
||||
return { success: false, msg: safeErrorMessage({ error }) };
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,41 @@
|
||||
import userAuth from "@/src/functions/backend/auth/user-auth";
|
||||
import grabSuperAdminUser from "@/src/functions/backend/backups/grab-super-admin-user";
|
||||
import grabTableRecords from "@/src/functions/backend/database/grab-table-records";
|
||||
import { DatabaseRecordsRequestSchema } from "@/src/schemas/database-schema";
|
||||
import type { DatabaseRecordsPayload } from "@/src/types";
|
||||
import parseWithSchema from "@/src/utils/parse-with-schema";
|
||||
import safeErrorMessage from "@/src/utils/safe-error-message";
|
||||
import type { APIResponseObject, BunextAPIRouteHandler } from "@moduletrace/bunext/types";
|
||||
|
||||
/**
|
||||
* Super admin only: one page of one table's records, read-only. Values
|
||||
* come back already cut short; a record opened in full is
|
||||
* `api/admin/database/record`.
|
||||
*/
|
||||
export const handler: BunextAPIRouteHandler<APIResponseObject<DatabaseRecordsPayload>> = async ({ req, query }) => {
|
||||
try {
|
||||
if (req.method != "GET") {
|
||||
return { success: false, msg: `Method not allowed` };
|
||||
}
|
||||
|
||||
const { user } = await userAuth({ req });
|
||||
|
||||
if (!user?.logged_in_status || !user.id) {
|
||||
return { success: false, msg: `Unauthorized`, logoutUser: true };
|
||||
}
|
||||
|
||||
if (!(await grabSuperAdminUser({ req }))) {
|
||||
return { success: false, msg: `Only a super admin can read the database.` };
|
||||
}
|
||||
|
||||
const request = parseWithSchema({
|
||||
schema: DatabaseRecordsRequestSchema,
|
||||
data: query,
|
||||
});
|
||||
const records = await grabTableRecords(request);
|
||||
|
||||
return { success: true, singleRes: records };
|
||||
} catch (error: unknown) {
|
||||
return { success: false, msg: safeErrorMessage({ error }) };
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,35 @@
|
||||
import { z } from "zod";
|
||||
import { AppData } from "@/src/data/app-data";
|
||||
|
||||
/**
|
||||
* The table a database browser request asks for, by its own name.
|
||||
*/
|
||||
const TableField = z
|
||||
.string()
|
||||
.min(1, "Pick a table")
|
||||
.max(120, "That isn't a table name");
|
||||
|
||||
/**
|
||||
* `GET api/admin/database/records`: one page of one table's records.
|
||||
*/
|
||||
export const DatabaseRecordsRequestSchema = z.object({
|
||||
table: TableField.optional(),
|
||||
page: z.coerce.number().int().positive().default(1),
|
||||
search: z.string().max(200).optional(),
|
||||
mode: z.enum(["every_word", "phrase"]).default("every_word"),
|
||||
});
|
||||
|
||||
export type DatabaseRecordsRequest = z.infer<typeof DatabaseRecordsRequestSchema>;
|
||||
|
||||
/**
|
||||
* `GET api/admin/database/record`: one record, by its key.
|
||||
*/
|
||||
export const DatabaseRecordRequestSchema = z.object({
|
||||
table: TableField,
|
||||
key: z
|
||||
.string()
|
||||
.min(1, "Pick a record")
|
||||
.max(AppData.DatabaseBrowserCellMaxChars, "That isn't a record key"),
|
||||
});
|
||||
|
||||
export type DatabaseRecordRequest = z.infer<typeof DatabaseRecordRequestSchema>;
|
||||
@@ -835,6 +835,97 @@ export type StorageFolderPayload = {
|
||||
missing?: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* What a database table's rows are identified by: a primary key column,
|
||||
* the rowid a table without one still has, or nothing at all (a virtual
|
||||
* table with neither).
|
||||
*/
|
||||
export type DatabaseTableKey = {
|
||||
kind: "rowid" | "column" | "none";
|
||||
/**
|
||||
* The primary key column's name, for `kind: "column"`.
|
||||
*/
|
||||
name?: string;
|
||||
};
|
||||
|
||||
export type DatabaseTableColumn = {
|
||||
name: string;
|
||||
/**
|
||||
* SQLite's declared type, empty when the column has none (a view or a
|
||||
* virtual table).
|
||||
*/
|
||||
type: string;
|
||||
pk: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* How a search matches: every word somewhere in the record's text, or the
|
||||
* whole phrase somewhere in it.
|
||||
*/
|
||||
export type DatabaseSearchMode = "every_word" | "phrase";
|
||||
|
||||
/**
|
||||
* One table of the database, as the database browser lists it.
|
||||
*/
|
||||
export type DatabaseTableSummary = {
|
||||
name: string;
|
||||
type: "table" | "virtual";
|
||||
/**
|
||||
* How many records it holds, when it could be counted.
|
||||
*/
|
||||
rows: number | null;
|
||||
/**
|
||||
* What the table's pages take on disk, when SQLite can measure it.
|
||||
*/
|
||||
size_bytes: number | null;
|
||||
columns: DatabaseTableColumn[];
|
||||
key: DatabaseTableKey;
|
||||
};
|
||||
|
||||
/**
|
||||
* One value in a record, as a cell shows it. `text` is the whole value,
|
||||
* or its first `AppData.DatabaseBrowserCellMaxChars` characters when
|
||||
* `truncated` says so.
|
||||
*/
|
||||
export type DatabaseCell = {
|
||||
text: string;
|
||||
kind: "text" | "number" | "blob" | "null";
|
||||
truncated: boolean;
|
||||
};
|
||||
|
||||
/**
|
||||
* One record of a table: what identifies it, and its columns' values.
|
||||
*/
|
||||
export type DatabaseRecord = {
|
||||
key: string | number | null;
|
||||
cells: Record<string, DatabaseCell>;
|
||||
};
|
||||
|
||||
/**
|
||||
* One page of a table's records.
|
||||
*/
|
||||
export type DatabaseRecordsPayload = {
|
||||
table: DatabaseTableSummary;
|
||||
records: DatabaseRecord[];
|
||||
total: number;
|
||||
page: number;
|
||||
page_size: number;
|
||||
/**
|
||||
* The search these records answer, so the page knows whether its pager
|
||||
* counts the table or a narrowing of it.
|
||||
*/
|
||||
search: string;
|
||||
mode: DatabaseSearchMode;
|
||||
};
|
||||
|
||||
/**
|
||||
* One record, in full, for its modal.
|
||||
*/
|
||||
export type DatabaseRecordPayload = {
|
||||
table: DatabaseTableSummary;
|
||||
record: DatabaseRecord | null;
|
||||
};
|
||||
|
||||
/**
|
||||
* A folder in the files browser's picker.
|
||||
*/
|
||||
@@ -1069,6 +1160,12 @@ export type PagePropsType = {
|
||||
*/
|
||||
storage_folders?: StorageSpecOption[] | null;
|
||||
storage_folder?: StorageFolderPayload | null;
|
||||
/**
|
||||
* The database browser's tables (super admin only) and the first page
|
||||
* of the one it is showing.
|
||||
*/
|
||||
database_tables?: DatabaseTableSummary[] | null;
|
||||
database_records?: DatabaseRecordsPayload | null;
|
||||
/**
|
||||
* The open conversation's files (a bot's own, or a workspace's shared
|
||||
* ones), for the files modal and the chips that name one.
|
||||
|
||||
Reference in New Issue
Block a user