tbenandClaude Opus 5.5
7c470e0225
A table with `queue_table` set works as a job queue that many workers can
pull from at once. Claims use SELECT ... FOR UPDATE SKIP LOCKED over the
idx_queue_claim index order, so each job goes to one worker and only the
claimed rows are locked.
- Schema: adds the queue_* columns and the idx_queue_claim /
idx_queue_lease indexes. Declaring these names yourself is an error.
- `queue_table` accepts `true` or { max_attempts, lease_ms, keep_completed }
- BunMariaDB.queue: push, claim, ack, fail, extend, cancel
- Statuses: pending, processing, done, failed, cancelled
- Leases use the database clock. Expired leases are reclaimed (or
dead-lettered on the last attempt) and old lock tokens stop working.
- insert/update can't set queue state fields. update/delete skip claimed
jobs.
Tests: allow the test container's self-signed certificate, and give the
queue test its own client so it doesn't reuse one closed by db.test.
Co-Authored-By: Claude Opus 5.5 <[email protected]>
1101 lines
35 KiB
Markdown
1101 lines
35 KiB
Markdown
# Bun MariaDB
|
||
|
||
`@moduletrace/bun-mariadb`
|
||
|
||
A schema-driven MariaDB manager for [Bun](https://bun.sh), featuring automatic schema synchronization, type-safe CRUD, native `VECTOR` support, HTML sanitization, and TypeScript type definition generation.
|
||
|
||
---
|
||
|
||
## Table of Contents
|
||
|
||
- [Features](#features)
|
||
- [Prerequisites](#prerequisites)
|
||
- [Installation](#installation)
|
||
- [Environment Variables](#environment-variables)
|
||
- [Quick Start](#quick-start)
|
||
- [Configuration](#configuration)
|
||
- [Schema Definition](#schema-definition)
|
||
- [CLI Commands](#cli-commands)
|
||
- [`schema`](#schema--sync-database-to-schema)
|
||
- [`typedef`](#typedef--generate-typescript-types-only)
|
||
- [`backup`](#backup--sql-dump-backup)
|
||
- [`restore`](#restore--restore-from-an-sql-dump)
|
||
- [`export`](#export--portable-sql--schema-archive)
|
||
- [`import`](#import--sql-or-full-export-archive)
|
||
- [`admin`](#admin--interactive-admin-tools)
|
||
- [CRUD API](#crud-api)
|
||
- [Select](#select)
|
||
- [Insert](#insert)
|
||
- [Update](#update)
|
||
- [Delete](#delete)
|
||
- [Raw SQL](#raw-sql)
|
||
- [Queue Tables](#queue-tables)
|
||
- [Query API Reference](#query-api-reference)
|
||
- [Vector Support](#vector-support)
|
||
- [HTML Sanitization](#html-sanitization)
|
||
- [TypeScript Type Generation](#typescript-type-generation)
|
||
- [Default Fields](#default-fields)
|
||
- [Project Structure](#project-structure)
|
||
- [Testing](#testing)
|
||
|
||
---
|
||
|
||
## Features
|
||
|
||
- **Schema-first design** — define your database in TypeScript; the library syncs MariaDB to match
|
||
- **Automatic migrations** — adds/modifies/drops columns, manages indexes & foreign keys, drops removed tables
|
||
- **Type-safe CRUD** — generic `select`, `insert`, `update`, `delete`, and raw `sql`
|
||
- **Rich query DSL** — filtering, ordering, pagination, joins, grouping, full-text search
|
||
- **Native VECTOR columns** — MariaDB `VECTOR(n)` types and `VECTOR INDEX` support
|
||
- **HTML sanitization** — fields with `html: true` are sanitized on insert/update
|
||
- **TypeScript codegen** — generate `.ts` type definitions from your schema
|
||
- **Zero-config defaults** — `id`, `created_at`, and `updated_at` are added to every table automatically
|
||
|
||
---
|
||
|
||
## Prerequisites
|
||
|
||
- [Bun](https://bun.sh) installed
|
||
- A running MariaDB server (11.7+ recommended for native `VECTOR` support)
|
||
|
||
---
|
||
|
||
## Installation
|
||
|
||
### Via the private npm registry
|
||
|
||
`@moduletrace/bun-mariadb` is published to a private Gitea npm registry. Configure the `@moduletrace` scope first:
|
||
|
||
```ini
|
||
# .npmrc
|
||
@moduletrace:registry=https://git.tben.me/api/packages/moduletrace/npm/
|
||
```
|
||
|
||
```bash
|
||
bun add @moduletrace/bun-mariadb
|
||
```
|
||
|
||
### Via Git
|
||
|
||
```bash
|
||
bun add github:moduletrace/bun-mariadb
|
||
```
|
||
|
||
---
|
||
|
||
## Environment Variables
|
||
|
||
Connection settings are read from the environment (not the config file):
|
||
|
||
| Variable | Required | Description |
|
||
| --------------------------------- | -------- | ---------------------------------- |
|
||
| `BUN_MARIADB_SERVER_HOST` | Yes | MariaDB host |
|
||
| `BUN_MARIADB_SERVER_USERNAME` | Yes | Database user |
|
||
| `BUN_MARIADB_SERVER_PASSWORD` | Yes | Database password |
|
||
| `BUN_MARIADB_SERVER_PORT` | No | Port (server default if omitted) |
|
||
| `BUN_MARIADB_SERVER_SSL_KEY_PATH` | No | Optional SSL key path (legacy/env) |
|
||
|
||
Example `.env`:
|
||
|
||
```env
|
||
BUN_MARIADB_SERVER_HOST=127.0.0.1
|
||
BUN_MARIADB_SERVER_PORT=3306
|
||
BUN_MARIADB_SERVER_USERNAME=root
|
||
BUN_MARIADB_SERVER_PASSWORD=secret
|
||
```
|
||
|
||
---
|
||
|
||
## Quick Start
|
||
|
||
### 1. Create the config file
|
||
|
||
Create `bun-mariadb.config.ts` at your project root:
|
||
|
||
```ts
|
||
import type { BunMariaDBConfig } from "@moduletrace/bun-mariadb";
|
||
|
||
const config: BunMariaDBConfig = {
|
||
db_name: "my_app",
|
||
db_dir: "./db",
|
||
typedef_file_path: "./db/types/db.ts",
|
||
// ssl_ca: "./db/ca-cert.pem",
|
||
};
|
||
|
||
export default config;
|
||
```
|
||
|
||
### 2. Define your schema
|
||
|
||
Create `./db/schema.ts` (filename is fixed as `schema.ts` inside `db_dir`):
|
||
|
||
```ts
|
||
import type { BUN_MARIADB_DatabaseSchemaType } from "@moduletrace/bun-mariadb";
|
||
|
||
const schema: BUN_MARIADB_DatabaseSchemaType = {
|
||
tables: [
|
||
{
|
||
tableName: "users",
|
||
fields: [
|
||
{
|
||
fieldName: "first_name",
|
||
dataType: "VARCHAR",
|
||
dataLength: 255,
|
||
},
|
||
{
|
||
fieldName: "last_name",
|
||
dataType: "VARCHAR",
|
||
dataLength: 255,
|
||
},
|
||
{
|
||
fieldName: "email",
|
||
dataType: "VARCHAR",
|
||
dataLength: 255,
|
||
unique: true,
|
||
},
|
||
{ fieldName: "bio", dataType: "LONGTEXT", html: true },
|
||
],
|
||
},
|
||
],
|
||
};
|
||
|
||
export default schema;
|
||
```
|
||
|
||
### 3. Sync the schema to MariaDB
|
||
|
||
```bash
|
||
bunx bun-mariadb schema
|
||
```
|
||
|
||
This creates/updates all tables, indexes, and foreign keys to match your schema.
|
||
|
||
### 4. Use the CRUD API
|
||
|
||
```ts
|
||
import BunMariaDB from "@moduletrace/bun-mariadb";
|
||
|
||
// Insert
|
||
await BunMariaDB.insert({
|
||
table: "users",
|
||
data: [{ first_name: "Alice", email: "[email protected]" }],
|
||
});
|
||
|
||
// Select
|
||
const result = await BunMariaDB.select({ table: "users" });
|
||
console.log(result.payload);
|
||
|
||
// Update
|
||
await BunMariaDB.update({
|
||
table: "users",
|
||
targetId: 1,
|
||
data: { first_name: "Alicia" },
|
||
});
|
||
|
||
// Delete
|
||
await BunMariaDB.delete({ table: "users", targetId: 1 });
|
||
```
|
||
|
||
---
|
||
|
||
## Configuration
|
||
|
||
The config file must be named `bun-mariadb.config.ts` and placed at the project root.
|
||
|
||
| Field | Type | Required | Description |
|
||
| -------------------- | -------- | -------- | ----------------------------------------------------------------------------- |
|
||
| `db_name` | `string` | Yes | MariaDB database name |
|
||
| `db_dir` | `string` | Yes | Directory for schema, types, and local artifacts (relative to project root) |
|
||
| `db_backup_dir` | `string` | No | Backup directory name, relative to `db_dir` (default: `.backups`) |
|
||
| `max_backups` | `number` | No | Max backup files to keep (default: `10`) |
|
||
| `max_exports` | `number` | No | Max export archives to keep (default: `10`) |
|
||
| `typedef_file_path` | `string` | No | Output path for generated TypeScript types (relative to project root) |
|
||
| `db_config` | `object` | No | Extra options passed to Bun's `SQL` MariaDB adapter |
|
||
| `charset` | `string` | No | Database charset (default: `utf8mb4`) |
|
||
| `connection_timeout` | `number` | No | Connection timeout in ms (default: `10000`) |
|
||
| `ssl_ca` | `string` | No | Path to SSL CA certificate (relative to project root) |
|
||
| `ssl_reject_unauthorized` | `boolean` | No | Verify the server TLS certificate (default: `true`). Set `false` only for trusted self-signed setups |
|
||
| `html_sanitize` | `object` | No | Extra HTML sanitizer allowlists (see [HTML Sanitization](#html-sanitization)) |
|
||
|
||
Schema file path is always: `{db_dir}/schema.ts`.
|
||
|
||
---
|
||
|
||
## Schema Definition
|
||
|
||
### Database Schema
|
||
|
||
```ts
|
||
interface BUN_MARIADB_DatabaseSchemaType {
|
||
tables: BUN_MARIADB_TableSchemaType[];
|
||
collation?: "utf8mb4_bin" | "utf8mb4_unicode_520_ci";
|
||
}
|
||
```
|
||
|
||
### Table Schema
|
||
|
||
```ts
|
||
interface BUN_MARIADB_TableSchemaType {
|
||
tableName: string;
|
||
tableDescription?: string;
|
||
fields: BUN_MARIADB_FieldSchemaType[];
|
||
indexes?: BUN_MARIADB_IndexSchemaType[];
|
||
uniqueConstraints?: BUN_MARIADB_UniqueConstraintSchemaType[];
|
||
parentTableName?: string; // inherit / merge fields from another table
|
||
tableNameOld?: string; // rename: old name triggers ALTER TABLE RENAME
|
||
collation?: "utf8mb4_bin" | "utf8mb4_unicode_520_ci";
|
||
queue_table?: boolean | BUN_MARIADB_QueueTableOptions; // job queue, see Queue Tables
|
||
}
|
||
```
|
||
|
||
### Field Schema
|
||
|
||
```ts
|
||
type BUN_MARIADB_FieldSchemaType = {
|
||
fieldName?: string;
|
||
dataType:
|
||
| "CHAR"
|
||
| "VARCHAR"
|
||
| "TEXT"
|
||
| "TINYTEXT"
|
||
| "MEDIUMTEXT"
|
||
| "LONGTEXT"
|
||
| "TINYINT"
|
||
| "SMALLINT"
|
||
| "MEDIUMINT"
|
||
| "INT"
|
||
| "BIGINT"
|
||
| "FLOAT"
|
||
| "DOUBLE"
|
||
| "DECIMAL"
|
||
| "BINARY"
|
||
| "VARBINARY"
|
||
| "BLOB"
|
||
| "TINYBLOB"
|
||
| "MEDIUMBLOB"
|
||
| "LONGBLOB"
|
||
| "DATE"
|
||
| "TIME"
|
||
| "DATETIME"
|
||
| "TIMESTAMP"
|
||
| "YEAR"
|
||
| "BOOLEAN"
|
||
| "UUID"
|
||
| "JSON"
|
||
| "INET6"
|
||
| "ENUM"
|
||
| "SET"
|
||
| "VECTOR";
|
||
primaryKey?: boolean;
|
||
autoIncrement?: boolean;
|
||
notNullValue?: boolean;
|
||
unique?: boolean;
|
||
defaultValue?: string | number;
|
||
defaultValueLiteral?: string; // raw SQL, e.g. "CURRENT_TIMESTAMP"
|
||
onUpdate?: string;
|
||
onUpdateLiteral?: string;
|
||
foreignKey?: BUN_MARIADB_ForeignKeyType;
|
||
dataLength?: string | number; // e.g. VARCHAR length
|
||
decimals?: string | number; // DECIMAL scale
|
||
options?: (string | number)[]; // ENUM / SET values
|
||
isVector?: boolean; // native VECTOR column
|
||
vectorSize?: number; // dimensions (default: 1536)
|
||
// Content mode flags (editor metadata); only `html: true` affects runtime:
|
||
html?: boolean;
|
||
markdown?: boolean;
|
||
plain?: boolean;
|
||
// ...
|
||
};
|
||
```
|
||
|
||
### Foreign Key
|
||
|
||
```ts
|
||
type BUN_MARIADB_ForeignKeyRule =
|
||
| "CASCADE"
|
||
| "SET NULL"
|
||
| "RESTRICT"
|
||
| "NO ACTION";
|
||
|
||
interface BUN_MARIADB_ForeignKeyType {
|
||
foreignKeyName?: string;
|
||
destinationTableName?: string;
|
||
destinationTableColumnName?: string;
|
||
cascadeDelete?: boolean; // shorthand for onDelete: "CASCADE"
|
||
cascadeUpdate?: boolean; // shorthand for onUpdate: "CASCADE"
|
||
onDelete?: BUN_MARIADB_ForeignKeyRule; // default RESTRICT
|
||
onUpdate?: BUN_MARIADB_ForeignKeyRule; // default RESTRICT
|
||
}
|
||
```
|
||
|
||
```ts
|
||
{
|
||
fieldName: "author_id",
|
||
dataType: "BIGINT",
|
||
foreignKey: {
|
||
foreignKeyName: "post_author_fk",
|
||
destinationTableName: "users",
|
||
destinationTableColumnName: "id",
|
||
onDelete: "SET NULL", // column must be nullable
|
||
},
|
||
}
|
||
```
|
||
|
||
- `onDelete` / `onUpdate` take precedence over the `cascadeDelete` / `cascadeUpdate` shorthands. The deprecated field-level `onDeleteLiteral` / `onDelete` are still honored as the FK's ON DELETE action when `foreignKey.onDelete` is not set.
|
||
- `SET NULL` on a `notNullValue` or primary key column is rejected when the schema syncs.
|
||
- Changing a rule on an existing FK drops and recreates the constraint. `NO ACTION` and `RESTRICT` are treated as the same rule, so switching between them causes no churn.
|
||
- Foreign keys are created after every table exists. Self references (e.g. `comments.parent_comment_id`), forward references and cycles between tables are all supported.
|
||
|
||
### Index
|
||
|
||
```ts
|
||
interface BUN_MARIADB_IndexSchemaType {
|
||
indexName?: string;
|
||
indexTableFields?: string[];
|
||
indexType?: "BTREE" | "HASH" | "FULLTEXT" | "SPATIAL" | "VECTOR";
|
||
vectorDistanceMetric?: "euclidean" | "cosine"; // VECTOR indexes only
|
||
comment?: string;
|
||
}
|
||
```
|
||
|
||
### Unique Constraint
|
||
|
||
```ts
|
||
interface BUN_MARIADB_UniqueConstraintSchemaType {
|
||
constraintName?: string;
|
||
constraintTableFields?: { value: string }[];
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## CLI Commands
|
||
|
||
CLI binary: `bun-mariadb`.
|
||
|
||
### `schema` — Sync database to schema
|
||
|
||
```bash
|
||
bunx bun-mariadb schema [options]
|
||
```
|
||
|
||
| Option | Description |
|
||
| ----------------- | ------------------------------------------------------------------------ |
|
||
| `-t`, `--typedef` | Also generate TypeScript type definitions after syncing |
|
||
| `--dry-run` | Print the planned schema changes without applying them |
|
||
| `-y`, `--yes` | Apply destructive changes (dropped tables / columns) without confirmation |
|
||
|
||
**Examples:**
|
||
|
||
```bash
|
||
# Sync schema only
|
||
bunx bun-mariadb schema
|
||
|
||
# Sync schema and regenerate types
|
||
bunx bun-mariadb schema --typedef
|
||
|
||
# Review planned changes first
|
||
bunx bun-mariadb schema --dry-run
|
||
```
|
||
|
||
Changes that discard data (dropping a table or column, or recreating a `VECTOR` column whose size changed) are listed and need confirmation before they run. Non-interactive runs (CI, scripts) refuse them unless `--yes` is passed.
|
||
|
||
What it does:
|
||
|
||
1. Ensures the internal `__db_schema_manager__` tracking table exists
|
||
2. Orders tables by foreign-key dependencies
|
||
3. Creates missing tables / surgically updates existing ones
|
||
4. Syncs indexes (including `VECTOR INDEX`)
|
||
5. Drops tables removed from the schema (skips tables still referenced by FKs)
|
||
6. Optionally writes TypeScript types and a live schema snapshot
|
||
|
||
### `typedef` — Generate TypeScript types only
|
||
|
||
```bash
|
||
bunx bun-mariadb typedef
|
||
```
|
||
|
||
Writes types to `typedef_file_path` from config.
|
||
|
||
### `backup` — SQL dump backup
|
||
|
||
```bash
|
||
bunx bun-mariadb backup
|
||
```
|
||
|
||
Runs `mariadb-dump` (or `mysqldump`) against the configured database and writes a timestamped `.sql` file into `db_backup_dir`. Old dumps are pruned to `max_backups`.
|
||
|
||
Requires `mariadb-dump` or `mysqldump` on `PATH`.
|
||
|
||
### `restore` — Restore from an SQL dump
|
||
|
||
```bash
|
||
bunx bun-mariadb restore
|
||
```
|
||
|
||
Interactive picker of available `.sql` backups (newest first). Imports the selected dump via `mariadb` / `mysql` into the configured database.
|
||
|
||
Requires `mariadb` or `mysql` on `PATH`.
|
||
|
||
### `export` — Portable SQL + schema archive
|
||
|
||
```bash
|
||
bunx bun-mariadb export [options]
|
||
```
|
||
|
||
| Option | Description |
|
||
| ---------------- | -------------------------------------------------------------------------- |
|
||
| `-o`, `--output` | Output archive path (`.tar.gz` or `.zip`). Defaults to `{db_dir}/.exports` |
|
||
| `-f`, `--format` | When `--output` is omitted: `tar.gz` (default) or `zip` |
|
||
|
||
Creates a portable archive containing:
|
||
|
||
1. `dump.sql` — full SQL dump (same options as `backup`)
|
||
2. `schema.ts` — the project’s TypeScript schema file from `db_dir`
|
||
|
||
Archives are written to `{db_dir}/.exports` (sibling of the backups directory). Old exports are pruned using `max_exports`.
|
||
|
||
**Examples:**
|
||
|
||
```bash
|
||
# Write {db_name}-{timestamp}.tar.gz into {db_dir}/.exports
|
||
bunx bun-mariadb export
|
||
|
||
# Explicit path / format
|
||
bunx bun-mariadb export -o ./my-app-export.tar.gz
|
||
bunx bun-mariadb export -f zip
|
||
bunx bun-mariadb export -o ./my-app-export.zip
|
||
```
|
||
|
||
Requires `mariadb-dump` or `mysqldump` on `PATH`. Zip output also requires `zip`.
|
||
|
||
### `import` — SQL or full export archive
|
||
|
||
```bash
|
||
bunx bun-mariadb import [file] [options]
|
||
```
|
||
|
||
| Option | Description |
|
||
| --------------- | --------------------------------------------------------------------- |
|
||
| `[file]` | Path to a `.sql` dump or export archive (`.tar.gz` / `.tar` / `.zip`) |
|
||
| `--sql-only` | Archive only: restore SQL, do not overwrite `schema.ts` |
|
||
| `--schema-only` | Archive only: write `schema.ts`, do not restore SQL |
|
||
| `-y, --yes` | Archive only: skip the confirmation before writing a `.ts`/`.js` schema |
|
||
|
||
Behavior:
|
||
|
||
- **`.sql` file** — restores SQL into the configured database (same as restore, but non-interactive)
|
||
- **Export archive** — restores `dump.sql` and writes `schema.ts` into `db_dir`. A `.ts`/`.js` schema is executed when loaded, so you are asked to confirm first (non-interactive runs require `--yes`). Only import archives from sources you trust.
|
||
- **No file argument** — interactive picker of archives in `{db_dir}/.exports`
|
||
|
||
**Examples:**
|
||
|
||
```bash
|
||
# Plain SQL dump
|
||
bunx bun-mariadb import ./db/.backups/my_app-1710000000000.sql
|
||
|
||
# Full export (SQL + schema.ts)
|
||
bunx bun-mariadb import ./db/exports/my_app-1710000000000.tar.gz
|
||
|
||
# Archive: SQL only / schema only
|
||
bunx bun-mariadb import ./export.tar.gz --sql-only
|
||
bunx bun-mariadb import ./export.tar.gz --schema-only
|
||
|
||
# Interactive picker (from {db_dir}/.exports)
|
||
bunx bun-mariadb import
|
||
```
|
||
|
||
Requires `mariadb` or `mysql` on `PATH`. Zip archives also require `unzip`.
|
||
|
||
### `admin` — Interactive admin tools
|
||
|
||
```bash
|
||
bunx bun-mariadb admin
|
||
```
|
||
|
||
Interactive menu to list tables / inspect data and run SQL.
|
||
|
||
---
|
||
|
||
## CRUD API
|
||
|
||
```ts
|
||
import BunMariaDB from "@moduletrace/bun-mariadb";
|
||
```
|
||
|
||
All methods return a `DBResponseObject<T>`:
|
||
|
||
```ts
|
||
{
|
||
success: boolean;
|
||
payload?: T[]; // rows (select / returning)
|
||
single_res?: T; // first row
|
||
count?: number;
|
||
insert_return?: {
|
||
count?: number;
|
||
last_insert_id?: number;
|
||
affected_rows?: number;
|
||
};
|
||
error?: any;
|
||
msg?: string;
|
||
debug?: any;
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### Select
|
||
|
||
```ts
|
||
BunMariaDB.select<T>({ table, query?, count?, targetId? })
|
||
```
|
||
|
||
| Parameter | Type | Description |
|
||
| ---------- | --------------------- | ------------------------------------------------------------ |
|
||
| `table` | `string` | Table name |
|
||
| `query` | `ServerQueryParam<T>` | Query/filter options (see [Query API](#query-api-reference)) |
|
||
| `count` | `boolean` | Return row count instead of rows |
|
||
| `targetId` | `string \| number` | Shorthand to filter by `id` |
|
||
|
||
**Examples:**
|
||
|
||
```ts
|
||
const res = await BunMariaDB.select({ table: "users" });
|
||
|
||
const res = await BunMariaDB.select({ table: "users", targetId: 42 });
|
||
|
||
const res = await BunMariaDB.select({
|
||
table: "users",
|
||
query: {
|
||
query: {
|
||
first_name: { value: "Ali", equality: "LIKE" },
|
||
},
|
||
},
|
||
});
|
||
|
||
const res = await BunMariaDB.select({ table: "users", count: true });
|
||
console.log(res.count);
|
||
|
||
const res = await BunMariaDB.select({
|
||
table: "users",
|
||
query: { limit: 10, page: 2 },
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
### Insert
|
||
|
||
```ts
|
||
BunMariaDB.insert<T>({ table, data, update_on_duplicate? })
|
||
```
|
||
|
||
| Parameter | Type | Description |
|
||
| --------------------- | --------- | ------------------------------------------------ |
|
||
| `table` | `string` | Table name |
|
||
| `data` | `T[]` | Array of row objects to insert |
|
||
| `update_on_duplicate` | `boolean` | Use `ON DUPLICATE KEY UPDATE` when unique exists |
|
||
|
||
`created_at` and `updated_at` are set automatically to `Date.now()`.
|
||
|
||
Fields with `html: true` are sanitized before insert.
|
||
|
||
```ts
|
||
const res = await BunMariaDB.insert({
|
||
table: "users",
|
||
data: [
|
||
{ first_name: "Alice", last_name: "Smith", email: "[email protected]" },
|
||
{ first_name: "Bob", last_name: "Jones", email: "[email protected]" },
|
||
],
|
||
});
|
||
|
||
console.log(res.insert_return?.last_insert_id);
|
||
```
|
||
|
||
---
|
||
|
||
### Update
|
||
|
||
```ts
|
||
BunMariaDB.update<T>({ table, data, query?, targetId? })
|
||
```
|
||
|
||
| Parameter | Type | Description |
|
||
| ---------- | --------------------- | --------------------------- |
|
||
| `table` | `string` | Table name |
|
||
| `data` | `Partial<T>` | Fields to update |
|
||
| `query` | `ServerQueryParam<T>` | WHERE clause conditions |
|
||
| `targetId` | `string \| number` | Shorthand to filter by `id` |
|
||
|
||
A WHERE clause is required. `updated_at` is set automatically. HTML fields are sanitized.
|
||
|
||
```ts
|
||
await BunMariaDB.update({
|
||
table: "users",
|
||
targetId: 1,
|
||
data: { first_name: "Alicia" },
|
||
});
|
||
|
||
await BunMariaDB.update({
|
||
table: "users",
|
||
data: { last_name: "Doe" },
|
||
query: {
|
||
query: {
|
||
email: { value: "[email protected]", equality: "EQUAL" },
|
||
},
|
||
},
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
### Delete
|
||
|
||
```ts
|
||
BunMariaDB.delete<T>({ table, query?, targetId? })
|
||
```
|
||
|
||
A WHERE clause is required.
|
||
|
||
```ts
|
||
await BunMariaDB.delete({ table: "users", targetId: 1 });
|
||
|
||
await BunMariaDB.delete({
|
||
table: "users",
|
||
query: {
|
||
query: {
|
||
first_name: { value: "Ben", equality: "LIKE" },
|
||
},
|
||
},
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
### Raw SQL
|
||
|
||
```ts
|
||
BunMariaDB.sql<T>({ sql, values?, config? })
|
||
```
|
||
|
||
`success` reflects whether the query ran; a `SELECT` with no rows returns `success: true` and an empty `payload`.
|
||
|
||
```ts
|
||
const res = await BunMariaDB.sql({
|
||
sql: "SELECT * FROM users WHERE id = ?",
|
||
values: [1],
|
||
});
|
||
console.log(res.payload);
|
||
```
|
||
|
||
---
|
||
|
||
## Queue Tables
|
||
|
||
Set `queue_table: true` on a table to use it as a job queue. Any number of workers can pull from it at the same time: claims use `SELECT ... FOR UPDATE SKIP LOCKED` (MariaDB 10.6+), so no job is handed to two workers and workers don't wait on each other.
|
||
|
||
```ts
|
||
{
|
||
tableName: "email_jobs",
|
||
queue_table: true,
|
||
fields: [
|
||
{ fieldName: "to", dataType: "VARCHAR", dataLength: 255 },
|
||
{ fieldName: "subject", dataType: "VARCHAR", dataLength: 255 },
|
||
],
|
||
}
|
||
```
|
||
|
||
Pass an options object instead of `true` to set the queue's defaults. Arguments to the `queue` methods override them.
|
||
|
||
```ts
|
||
queue_table: {
|
||
max_attempts: 5, // attempts before a job is marked `failed` (default 5)
|
||
lease_ms: 60_000, // how long a claim lasts (default 30s)
|
||
keep_completed: true, // `ack` marks jobs `done` instead of deleting them (default false)
|
||
}
|
||
```
|
||
|
||
`bunx bun-mariadb schema` adds these columns, plus the `idx_queue_claim` and `idx_queue_lease` indexes. They are managed by the library: declaring a field or index with one of these names on a queue table is an error.
|
||
|
||
| Field | Type | Description |
|
||
| ------------------------ | --------------------------------------- | --------------------------------------------------- |
|
||
| `queue_status` | `ENUM(...)` | Job state, see below |
|
||
| `queue_available_at` | `BIGINT` (ms) | The job can't be claimed before this time |
|
||
| `queue_attempts` | `INT` | How many times the job has been claimed |
|
||
| `queue_lease_expires_at` | `BIGINT` (ms) | When the current claim expires |
|
||
| `queue_lock_token` | `VARCHAR(64)` | Token of the current claim |
|
||
| `queue_last_error` | `TEXT` | Error from the last failed attempt |
|
||
|
||
Statuses:
|
||
|
||
| Status | Meaning |
|
||
| ------------ | ----------------------------------------------------------------- |
|
||
| `pending` | Waiting to be claimed. The only status `claim` picks up |
|
||
| `processing` | Claimed by a worker |
|
||
| `done` | Acknowledged and kept (`keep_completed`). Otherwise jobs are deleted |
|
||
| `failed` | Used up its attempts (dead letter) |
|
||
| `cancelled` | Taken out of the queue with `queue.cancel` |
|
||
|
||
The statuses are fixed because the queue's SQL depends on them. For your own job outcomes (`sent`, `bounced`, ...), add a field of your own and set it before calling `ack`.
|
||
|
||
### Worker loop
|
||
|
||
```ts
|
||
await BunMariaDB.queue.push({
|
||
table: "email_jobs",
|
||
data: [{ to: "[email protected]", subject: "Hi" }],
|
||
delay_ms: 5_000, // optional
|
||
});
|
||
|
||
const { payload: jobs } = await BunMariaDB.queue.claim({
|
||
table: "email_jobs",
|
||
limit: 10,
|
||
lease_ms: 60_000,
|
||
});
|
||
|
||
for (const job of jobs || []) {
|
||
try {
|
||
await sendEmail(job);
|
||
await BunMariaDB.queue.ack({
|
||
table: "email_jobs",
|
||
id: job.id,
|
||
lock_token: job.queue_lock_token,
|
||
});
|
||
} catch (e: any) {
|
||
await BunMariaDB.queue.fail({
|
||
table: "email_jobs",
|
||
id: job.id,
|
||
lock_token: job.queue_lock_token,
|
||
error: e.message,
|
||
retry_delay_ms: 30_000,
|
||
});
|
||
}
|
||
}
|
||
```
|
||
|
||
| Method | Description |
|
||
| ------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
|
||
| `queue.push({ table, data, delay_ms? })` | Insert jobs as `pending` |
|
||
| `queue.claim({ table, limit?, lease_ms?, max_attempts? })` | Claim up to `limit` (default 1) jobs, oldest first, for `lease_ms` (default 30s) |
|
||
| `queue.ack({ table, id, lock_token })` | Job done: deletes it, or marks it `done` with `keep_completed` |
|
||
| `queue.fail({ table, id, lock_token, error?, retry_delay_ms?, max_attempts? })` | Back to `pending` after `retry_delay_ms`, or `failed` once `max_attempts` (default 5) is reached |
|
||
| `queue.extend({ table, id, lock_token, lease_ms? })` | Extend the lease for a long-running job |
|
||
| `queue.cancel({ table, id })` | Take a `pending` job out of the queue. Claimed jobs can't be cancelled |
|
||
|
||
Delivery is **at least once**. If a worker crashes or runs past its lease, the next `claim` puts the job back in the queue (or marks it `failed` on its last attempt), and the next claim gets a new `queue_lock_token`. After that, `ack` / `fail` / `extend` with the old token return `success: false`. Make job handlers idempotent. Lease times use the database clock, so worker clock drift doesn't matter.
|
||
|
||
How regular CRUD treats queue tables:
|
||
|
||
- `insert` / `update` refuse to set `queue_status`, `queue_attempts`, `queue_lease_expires_at`, `queue_lock_token` or `queue_last_error`. Only the queue API changes those.
|
||
- `update` / `delete` skip jobs that are `processing`, so a worker's claimed job can't change or disappear while it runs.
|
||
- `select` is unchanged. Filter on `queue_status` to get the rows you want:
|
||
|
||
```ts
|
||
// Dead letters
|
||
await BunMariaDB.select({
|
||
table: "email_jobs",
|
||
query: { query: { queue_status: { value: "failed" } } },
|
||
});
|
||
|
||
// Everything still in the queue
|
||
await BunMariaDB.select({
|
||
table: "email_jobs",
|
||
count: true,
|
||
query: { query: { queue_status: { value: ["pending", "processing"], equality: "IN" } } },
|
||
});
|
||
```
|
||
|
||
- To retry a dead-lettered job, delete it and push it again, or reset it with raw `sql`.
|
||
- With `keep_completed`, `done` rows stay until you delete them, e.g. `delete` with a `queue_status` / `updated_at` filter.
|
||
|
||
---
|
||
|
||
## Query API Reference
|
||
|
||
`query` on `select` / `update` / `delete` accepts `ServerQueryParam<T>`:
|
||
|
||
```ts
|
||
type ServerQueryParam<T> = {
|
||
query?: { [key in keyof T]?: ServerQueryObject };
|
||
selectFields?: (keyof T | { fieldName: keyof T; alias?: string })[];
|
||
omitFields?: (keyof T)[];
|
||
limit?: number;
|
||
page?: number;
|
||
offset?: number;
|
||
order?:
|
||
| { field: keyof T; strategy: "ASC" | "DESC" }
|
||
| { field: keyof T; strategy: "ASC" | "DESC" }[];
|
||
searchOperator?: "AND" | "OR";
|
||
join?: ServerQueryParamsJoin[];
|
||
group?: keyof T | { field: keyof T; table?: string } | Array<...>;
|
||
countSubQueries?: ServerQueryParamsCount[];
|
||
fullTextSearch?: {
|
||
fields: (keyof T)[];
|
||
searchTerm: string;
|
||
scoreAlias: string;
|
||
};
|
||
};
|
||
```
|
||
|
||
### Equality Operators
|
||
|
||
| Equality | SQL Equivalent |
|
||
| ----------------------- | ----------------------------- |
|
||
| `EQUAL` (default) | `=` |
|
||
| `NOT EQUAL` | `!=` |
|
||
| `LIKE` | `LIKE '%value%'` |
|
||
| `LIKE_RAW` | `LIKE 'value'` |
|
||
| `LIKE_LOWER` | `LOWER(field) LIKE '%value%'` |
|
||
| `NOT LIKE` | `NOT LIKE '%value%'` |
|
||
| `GREATER THAN` | `>` |
|
||
| `GREATER THAN OR EQUAL` | `>=` |
|
||
| `LESS THAN` | `<` |
|
||
| `LESS THAN OR EQUAL` | `<=` |
|
||
| `IN` | `IN (...)` |
|
||
| `NOT IN` | `NOT IN (...)` |
|
||
| `BETWEEN` | `BETWEEN a AND b` |
|
||
| `IS NULL` | `IS NULL` |
|
||
| `IS NOT NULL` | `IS NOT NULL` |
|
||
| `REGEXP` | `REGEXP` |
|
||
| `FULLTEXT` | full-text match |
|
||
| `MATCH` | vector / specialized match |
|
||
|
||
```ts
|
||
const res = await BunMariaDB.select({
|
||
table: "users",
|
||
query: {
|
||
query: {
|
||
email: { equality: "IS NOT NULL" },
|
||
},
|
||
order: { field: "created_at", strategy: "DESC" },
|
||
limit: 20,
|
||
},
|
||
});
|
||
```
|
||
|
||
### JOIN
|
||
|
||
```ts
|
||
const res = await BunMariaDB.select({
|
||
table: "posts",
|
||
query: {
|
||
join: [
|
||
{
|
||
joinType: "LEFT JOIN",
|
||
tableName: "users",
|
||
match: { source: "user_id", target: "id" },
|
||
selectFields: ["first_name", "email"],
|
||
},
|
||
],
|
||
},
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## Vector Support
|
||
|
||
MariaDB native `VECTOR(n)` columns and `VECTOR INDEX` are supported (MariaDB 11.7+).
|
||
|
||
### Schema
|
||
|
||
```ts
|
||
{
|
||
tableName: "documents",
|
||
fields: [
|
||
{
|
||
fieldName: "embedding",
|
||
dataType: "VECTOR",
|
||
isVector: true,
|
||
vectorSize: 1536,
|
||
},
|
||
{
|
||
fieldName: "title",
|
||
dataType: "VARCHAR",
|
||
dataLength: 255,
|
||
},
|
||
],
|
||
indexes: [
|
||
{
|
||
indexName: "idx_documents_embedding",
|
||
indexType: "VECTOR",
|
||
indexTableFields: ["embedding"],
|
||
vectorDistanceMetric: "cosine", // or "euclidean" (default)
|
||
},
|
||
],
|
||
}
|
||
```
|
||
|
||
Notes:
|
||
|
||
- `isVector: true` or `dataType: "VECTOR"` maps to `VECTOR(n)` (`n` defaults to `1536`)
|
||
- Vector columns are created as `NOT NULL` (required for vector indexes)
|
||
- Dimension / storage changes trigger an automatic table recreate and data copy
|
||
- No special CLI flag is required
|
||
|
||
### Sync
|
||
|
||
```bash
|
||
bunx bun-mariadb schema
|
||
```
|
||
|
||
---
|
||
|
||
## HTML Sanitization
|
||
|
||
Fields with **explicit** `html: true` are sanitized on **insert** and **update** only.
|
||
|
||
```ts
|
||
{ fieldName: "body", dataType: "LONGTEXT", html: true }
|
||
```
|
||
|
||
Other fields (including text that happens to contain HTML tags) are **not** sanitized.
|
||
|
||
### Extend the allowlist
|
||
|
||
```ts
|
||
// bun-mariadb.config.ts
|
||
const config: BunMariaDBConfig = {
|
||
db_name: "my_app",
|
||
db_dir: "./db",
|
||
html_sanitize: {
|
||
allowed_tags: ["iframe", "video"],
|
||
allowed_attributes: {
|
||
iframe: ["src", "width", "height", "allow", "allowfullscreen"],
|
||
video: ["src", "controls", "width", "height"],
|
||
"*": ["data-id"],
|
||
},
|
||
},
|
||
};
|
||
```
|
||
|
||
Config values are **appended** to the built-in defaults (safe rich-text tags/attributes).
|
||
|
||
`style` and `id` attributes (and the `button` tag) are **not** allowed by default, since they enable CSS overlays and DOM clobbering. Add them through `allowed_attributes` / `allowed_tags` if you need them.
|
||
|
||
---
|
||
|
||
## TypeScript Type Generation
|
||
|
||
```bash
|
||
bunx bun-mariadb typedef
|
||
# or
|
||
bunx bun-mariadb schema --typedef
|
||
```
|
||
|
||
Generates:
|
||
|
||
- A `const` array of table names
|
||
- A type per table (`BUN_MARIADB_<DB>_<TABLE>`)
|
||
- A union of all table types
|
||
|
||
Example:
|
||
|
||
```ts
|
||
export const BunMariaDBTables = ["users"] as const;
|
||
|
||
export type BUN_MARIADB_MY_APP_USERS = {
|
||
/** The unique identifier of the record. */
|
||
id?: number;
|
||
/** The time when the record was created. (Unix Timestamp) */
|
||
created_at?: number;
|
||
/** The time when the record was updated. (Unix Timestamp) */
|
||
updated_at?: number;
|
||
first_name?: string;
|
||
last_name?: string;
|
||
email?: string;
|
||
};
|
||
|
||
export type BUN_MARIADB_MY_APP_ALL_TYPEDEFS = BUN_MARIADB_MY_APP_USERS;
|
||
```
|
||
|
||
```ts
|
||
import BunMariaDB from "@moduletrace/bun-mariadb";
|
||
import type { BUN_MARIADB_MY_APP_USERS } from "./db/types/db";
|
||
|
||
const res = await BunMariaDB.select<BUN_MARIADB_MY_APP_USERS>({
|
||
table: "users",
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## Default Fields
|
||
|
||
Every table automatically receives:
|
||
|
||
| Field | Type | Description |
|
||
| ------------ | -------------------------------------------- | -------------------------------------- |
|
||
| `id` | `BIGINT PRIMARY KEY AUTO_INCREMENT NOT NULL` | Unique row identifier |
|
||
| `created_at` | `BIGINT` | Unix timestamp set on insert |
|
||
| `updated_at` | `BIGINT` | Unix timestamp updated on every update |
|
||
|
||
You do not need to declare these in your schema.
|
||
|
||
---
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
bun-mariadb/
|
||
├── src/
|
||
│ ├── index.ts # Main export (BunMariaDB object)
|
||
│ ├── commands/
|
||
│ │ ├── index.ts # CLI entry point
|
||
│ │ ├── schema.ts # `schema` command
|
||
│ │ ├── typedef.ts # `typedef` command
|
||
│ │ ├── backup.ts # `backup` command
|
||
│ │ ├── restore.ts # `restore` command
|
||
│ │ └── admin/ # Interactive admin tools
|
||
│ ├── functions/
|
||
│ │ ├── init.ts # Config + schema + client loader
|
||
│ │ ├── grab-mariadb-client.ts # Bun SQL MariaDB client
|
||
│ │ └── live-schema.ts # Live schema snapshot I/O
|
||
│ ├── lib/
|
||
│ │ ├── db-handler.ts # Shared query runner
|
||
│ │ ├── mariadb/
|
||
│ │ │ ├── db-select.ts
|
||
│ │ │ ├── db-insert.ts
|
||
│ │ │ ├── db-update.ts
|
||
│ │ │ ├── db-delete.ts
|
||
│ │ │ ├── db-sql.ts
|
||
│ │ │ └── schema-to-typedef.ts
|
||
│ │ └── schema/ # Schema sync engine (modular)
|
||
│ │ ├── create-db-schema.ts
|
||
│ │ ├── create-table.ts
|
||
│ │ ├── update-table.ts
|
||
│ │ ├── sync-indexes.ts
|
||
│ │ └── ...
|
||
│ ├── types/
|
||
│ │ └── index.ts
|
||
│ └── utils/
|
||
│ ├── sql-generator.ts
|
||
│ ├── sql-insert-generator.ts
|
||
│ ├── sanitize-html-fields.ts
|
||
│ └── ...
|
||
└── tests/ # bun test suite
|
||
```
|
||
|
||
---
|
||
|
||
## Testing
|
||
|
||
```bash
|
||
bun run test
|
||
```
|
||
|
||
Starts a throwaway MariaDB container (Docker required, image `mariadb:11.8` by default), runs every test file against it, then removes the container. Extra arguments are passed to `bun test`, e.g. `bun run test tests/db.test.ts`.
|
||
|
||
| Variable | Description |
|
||
| ------------------------ | --------------------------------------------------------------------------- |
|
||
| `BUN_MARIADB_TEST_IMAGE` | Docker image to use (MariaDB 11.7+ for `VECTOR`) |
|
||
| `BUN_MARIADB_TEST_DB` | Skip Docker and use this existing database (with `BUN_MARIADB_SERVER_*`). The tests create and drop tables, so use a throwaway database. |
|
||
|
||
`bun run test:unit` (or plain `bun test`) runs without a database; the integration tests are skipped.
|
||
|
||
---
|
||
|
||
## License
|
||
|
||
MIT
|