Moduletrace/bun-mariadb
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
9f2db66760
bun-mariadb/README.md
Benjamin Toby 225e3eba4c Updates
2026-07-14 09:34:10 +01:00

887 lines
26 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Bun MariaDB
`@moduletrace/bun-mariadb`
A schema-driven MariaDB manager for [Bun](https://bun.sh), featuring automatic schema synchronization, type-safe CRUD, native `VECTOR` support, HTML sanitization, and TypeScript type definition generation.
---
## Table of Contents
- [Features](#features)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Environment Variables](#environment-variables)
- [Quick Start](#quick-start)
- [Configuration](#configuration)
- [Schema Definition](#schema-definition)
- [CLI Commands](#cli-commands)
- [`schema`](#schema--sync-database-to-schema)
- [`typedef`](#typedef--generate-typescript-types-only)
- [`backup`](#backup--sql-dump-backup)
- [`restore`](#restore--restore-from-an-sql-dump)
- [`export`](#export--portable-sql--schema-archive)
- [`import`](#import--sql-or-full-export-archive)
- [`admin`](#admin--interactive-admin-tools)
- [CRUD API](#crud-api)
- [Select](#select)
- [Insert](#insert)
- [Update](#update)
- [Delete](#delete)
- [Raw SQL](#raw-sql)
- [Query API Reference](#query-api-reference)
- [Vector Support](#vector-support)
- [HTML Sanitization](#html-sanitization)
- [TypeScript Type Generation](#typescript-type-generation)
- [Default Fields](#default-fields)
- [Project Structure](#project-structure)
---
## Features
- **Schema-first design** — define your database in TypeScript; the library syncs MariaDB to match
- **Automatic migrations** — adds/modifies/drops columns, manages indexes & foreign keys, drops removed tables
- **Type-safe CRUD** — generic `select`, `insert`, `update`, `delete`, and raw `sql`
- **Rich query DSL** — filtering, ordering, pagination, joins, grouping, full-text search
- **Native VECTOR columns** — MariaDB `VECTOR(n)` types and `VECTOR INDEX` support
- **HTML sanitization** — fields with `html: true` are sanitized on insert/update
- **TypeScript codegen** — generate `.ts` type definitions from your schema
- **Zero-config defaults** — `id`, `created_at`, and `updated_at` are added to every table automatically
---
## Prerequisites
- [Bun](https://bun.sh) installed
- A running MariaDB server (11.7+ recommended for native `VECTOR` support)
---
## Installation
### Via the private npm registry
`@moduletrace/bun-mariadb` is published to a private Gitea npm registry. Configure the `@moduletrace` scope first:
```ini
# .npmrc
@moduletrace:registry=https://git.tben.me/api/packages/moduletrace/npm/
```
```bash
bun add @moduletrace/bun-mariadb
```
### Via Git
```bash
bun add github:moduletrace/bun-mariadb
```
---
## Environment Variables
Connection settings are read from the environment (not the config file):
| Variable | Required | Description |
| --------------------------------- | -------- | ------------------------------------ |
| `BUN_MARIADB_SERVER_HOST` | Yes | MariaDB host |
| `BUN_MARIADB_SERVER_USERNAME` | Yes | Database user |
| `BUN_MARIADB_SERVER_PASSWORD` | Yes | Database password |
| `BUN_MARIADB_SERVER_PORT` | No | Port (server default if omitted) |
| `BUN_MARIADB_SERVER_SSL_KEY_PATH` | No | Optional SSL key path (legacy/env) |
Example `.env`:
```env
BUN_MARIADB_SERVER_HOST=127.0.0.1
BUN_MARIADB_SERVER_PORT=3306
BUN_MARIADB_SERVER_USERNAME=root
BUN_MARIADB_SERVER_PASSWORD=secret
```
---
## Quick Start
### 1. Create the config file
Create `bun-mariadb.config.ts` at your project root:
```ts
import type { BunMariaDBConfig } from "@moduletrace/bun-mariadb";
const config: BunMariaDBConfig = {
db_name: "my_app",
db_dir: "./db",
typedef_file_path: "./db/types/db.ts",
// ssl_ca: "./db/ca-cert.pem",
};
export default config;
```
### 2. Define your schema
Create `./db/schema.ts` (filename is fixed as `schema.ts` inside `db_dir`):
```ts
import type { BUN_MARIADB_DatabaseSchemaType } from "@moduletrace/bun-mariadb";
const schema: BUN_MARIADB_DatabaseSchemaType = {
tables: [
{
tableName: "users",
fields: [
{ fieldName: "first_name", dataType: "VARCHAR", integerLength: 255 },
{ fieldName: "last_name", dataType: "VARCHAR", integerLength: 255 },
{ fieldName: "email", dataType: "VARCHAR", integerLength: 255, unique: true },
{ fieldName: "bio", dataType: "LONGTEXT", html: true },
],
},
],
};
export default schema;
```
### 3. Sync the schema to MariaDB
```bash
bunx bun-mariadb schema
```
This creates/updates all tables, indexes, and foreign keys to match your schema.
### 4. Use the CRUD API
```ts
import BunMariaDB from "@moduletrace/bun-mariadb";
// Insert
await BunMariaDB.insert({
table: "users",
data: [{ first_name: "Alice", email: "alice@example.com" }],
});
// Select
const result = await BunMariaDB.select({ table: "users" });
console.log(result.payload);
// Update
await BunMariaDB.update({
table: "users",
targetId: 1,
data: { first_name: "Alicia" },
});
// Delete
await BunMariaDB.delete({ table: "users", targetId: 1 });
```
---
## Configuration
The config file must be named `bun-mariadb.config.ts` and placed at the project root.
| Field | Type | Required | Description |
| ------------------- | -------- | -------- | --------------------------------------------------------------------------- |
| `db_name` | `string` | Yes | MariaDB database name |
| `db_dir` | `string` | Yes | Directory for schema, types, and local artifacts (relative to project root) |
| `db_backup_dir` | `string` | No | Backup directory name, relative to `db_dir` (default: `.backups`) |
| `max_backups` | `number` | No | Max backup files to keep (default: `10`) |
| `max_exports` | `number` | No | Max export archives to keep (default: `10`) |
| `typedef_file_path` | `string` | No | Output path for generated TypeScript types (relative to project root) |
| `db_config` | `object` | No | Extra options passed to Bun's `SQL` MariaDB adapter |
| `charset` | `string` | No | Database charset (default: `utf8mb4`) |
| `connection_timeout`| `number` | No | Connection timeout in ms (default: `10000`) |
| `ssl_ca` | `string` | No | Path to SSL CA certificate (relative to project root) |
| `html_sanitize` | `object` | No | Extra HTML sanitizer allowlists (see [HTML Sanitization](#html-sanitization)) |
Schema file path is always: `{db_dir}/schema.ts`.
---
## Schema Definition
### Database Schema
```ts
interface BUN_MARIADB_DatabaseSchemaType {
tables: BUN_MARIADB_TableSchemaType[];
collation?: "utf8mb4_bin" | "utf8mb4_unicode_520_ci";
}
```
### Table Schema
```ts
interface BUN_MARIADB_TableSchemaType {
tableName: string;
tableDescription?: string;
fields: BUN_MARIADB_FieldSchemaType[];
indexes?: BUN_MARIADB_IndexSchemaType[];
uniqueConstraints?: BUN_MARIADB_UniqueConstraintSchemaType[];
parentTableName?: string; // inherit / merge fields from another table
tableNameOld?: string; // rename: old name triggers ALTER TABLE RENAME
collation?: "utf8mb4_bin" | "utf8mb4_unicode_520_ci";
isVector?: boolean; // mark as vector-oriented table
}
```
### Field Schema
```ts
type BUN_MARIADB_FieldSchemaType = {
fieldName?: string;
dataType:
| "CHAR" | "VARCHAR" | "TEXT" | "TINYTEXT" | "MEDIUMTEXT" | "LONGTEXT"
| "TINYINT" | "SMALLINT" | "MEDIUMINT" | "INT" | "BIGINT"
| "FLOAT" | "DOUBLE" | "DECIMAL"
| "BINARY" | "VARBINARY" | "BLOB" | "TINYBLOB" | "MEDIUMBLOB" | "LONGBLOB"
| "DATE" | "TIME" | "DATETIME" | "TIMESTAMP" | "YEAR"
| "BOOLEAN" | "UUID" | "JSON" | "INET6" | "ENUM" | "SET" | "VECTOR";
primaryKey?: boolean;
autoIncrement?: boolean;
notNullValue?: boolean;
unique?: boolean;
defaultValue?: string | number;
defaultValueLiteral?: string; // raw SQL, e.g. "CURRENT_TIMESTAMP"
onUpdate?: string;
onUpdateLiteral?: string;
foreignKey?: BUN_MARIADB_ForeignKeyType;
integerLength?: string | number; // e.g. VARCHAR length
decimals?: string | number; // DECIMAL scale
options?: (string | number)[]; // ENUM / SET values
isVector?: boolean; // native VECTOR column
vectorSize?: number; // dimensions (default: 1536)
// Content mode flags (editor metadata); only `html: true` affects runtime:
html?: boolean;
markdown?: boolean;
plain?: boolean;
// ...
};
```
### Foreign Key
```ts
interface BUN_MARIADB_ForeignKeyType {
foreignKeyName?: string;
destinationTableName?: string;
destinationTableColumnName?: string;
cascadeDelete?: boolean;
cascadeUpdate?: boolean;
}
```
### Index
```ts
interface BUN_MARIADB_IndexSchemaType {
indexName?: string;
indexTableFields?: string[];
indexType?: "BTREE" | "HASH" | "FULLTEXT" | "SPATIAL" | "VECTOR";
vectorDistanceMetric?: "euclidean" | "cosine"; // VECTOR indexes only
comment?: string;
}
```
### Unique Constraint
```ts
interface BUN_MARIADB_UniqueConstraintSchemaType {
constraintName?: string;
constraintTableFields?: { value: string }[];
}
```
---
## CLI Commands
CLI binary: `bun-mariadb`.
### `schema` — Sync database to schema
```bash
bunx bun-mariadb schema [options]
```
| Option | Description |
| ----------------- | ------------------------------------------------------- |
| `-t`, `--typedef` | Also generate TypeScript type definitions after syncing |
**Examples:**
```bash
# Sync schema only
bunx bun-mariadb schema
# Sync schema and regenerate types
bunx bun-mariadb schema --typedef
```
What it does:
1. Ensures the internal `__db_schema_manager__` tracking table exists
2. Orders tables by foreign-key dependencies
3. Creates missing tables / surgically updates existing ones
4. Syncs indexes (including `VECTOR INDEX`)
5. Drops tables removed from the schema (skips tables still referenced by FKs)
6. Optionally writes TypeScript types and a live schema snapshot
### `typedef` — Generate TypeScript types only
```bash
bunx bun-mariadb typedef
```
Writes types to `typedef_file_path` from config.
### `backup` — SQL dump backup
```bash
bunx bun-mariadb backup
```
Runs `mariadb-dump` (or `mysqldump`) against the configured database and writes a timestamped `.sql` file into `db_backup_dir`. Old dumps are pruned to `max_backups`.
Requires `mariadb-dump` or `mysqldump` on `PATH`.
### `restore` — Restore from an SQL dump
```bash
bunx bun-mariadb restore
```
Interactive picker of available `.sql` backups (newest first). Imports the selected dump via `mariadb` / `mysql` into the configured database.
Requires `mariadb` or `mysql` on `PATH`.
### `export` — Portable SQL + schema archive
```bash
bunx bun-mariadb export [options]
```
| Option | Description |
| ------------------- | --------------------------------------------------------------------------- |
| `-o`, `--output` | Output archive path (`.tar.gz` or `.zip`). Defaults to `{db_dir}/.exports` |
| `-f`, `--format` | When `--output` is omitted: `tar.gz` (default) or `zip` |
Creates a portable archive containing:
1. `dump.sql` — full SQL dump (same options as `backup`)
2. `schema.ts` — the project’s TypeScript schema file from `db_dir`
Archives are written to `{db_dir}/.exports` (sibling of the backups directory). Old exports are pruned using `max_exports`.
**Examples:**
```bash
# Write {db_name}-{timestamp}.tar.gz into {db_dir}/.exports
bunx bun-mariadb export
# Explicit path / format
bunx bun-mariadb export -o ./my-app-export.tar.gz
bunx bun-mariadb export -f zip
bunx bun-mariadb export -o ./my-app-export.zip
```
Requires `mariadb-dump` or `mysqldump` on `PATH`. Zip output also requires `zip`.
### `import` — SQL or full export archive
```bash
bunx bun-mariadb import [file] [options]
```
| Option | Description |
| ---------------- | --------------------------------------------------------------------------- |
| `[file]` | Path to a `.sql` dump or export archive (`.tar.gz` / `.tar` / `.zip`) |
| `--sql-only` | Archive only: restore SQL, do not overwrite `schema.ts` |
| `--schema-only` | Archive only: write `schema.ts`, do not restore SQL |
Behavior:
- **`.sql` file** — restores SQL into the configured database (same as restore, but non-interactive)
- **Export archive** — restores `dump.sql` and writes `schema.ts` into `db_dir`
- **No file argument** — interactive picker of archives in `{db_dir}/.exports`
**Examples:**
```bash
# Plain SQL dump
bunx bun-mariadb import ./db/.backups/my_app-1710000000000.sql
# Full export (SQL + schema.ts)
bunx bun-mariadb import ./db/exports/my_app-1710000000000.tar.gz
# Archive: SQL only / schema only
bunx bun-mariadb import ./export.tar.gz --sql-only
bunx bun-mariadb import ./export.tar.gz --schema-only
# Interactive picker (from {db_dir}/.exports)
bunx bun-mariadb import
```
Requires `mariadb` or `mysql` on `PATH`. Zip archives also require `unzip`.
### `admin` — Interactive admin tools
```bash
bunx bun-mariadb admin
```
Interactive menu to list tables / inspect data and run SQL.
---
## CRUD API
```ts
import BunMariaDB from "@moduletrace/bun-mariadb";
```
All methods return a `DBResponseObject<T>`:
```ts
{
success: boolean;
payload?: T[]; // rows (select / returning)
single_res?: T; // first row
count?: number;
insert_return?: {
count?: number;
last_insert_id?: number;
affected_rows?: number;
};
error?: any;
msg?: string;
debug?: any;
}
```
---
### Select
```ts
BunMariaDB.select<T>({ table, query?, count?, targetId? })
```
| Parameter | Type | Description |
| ---------- | --------------------- | ------------------------------------------------------------ |
| `table` | `string` | Table name |
| `query` | `ServerQueryParam<T>` | Query/filter options (see [Query API](#query-api-reference)) |
| `count` | `boolean` | Return row count instead of rows |
| `targetId` | `string \| number` | Shorthand to filter by `id` |
**Examples:**
```ts
const res = await BunMariaDB.select({ table: "users" });
const res = await BunMariaDB.select({ table: "users", targetId: 42 });
const res = await BunMariaDB.select({
table: "users",
query: {
query: {
first_name: { value: "Ali", equality: "LIKE" },
},
},
});
const res = await BunMariaDB.select({ table: "users", count: true });
console.log(res.count);
const res = await BunMariaDB.select({
table: "users",
query: { limit: 10, page: 2 },
});
```
---
### Insert
```ts
BunMariaDB.insert<T>({ table, data, update_on_duplicate? })
```
| Parameter | Type | Description |
| --------------------- | --------- | ------------------------------------------------ |
| `table` | `string` | Table name |
| `data` | `T[]` | Array of row objects to insert |
| `update_on_duplicate` | `boolean` | Use `ON DUPLICATE KEY UPDATE` when unique exists |
`created_at` and `updated_at` are set automatically to `Date.now()`.
Fields with `html: true` are sanitized before insert.
```ts
const res = await BunMariaDB.insert({
table: "users",
data: [
{ first_name: "Alice", last_name: "Smith", email: "alice@example.com" },
{ first_name: "Bob", last_name: "Jones", email: "bob@example.com" },
],
});
console.log(res.insert_return?.last_insert_id);
```
---
### Update
```ts
BunMariaDB.update<T>({ table, data, query?, targetId? })
```
| Parameter | Type | Description |
| ---------- | --------------------- | --------------------------- |
| `table` | `string` | Table name |
| `data` | `Partial<T>` | Fields to update |
| `query` | `ServerQueryParam<T>` | WHERE clause conditions |
| `targetId` | `string \| number` | Shorthand to filter by `id` |
A WHERE clause is required. `updated_at` is set automatically. HTML fields are sanitized.
```ts
await BunMariaDB.update({
table: "users",
targetId: 1,
data: { first_name: "Alicia" },
});
await BunMariaDB.update({
table: "users",
data: { last_name: "Doe" },
query: {
query: {
email: { value: "alice@example.com", equality: "EQUAL" },
},
},
});
```
---
### Delete
```ts
BunMariaDB.delete<T>({ table, query?, targetId? })
```
A WHERE clause is required.
```ts
await BunMariaDB.delete({ table: "users", targetId: 1 });
await BunMariaDB.delete({
table: "users",
query: {
query: {
first_name: { value: "Ben", equality: "LIKE" },
},
},
});
```
---
### Raw SQL
```ts
BunMariaDB.sql<T>({ sql, values? })
```
```ts
const res = await BunMariaDB.sql({
sql: "SELECT * FROM users WHERE id = ?",
values: [1],
});
console.log(res.payload);
```
---
## Query API Reference
`query` on `select` / `update` / `delete` accepts `ServerQueryParam<T>`:
```ts
type ServerQueryParam<T> = {
query?: { [key in keyof T]?: ServerQueryObject };
selectFields?: (keyof T | { fieldName: keyof T; alias?: string })[];
omitFields?: (keyof T)[];
limit?: number;
page?: number;
offset?: number;
order?:
| { field: keyof T; strategy: "ASC" | "DESC" }
| { field: keyof T; strategy: "ASC" | "DESC" }[];
searchOperator?: "AND" | "OR";
join?: ServerQueryParamsJoin[];
group?: keyof T | { field: keyof T; table?: string } | Array<...>;
countSubQueries?: ServerQueryParamsCount[];
fullTextSearch?: {
fields: (keyof T)[];
searchTerm: string;
scoreAlias: string;
};
};
```
### Equality Operators
| Equality | SQL Equivalent |
| ----------------------- | ------------------------------------------------------ |
| `EQUAL` (default) | `=` |
| `NOT EQUAL` | `!=` |
| `LIKE` | `LIKE '%value%'` |
| `LIKE_RAW` | `LIKE 'value'` |
| `LIKE_LOWER` | `LOWER(field) LIKE '%value%'` |
| `NOT LIKE` | `NOT LIKE '%value%'` |
| `GREATER THAN` | `>` |
| `GREATER THAN OR EQUAL` | `>=` |
| `LESS THAN` | `<` |
| `LESS THAN OR EQUAL` | `<=` |
| `IN` | `IN (...)` |
| `NOT IN` | `NOT IN (...)` |
| `BETWEEN` | `BETWEEN a AND b` |
| `IS NULL` | `IS NULL` |
| `IS NOT NULL` | `IS NOT NULL` |
| `REGEXP` | `REGEXP` |
| `FULLTEXT` | full-text match |
| `MATCH` | vector / specialized match |
```ts
const res = await BunMariaDB.select({
table: "users",
query: {
query: {
email: { equality: "IS NOT NULL" },
},
order: { field: "created_at", strategy: "DESC" },
limit: 20,
},
});
```
### JOIN
```ts
const res = await BunMariaDB.select({
table: "posts",
query: {
join: [
{
joinType: "LEFT JOIN",
tableName: "users",
match: { source: "user_id", target: "id" },
selectFields: ["first_name", "email"],
},
],
},
});
```
---
## Vector Support
MariaDB native `VECTOR(n)` columns and `VECTOR INDEX` are supported (MariaDB 11.7+).
### Schema
```ts
{
tableName: "documents",
isVector: true,
fields: [
{
fieldName: "embedding",
dataType: "VECTOR",
isVector: true,
vectorSize: 1536,
},
{
fieldName: "title",
dataType: "VARCHAR",
integerLength: 255,
},
],
indexes: [
{
indexName: "idx_documents_embedding",
indexType: "VECTOR",
indexTableFields: ["embedding"],
vectorDistanceMetric: "cosine", // or "euclidean" (default)
},
],
}
```
Notes:
- `isVector: true` or `dataType: "VECTOR"` maps to `VECTOR(n)` (`n` defaults to `1536`)
- Vector columns are created as `NOT NULL` (required for vector indexes)
- Dimension / storage changes trigger an automatic table recreate and data copy
- No special CLI flag is required
### Sync
```bash
bunx bun-mariadb schema
```
---
## HTML Sanitization
Fields with **explicit** `html: true` are sanitized on **insert** and **update** only.
```ts
{ fieldName: "body", dataType: "LONGTEXT", html: true }
```
Other fields (including text that happens to contain HTML tags) are **not** sanitized.
### Extend the allowlist
```ts
// bun-mariadb.config.ts
const config: BunMariaDBConfig = {
db_name: "my_app",
db_dir: "./db",
html_sanitize: {
allowed_tags: ["iframe", "video"],
allowed_attributes: {
iframe: ["src", "width", "height", "allow", "allowfullscreen"],
video: ["src", "controls", "width", "height"],
"*": ["data-id"],
},
},
};
```
Config values are **appended** to the built-in defaults (safe rich-text tags/attributes).
---
## TypeScript Type Generation
```bash
bunx bun-mariadb typedef
# or
bunx bun-mariadb schema --typedef
```
Generates:
- A `const` array of table names
- A type per table (`BUN_MARIADB_<DB>_<TABLE>`)
- A union of all table types
Example:
```ts
export const BunMariaDBTables = ["users"] as const;
export type BUN_MARIADB_MY_APP_USERS = {
/** The unique identifier of the record. */
id?: number;
/** The time when the record was created. (Unix Timestamp) */
created_at?: number;
/** The time when the record was updated. (Unix Timestamp) */
updated_at?: number;
first_name?: string;
last_name?: string;
email?: string;
};
export type BUN_MARIADB_MY_APP_ALL_TYPEDEFS = BUN_MARIADB_MY_APP_USERS;
```
```ts
import BunMariaDB from "@moduletrace/bun-mariadb";
import type { BUN_MARIADB_MY_APP_USERS } from "./db/types/db";
const res = await BunMariaDB.select<BUN_MARIADB_MY_APP_USERS>({
table: "users",
});
```
---
## Default Fields
Every table automatically receives:
| Field | Type | Description |
| ------------ | ------------------------------- | -------------------------------------- |
| `id` | `BIGINT PRIMARY KEY AUTO_INCREMENT NOT NULL` | Unique row identifier |
| `created_at` | `BIGINT` | Unix timestamp set on insert |
| `updated_at` | `BIGINT` | Unix timestamp updated on every update |
You do not need to declare these in your schema.
---
## Project Structure
```
bun-mariadb/
├── src/
│ ├── index.ts # Main export (BunMariaDB object)
│ ├── commands/
│ │ ├── index.ts # CLI entry point
│ │ ├── schema.ts # `schema` command
│ │ ├── typedef.ts # `typedef` command
│ │ ├── backup.ts # `backup` command
│ │ ├── restore.ts # `restore` command
│ │ └── admin/ # Interactive admin tools
│ ├── functions/
│ │ ├── init.ts # Config + schema + client loader
│ │ ├── grab-mariadb-client.ts # Bun SQL MariaDB client
│ │ └── live-schema.ts # Live schema snapshot I/O
│ ├── lib/
│ │ ├── db-handler.ts # Shared query runner
│ │ ├── mariadb/
│ │ │ ├── db-select.ts
│ │ │ ├── db-insert.ts
│ │ │ ├── db-update.ts
│ │ │ ├── db-delete.ts
│ │ │ ├── db-sql.ts
│ │ │ └── schema-to-typedef.ts
│ │ └── schema/ # Schema sync engine (modular)
│ │ ├── create-db-schema.ts
│ │ ├── create-table.ts
│ │ ├── update-table.ts
│ │ ├── recreate-table.ts
│ │ ├── sync-indexes.ts
│ │ └── ...
│ ├── types/
│ │ └── index.ts
│ └── utils/
│ ├── sql-generator.ts
│ ├── sql-insert-generator.ts
│ ├── sanitize-html-fields.ts
│ └── ...
└── test/
└── coderank/ # Example project
```
---
## License
MIT
Reference in New Issue View Git Blame Copy Permalink