Add knowledgebases

Knowledgebases give bots RAG over documents, scoped like extensions: to
particular bots, to a workspace, or to every bot, with their own button
beside Tools in the composer. Documents are chunked and embedded by the
file-ingest worker with the install's embedding model, which is now an
install-wide setting rather than a per-bot one.

- Word search runs on an FTS5 index ranked by BM25, with a scored LIKE
  fallback for scripts written without spaces.
- Changing the embedding model re-indexes documents automatically.
- Ingest jobs wait without spending attempts while every embedding model
  rests, fail at once on errors retrying can't fix, and keep passages
  searchable by words when the model refuses.
- Settings > Knowledgebases shows each knowledgebase's and document's
  index state: spinner, warning, grey check (words only) or green check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-26 16:41:01 +01:00
co-authored by Claude Opus 5.5
parent 2259b6895f
commit e7b23b7454
111 changed files with 4573 additions and 148 deletions
@@ -4,7 +4,7 @@ import type {
BUN_SQLITE_COBBLE_HARNESS_SETTINGS,
BUN_SQLITE_COBBLE_WORKSPACE_HARNESS_SETTINGS,
} from "@/db/types/db";
import type { HarnessKind } from "@/src/dict/harness-dict";
import { isInstallWideHarness, type HarnessKind } from "@/src/dict/harness-dict";
import parseJobJson from "@/src/bots-queue/parse-job-json";
import grabDefaultHarnessProviderId from "./grab-default-harness-provider-id";
@@ -51,14 +51,18 @@ type SettingRow = Pick<
/**
* A harness's setting for a call: the workspace's, else the bot's, else
* the default. With no row it is automatic on the default provider
* (OpenRouter), on, with fallback. Text is always on.
* (OpenRouter), on, with fallback. Text is always on. An install-wide
* harness (embeddings) always takes the default: rows a bot or workspace
* saved before it was one are ignored.
*/
export default async function grabHarnessSetting({
harness,
bot_id,
workspace_id,
}: Params): Promise<HarnessSetting> {
const workspace_row = workspace_id
const install_wide = isInstallWideHarness({ harness });
const workspace_row = workspace_id && !install_wide
? await grabWorkspaceOverride({ harness, workspace_id })
: undefined;
@@ -66,7 +70,7 @@ export default async function grabHarnessSetting({
return toSetting({ harness, row: workspace_row, source: "workspace" });
}
const bot_row = bot_id
const bot_row = bot_id && !install_wide
? await grabBotOverride({ harness, bot_id })
: undefined;