Add knowledgebases

Knowledgebases give bots RAG over documents, scoped like extensions: to
particular bots, to a workspace, or to every bot, with their own button
beside Tools in the composer. Documents are chunked and embedded by the
file-ingest worker with the install's embedding model, which is now an
install-wide setting rather than a per-bot one.

- Word search runs on an FTS5 index ranked by BM25, with a scored LIKE
  fallback for scripts written without spaces.
- Changing the embedding model re-indexes documents automatically.
- Ingest jobs wait without spending attempts while every embedding model
  rests, fail at once on errors retrying can't fix, and keep passages
  searchable by words when the model refuses.
- Settings > Knowledgebases shows each knowledgebase's and document's
  index state: spinner, warning, grey check (words only) or green check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

src/functions/backend/db/knowledgebases/grab-knowledgebase-document-text.ts

@@ -0,0 +1,36 @@
+import BunSQLite from "@moduletrace/bun-sqlite";
+import extractBotFileText from "../bot-files/extract-bot-file-text";
+
+type Params = {
+    document: {
+        source?: string | null;
+        content?: string | null;
+        media_id?: number | null;
+        name?: string | null;
+        mime_type?: string | null;
+    };
+};
+
+/**
+ * A knowledgebase document's whole text: what was written in, or what its
+ * file holds (the text pulled out at upload, else unpacked now).
+ */
+export default async function grabKnowledgebaseDocumentText({ document }: Params) {
+    if (document.source == "text") return document.content || "";
+
+    if (!document.media_id) return "";
+
+    const media = await BunSQLite.sql<{ media_write_path?: string; text_content?: string }>({
+        sql: `SELECT media_write_path, text_content FROM media WHERE id = ? LIMIT 1`,
+        values: [document.media_id],
+    }).then((res) => res.payload?.[0]);
+
+    return (
+        (await extractBotFileText({
+            write_path: media?.media_write_path,
+            text_content: media?.text_content,
+            name: document.name,
+            mime_type: document.mime_type,
+        })) || ""
+    );
+}