Add knowledgebases

Knowledgebases give bots RAG over documents, scoped like extensions: to
particular bots, to a workspace, or to every bot, with their own button
beside Tools in the composer. Documents are chunked and embedded by the
file-ingest worker with the install's embedding model, which is now an
install-wide setting rather than a per-bot one.

- Word search runs on an FTS5 index ranked by BM25, with a scored LIKE
  fallback for scripts written without spaces.
- Changing the embedding model re-indexes documents automatically.
- Ingest jobs wait without spending attempts while every embedding model
  rests, fail at once on errors retrying can't fix, and keep passages
  searchable by words when the model refuses.
- Settings > Knowledgebases shows each knowledgebase's and document's
  index state: spinner, warning, grey check (words only) or green check.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
2026-10-26 16:41:01 +01:00
co-authored by Claude Opus 5.5
parent 2259b6895f
commit e7b23b7454
111 changed files with 4573 additions and 148 deletions
+4
View File
@@ -11,6 +11,7 @@ import liveRelay from "./live-relay";
import enableForeignKeys from "./functions/backend/db/enable-foreign-keys";
import migrateBotTypes from "./functions/backend/db/bot-types/migrate-bot-types";
import clearImplicitSessionLinks from "./functions/backend/db/bot-sessions/clear-implicit-session-links";
import syncKnowledgebaseFts from "./functions/backend/db/knowledgebases/fts/sync-knowledgebase-fts";
import handleComputerProxy from "./functions/server/computers/handle-computer-proxy";
import handlePwaFile from "./functions/server/pwa/handle-pwa-file";
import handleInstallIconFile from "./functions/server/pwa/handle-install-icon-file";
@@ -38,6 +39,9 @@ await migrateBotTypes().catch((error) =>
await clearImplicitSessionLinks().catch((error) =>
logError({ title: "Could not clear implicit session links", error }).catch(() => {}),
);
await syncKnowledgebaseFts().catch((error) =>
logError({ title: "Could not sync the knowledgebase search index", error }).catch(() => {}),
);
await bunext.bunextInit();
/**