Commit Graph
3 Commits
Author SHA1 Message Date
tbenandClaude Opus 5.5 ac3ebc43d4 Keep searchable text only in knowledgebases
Every bot file was chunked into bot_file_chunks and embedded into
bot_file_vectors, and uploads kept their extracted text in
media.text_content, which together took most of the database. Bot
files are no longer indexed: bots find them by name or path with
list_files and read them with read_file, and search_files, ingest_file
and the files modal's re-read button are gone. Attachments keep no
text; a reply reads a document from the file. Only knowledgebase
documents keep text.

The schema drops both tables and the ingest columns. At boot,
dropStoredFileText removes what older versions stored (file jobs,
upload text) and vacuums once much of the database is free. The
media:prune-text script it replaces is removed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-28 04:44:46 +01:00
tben 43e70de983 Delete a bot file's chunks with the file
ON DELETE CASCADE only fires when PRAGMA foreign_keys is on, and only
src/server.ts turns it on. A script or CLI process deleting a bot file
therefore left every bot_file_chunks row behind, and the next
`bun run db:schema` failed with

    Foreign key violations after schema sync: bot_file_chunks -> bot_files

deleteBotFile and purgeStorage now delete the chunks themselves, so the
cascade is no longer load-bearing and a maintenance script behaves like
the server.
2026-10-22 07:37:24 +01:00
tben d17a4a7bdd Add bots files handling logic 2026-10-21 17:56:09 +01:00