Moduletrace/cobble
2
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 32 Wiki Activity

Keep searchable text only in knowledgebases

Browse Source 
Every bot file was chunked into bot_file_chunks and embedded into
bot_file_vectors, and uploads kept their extracted text in
media.text_content, which together took most of the database. Bot
files are no longer indexed: bots find them by name or path with
list_files and read them with read_file, and search_files, ingest_file
and the files modal's re-read button are gone. Attachments keep no
text; a reply reads a document from the file. Only knowledgebase
documents keep text.

The schema drops both tables and the ingest columns. At boot,
dropStoredFileText removes what older versions stored (file jobs,
upload text) and vacuums once much of the database is free. The
media:prune-text script it replaces is removed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
tben
2026-10-28 04:44:46 +01:00
co-authored by Claude Opus 5.5
parent 2896e76256
commit ac3ebc43d4
55 changed files with 271 additions and 1712 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/harness/tools/storage/write-file-tool.ts
+2 -5
View File
@@ -1,7 +1,6 @@
import { z } from "zod";
import { AppData } from "@/src/data/app-data";
import type { BotFilePurpose } from "@/src/dict/bot-file-dict";
import enqueueFileIngest from "@/src/file-ingest/enqueue-file-ingest";
import recordBotFile from "@/src/functions/backend/db/bot-files/record-bot-file";
import type { HarnessTool } from "../../types";
import {
@@ -23,7 +22,7 @@ type Args = {
/**
* Writes a text file (markdown, CSV, JSON, code, plain text) into one of
* the bot's directories, making folders as needed. The file stays for
* later runs and becomes searchable.
* later runs.
*/
export default function writeFileTool(context: StorageToolContext): HarnessTool<Args> {
const own = context.scopes[0];
@@ -32,7 +31,7 @@ export default function writeFileTool(context: StorageToolContext): HarnessTool<
name: "write_file",
description: `Save a text file for later: markdown, CSV, JSON, code or plain text. Give it a path like "reports/q3.md"; folders are made as needed. Files belong to ${
own?.scope == "workspace" ? "the workspace you're working in" : "you"
} and stay between runs: write a file before telling the user something is saved, and search_files to find one you wrote before. It is readable and searchable at once.${
} and stay between runs: write a file before telling the user something is saved, and list_files (by name or path) to find one you wrote before. It is readable at once.${
context.scopes.length > 1
? ` "scope" picks a directory: ${context.scopes.map((s) => `${s.scope}:${s.owner_id}`).join(", ")}.`
: ""
@@ -74,8 +73,6 @@ export default function writeFileTool(context: StorageToolContext): HarnessTool<
user_id: context.user_id,
});
await enqueueFileIngest({ file_id: file.id! });
if (args.attach && file.media_id) {
context.onAttach?.({ media_id: Number(file.media_id), name: file.name || "" });
}