Stream computer files instead of reading them through the shell

import_file sent a computer's bytes back with base64 over /v1/shell/exec,
whose stdout is capped at 30,047 characters. Every import was therefore
cut at 22,528 bytes and saved anyway, with the tool reporting success: a
1.12 MiB archive became a 22 KB file that would not open, and a crawler's
crawl.json lost half its pages.

The bytes now come from the sandbox's streaming GET /v1/file/download.
The size is read first so nothing over BotFileImportMaxBytes is
transferred, the download asks for change_policy=abort so a file that
changes mid-send is refused rather than half-saved, and the received
length is checked against the size read, so a cut-short transfer fails
loudly. Folders go the same way, as a gzipped tar unpacked into one row
per file.

Bots also name a path as [path:volume:owner:path] now, so what a bot
leaves on a computer is a link into the files browser instead of text.
This commit is contained in:
2026-10-22 07:37:19 +01:00
parent 956942bba9
commit f1b7ba535f
7 changed files with 658 additions and 6 deletions
@@ -0,0 +1,36 @@
import Link from "@moduletrace/twui/layout/Link";
import Span from "@moduletrace/twui/layout/Span";
import { FolderOpen, Paperclip } from "lucide-react";
import { pathRefUrl } from "@/src/utils/path-ref";
import { storagePathName, type StorageSpec } from "@/src/utils/storage-volumes";
type Props = {
spec: StorageSpec;
path: string;
};
/**
* A folder or file a bot named in a message (`pathRef`): the name it ends
* with, linked into the files browser, where the user can look at
* everything around it. That is how evidence a bot left on a computer is
* reached without copying it into its own files first.
*/
export default function PathRefChip({ spec, path }: Props) {
const name = storagePathName({ volume: spec.volume, path });
const is_folder = !/\.[a-z0-9]{1,8}$/i.test(name);
return (
<Link
href={pathRefUrl({ volume: spec.volume, owner_id: spec.owner_id, path })}
title={`Open ${path || name} in Files`}
className="inline-flex h-6 max-w-full gap-1.5 items-center px-1.5 align-middle rounded-default-xs border-0 bg-sunken text-[13px] font-bold leading-4 text-ink hover:opacity-80"
>
{is_folder ? (
<FolderOpen size={14} aria-hidden className="shrink-0" />
) : (
<Paperclip size={14} aria-hidden className="shrink-0" />
)}
<Span className="truncate">{name}</Span>
</Link>
);
}