Stream computer files instead of reading them through the shell
import_file sent a computer's bytes back with base64 over /v1/shell/exec, whose stdout is capped at 30,047 characters. Every import was therefore cut at 22,528 bytes and saved anyway, with the tool reporting success: a 1.12 MiB archive became a 22 KB file that would not open, and a crawler's crawl.json lost half its pages. The bytes now come from the sandbox's streaming GET /v1/file/download. The size is read first so nothing over BotFileImportMaxBytes is transferred, the download asks for change_policy=abort so a file that changes mid-send is refused rather than half-saved, and the received length is checked against the size read, so a cut-short transfer fails loudly. Folders go the same way, as a gzipped tar unpacked into one row per file. Bots also name a path as [path:volume:owner:path] now, so what a bot leaves on a computer is a link into the files browser instead of text.
This commit is contained in:
@@ -417,6 +417,17 @@ export const AppData = {
|
||||
* The files modal and the bot's file list: one page of them.
|
||||
*/
|
||||
BotFilesPageSize: 50,
|
||||
/**
|
||||
* The files browser at `/admin/files`: how many entries one folder
|
||||
* shows, and the most of a file it hands back at once.
|
||||
*/
|
||||
StorageBrowserPageSize: 200,
|
||||
StorageBrowserReadMaxBytes: 50 * 1024 * 1024,
|
||||
/**
|
||||
* The most a bot copies off a computer into its own files in one
|
||||
* call, so a folder of raw evidence can't fill the data volume.
|
||||
*/
|
||||
BotFileImportMaxBytes: 20 * 1024 * 1024,
|
||||
/**
|
||||
* Ingesting a bot file: how much text one chunk holds, how much of the
|
||||
* previous chunk it repeats, and the largest file it reads at all.
|
||||
|
||||
Reference in New Issue
Block a user