Moduletrace/cobble
2
0
Fork 0
Code Issues Pull Requests Actions 2 Packages Projects Releases 11 Wiki Activity

Keep every bot's computer work in its own workspace folder

Browse Source 
Computers now mount only /home/gem/workspace from DATA_DIR/computers/<id>,
so only the workspace persists and goes into the data archive. An older
computer's whole home is moved aside to <id>-home-<ts> (kept, never
archived) and its container recreated once by sync-computers.

Each run works in bots/<bot>/users/<user> (also its cwd and HOME), with
bots/<bot> for work every user shares and workspaces/<id> shared by a
workspace's bots; the coordinator reaches the whole workspace. Computer
tools are gated by withComputerDirs: bash and code start there, file tool
paths and file: URLs outside the run's folders are refused, and so is sudo.
search_computers, import_file and script tools start there too.

The files browser shows members only the bots' folders they reach, their
own user folder in each and their workspaces', with names on the id folders.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
This commit is contained in:
tben
2026-10-25 08:24:49 +01:00
co-authored by Claude Opus 5.5
parent fefc295d15
commit 86becd50bc
41 changed files with 1012 additions and 105 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/functions/backend/db/computers/migrate-computer-layout.ts
+36
View File
@@ -0,0 +1,36 @@
import parseJobJson from "@/src/bots-queue/parse-job-json";
import { AppData } from "@/src/data/app-data";
import logEvent from "@/src/functions/backend/logger";
import provisionComputer from "./provision-computer";
type Params = {
computer_id: number;
/**
* Its container's `metadata`.
*/
metadata?: string | null;
};
/**
* Recreates a computer whose container still mounts its whole home (set up
* before only `AppData.ComputerWorkspaceDir` was mounted), so its bots work
* in the workspace alone. `provisionComputer` moves the old home aside
* (`prepareComputerDataDir`) and records the layout, so this runs once per
* computer. Returns whether it did.
*/
export default async function migrateComputerLayout({ computer_id, metadata }: Params) {
const layout = parseJobJson({ text: metadata || "" })?.layout;
if (layout == AppData.ComputerWorkspaceLayout) return false;
await provisionComputer({ computer_id });
await logEvent({
title: `Computer #${computer_id} now mounts only its workspace`,
category: "system",
plain_text: `Recreated computer #${computer_id} so only ${AppData.ComputerWorkspaceDir} is kept; its old home was moved aside and kept on disk.`,
metadata: { computer_id },
});
return true;
}
src/functions/backend/db/computers/prepare-computer-data-dir.ts
+31 -13
View File
@@ -1,4 +1,5 @@
import fs from "fs/promises";
import path from "path";
import { AppData } from "@/src/data/app-data";
import grabComputerDataDir from "./grab-computer-data-dir";
@@ -13,28 +14,45 @@ type Params = {
};
/**
* Creates a computer's home directory, owned by the sandbox's user so it
* can write there, and returns its path. Docker would otherwise create a
* missing mount source owned by root.
* Creates a computer's workspace directory (mounted at
* `AppData.ComputerWorkspaceDir`), with its `bots` and `workspaces`
* folders, owned by the sandbox's user so it can write there, and returns
* its path. Docker would otherwise create a missing mount source owned by
* root.
*
* A directory without the layout marker (`AppData.ComputerWorkspaceMarker`)
* is a computer's whole home from before only the workspace was mounted:
* it is moved aside to `<id>-home-<timestamp>` and kept on disk (left out
* of data backups), so the computer starts with a clean workspace.
*/
export default async function prepareComputerDataDir({ computer_id, fresh }: Params) {
const dir = grabComputerDataDir({ computer_id });
const marker = path.join(dir, AppData.ComputerWorkspaceMarker);
const existing = await fs.readdir(dir).catch(() => null);
if (fresh) {
const existing = await fs.readdir(dir).catch(() => null);
if (existing?.length) {
if (existing?.length) {
if (fresh) {
await fs.rename(dir, `${dir}-${Date.now()}`);
} else if (!(await fs.stat(marker).catch(() => null))) {
await fs.rename(dir, `${dir}${AppData.ComputerOldHomeSuffix}${Date.now()}`);
}
}
await fs.mkdir(dir, { recursive: true });
for (const folder of [dir, path.join(dir, "bots"), path.join(dir, "workspaces")]) {
await fs.mkdir(folder, { recursive: true });
await chownToSandbox({ target: folder });
}
/**
* Only root can hand a directory to another user; a dev server
* running as that user already owns it.
*/
await fs.chown(dir, AppData.ComputerUserUid, AppData.ComputerUserGid).catch(() => {});
await fs.writeFile(marker, "", { flag: "a" });
await chownToSandbox({ target: marker });
return dir;
}
/**
* Only root can hand a file to another user; a dev server running as that
* user already owns it.
*/
async function chownToSandbox({ target }: { target: string }) {
await fs.chown(target, AppData.ComputerUserUid, AppData.ComputerUserGid).catch(() => {});
}
src/functions/backend/db/computers/provision-computer.ts
+11 -4
View File
@@ -10,18 +10,20 @@ import pushComputerUpdate from "./push-computer-update";
type Params = {
computer_id: number;
/**
* A brand new computer: start from an empty home directory.
* A brand new computer: start from an empty workspace.
*/
fresh?: boolean;
};
/**
* Sets up a computer's container: prepares its home directory, pulls the
* Sets up a computer's container: prepares its workspace directory, pulls the
* sandbox image and starts the container with the computer's resources
* (`cpus`, `memory_mb`, else the `AppData` defaults), then records it (`containers`:
* docker id, state, and its IP address on the network). A failure is
* recorded as `failed` with its reason, which the computer lists show. The
* owner's tabs hear about it either way. Never throws.
* owner's tabs hear about it either way. The metadata's `layout` says the
* container mounts only the workspace (`syncComputers` recreates one that
* doesn't). Never throws.
*/
export default async function provisionComputer({ computer_id, fresh }: Params) {
const row = (
@@ -65,7 +67,12 @@ export default async function provisionComputer({ computer_id, fresh }: Params)
created.ip_address,
AppData.ComputerPort,
created.status,
JSON.stringify({ ...kept, data_dir, host_dir: created.host_dir }),
JSON.stringify({
...kept,
data_dir,
host_dir: created.host_dir,
layout: AppData.ComputerWorkspaceLayout,
}),
Date.now(),
Date.now(),
row.id,
src/functions/backend/db/computers/resolve-computer-home-dir.test.ts → src/functions/backend/db/computers/resolve-computer-workspace-dir.test.ts
+10 -10
View File
@@ -1,16 +1,16 @@
import { describe, expect, test } from "bun:test";
import resolveComputerHomeDir from "./resolve-computer-home-dir";
import resolveComputerWorkspaceDir from "./resolve-computer-workspace-dir";
const data_dir = "/data";
describe("resolveComputerHomeDir", () => {
describe("resolveComputerWorkspaceDir", () => {
test("uses the directory the app bind-mounted", () => {
/**
* The truth for a container the app provisioned: on a bind mount
* that can be anywhere on the docker host, not under DATA_DIR.
*/
expect(
resolveComputerHomeDir({
resolveComputerWorkspaceDir({
computer_id: 2,
kind: "sandbox",
metadata: JSON.stringify({ data_dir: "/srv/homes/2", host_dir: "/host/2" }),
@@ -22,7 +22,7 @@ describe("resolveComputerHomeDir", () => {
test("falls back to MAIN_COMPUTER_DATA_DIR for the main computer", () => {
expect(
resolveComputerHomeDir({
resolveComputerWorkspaceDir({
computer_id: 1,
kind: "main",
metadata: null,
@@ -34,7 +34,7 @@ describe("resolveComputerHomeDir", () => {
test("leaves a sandbox alone whatever the main setting says", () => {
expect(
resolveComputerHomeDir({
resolveComputerWorkspaceDir({
computer_id: 3,
kind: "sandbox",
metadata: null,
@@ -45,26 +45,26 @@ describe("resolveComputerHomeDir", () => {
});
test("uses the conventional folder when nothing is recorded", () => {
expect(resolveComputerHomeDir({ computer_id: 4, kind: "main", data_dir })).toBe(
expect(resolveComputerWorkspaceDir({ computer_id: 4, kind: "main", data_dir })).toBe(
"/data/computers/4",
);
});
test("ignores an empty or unreadable record", () => {
expect(
resolveComputerHomeDir({ computer_id: 5, metadata: JSON.stringify({ data_dir: " " }), data_dir }),
resolveComputerWorkspaceDir({ computer_id: 5, metadata: JSON.stringify({ data_dir: " " }), data_dir }),
).toBe("/data/computers/5");
expect(resolveComputerHomeDir({ computer_id: 5, metadata: "not json", data_dir })).toBe(
expect(resolveComputerWorkspaceDir({ computer_id: 5, metadata: "not json", data_dir })).toBe(
"/data/computers/5",
);
expect(resolveComputerHomeDir({ computer_id: 5, metadata: null, data_dir })).toBe(
expect(resolveComputerWorkspaceDir({ computer_id: 5, metadata: null, data_dir })).toBe(
"/data/computers/5",
);
});
test("normalizes a recorded relative path", () => {
expect(
resolveComputerHomeDir({ computer_id: 6, metadata: JSON.stringify({ data_dir: "/srv/a/../b" }), data_dir }),
resolveComputerWorkspaceDir({ computer_id: 6, metadata: JSON.stringify({ data_dir: "/srv/a/../b" }), data_dir }),
).toBe("/srv/b");
});
});
src/functions/backend/db/computers/resolve-computer-home-dir.ts → src/functions/backend/db/computers/resolve-computer-workspace-dir.ts
+7 -5
View File
@@ -27,8 +27,9 @@ type Params = {
};
/**
* Where a computer's home (`AppData.ComputerHomeDir` in its container)
* actually is on this app's disk, for the files browser and for anything
* Where a computer's workspace (`AppData.ComputerWorkspaceDir` in its
* container, the only directory mounted from the data volume) actually is
* on this app's disk, for the files browser and for anything
* else that has to read a computer's files.
*
* A container the app provisioned recorded the directory it bind-mounted
@@ -36,15 +37,16 @@ type Params = {
* that path can be anywhere the docker host allows. A computer the app
* didn't provision has no record. That is the install's main computer on a
* server reached at `MAIN_COMPUTER_URL` (a dev server outside docker),
* whose sandbox keeps its home wherever its compose file says, so
* `MAIN_COMPUTER_DATA_DIR` names that directory for this app. Without
* whose sandbox keeps its workspace wherever its compose file says, so
* `MAIN_COMPUTER_DATA_DIR` names that directory (the host side of
* `AppData.ComputerWorkspaceDir`) for this app. Without
* either, the conventional place every app-provisioned computer uses:
* `DATA_DIR/<AppData.ComputerDataDir>/<id>`.
*
* Kept apart from the docker and database calls so the precedence is
* testable on its own.
*/
export default function resolveComputerHomeDir({
export default function resolveComputerWorkspaceDir({
computer_id,
kind,
metadata,