Keep SELinux mount options out of the mount destination

Appending `,z` to a mount destination (`$DATA_DIR:/data,z`) makes docker
mount a directory literally named `/data,z`. The app then writes its
database into its own container layer instead of the host volume,
`container_matches` never matches so every update recreates the container
without its data, `backup_database` silently skips, and `resolveHostPath`
refuses to provision computers (`/data/computers/1 isn't on a volume`).

Build each spec as `source:destination[:options]` instead, with the `z`
and `ro` options comma separated in the third field and the destination a
plain absolute path. `backup_database` uses the same helper.

`rescue_container_data` runs before a container whose /data is not the
host's directory is replaced: it stops the app, copies `CONTAINER:/data`
out with `docker cp`, moves a non-empty host directory aside and writes
the copy in, so repairing an install from the broken form does not lose
the database, the media or the computers' home directories.

`install.test.ts` lifts the marked `selinux-detection` and `mount-specs`
blocks out of install.sh, runs them in bash and parses the specs the way
docker does, so the two forms can't drift back.

AGENTS.md

@@ -126,6 +126,8 @@ Inspect live data with `bun llm-code-assistant/query-db.ts "<sql>"` - a read-onl
 
 `install.sh` is the only supported install and update path: `curl -fsSL https://url.moduletrace.com/cobble | sudo bash`. It is idempotent, so the same script is the updater, and `--uninstall [--purge]` is the way back out. It installs Docker when missing (Linux), creates the `cobble` network, writes `$COBBLE_HOME/install.conf` and a generated `.env` (mode 600), and starts one `cobble` container. Existing secrets are never regenerated; the database is backed up to `$COBBLE_HOME/backups` before a container is replaced. The default home is `/opt/cobble`, configurable with `--home`, and everything that must survive an update lives in its `data/` directory (`db`, `media`, `computers`, `bots`, `workspaces`, `mcp`).
 
+The bind mounts are built in the marked `selinux-detection` and `mount-specs` blocks, and `install.test.ts` runs those blocks in bash and parses the specs the way docker does. A mount spec is `source:destination[:options]`: docker takes the SELinux `z` and the read-only `ro` as comma separated options in that third field, so a destination is always a plain absolute path. Appending an option to the destination (`$DATA_DIR:/data,z`) makes docker mount a directory literally named `/data,z`: the app then writes its database to its own container layer, `container_matches` never matches, every update recreates the container without its data, `backup_database` silently skips, and `resolveHostPath` refuses to provision computers (`/data/computers/1 isn't on a volume`). `rescue_container_data` runs before a container is replaced and copies a container's unmounted `/data` onto the host, so repairing an install from the broken form does not lose its database.
+
 The app serves plain HTTP only; TLS is the reverse proxy's or tunnel's job, and the installer, the image's `HEALTHCHECK` and the dev server all assume it (`Bun.serve` gets no `tls`, there is no certificate handling anywhere). `isSecureRequest` still reads the proxy's `X-Forwarded-Proto`, so the auth and CSRF cookies get `Secure` and the installed app, Web Push and voice keep their secure context behind HTTPS; the installer binds `0.0.0.0` by default, and `--bind 127.0.0.1` limits it to a local proxy. Updates drop any `TLS_*` or `HTTP_REDIRECT_PORT` entries an older install left in `.env` and recreate a container that still mounts `/app/certs`.
 
 The container runs with `--network cobble`, `-p BIND:PORT`, `.env` at `/app/.env` read only, the host data directory at `/data`, the docker socket at `/var/run/docker.sock`, `MAIN_SERVER=1` and `PORT`, `--init`, `--cpus 1` and `--memory 2048m` (a light orchestration process: the heavy work happens in each computer's own container; overridable with `--cpus`/`--memory`), a `unless-stopped` restart policy and bounded json-file logs. `/data` must stay a bind mount: `resolveHostPath` turns paths under the app's own mounts into host paths for computer containers, and a named volume would break them. `DATA_DIR=/data` and `DB_DIR=/data/db` come from the Dockerfile; `entrypoint.sh` keeps bun-sqlite's schema record on the same volume.

install.sh

@@ -345,23 +345,35 @@ if [ "$OS" = "Linux" ] && [ "$(id -u)" -ne 0 ]; then
     fail "Run the installer as root: curl -fsSL $INSTALL_URL | sudo bash"
 fi
 
-SELINUX_SUFFIX=""
+# SELinux hosts need the `z` mount option so docker relabels the bind
-if [ "$OS" = "Linux" ]; then
+# mount. It is a third colon-separated field (`src:/data:z`), never a
+# suffix on the destination: docker reads `src:/data,z` as the directory
+# "/data,z", which silently leaves the app writing to its own container
+# layer, breaks every update (the container is recreated without its data)
+# and makes computer provisioning fail. Never append to the destination.
+# MARK:selinux-detection
+selinux_enforcing() {
     if command -v getenforce >/dev/null 2>&1 && [ "$(getenforce 2>/dev/null || true)" != "Disabled" ]; then
-        SELINUX_SUFFIX=",z"
+        return 0
-    elif [ -e /sys/fs/selinux/enforce ]; then
-        SELINUX_SUFFIX=",z"
     fi
-fi
+    [ -e /sys/fs/selinux/enforce ]
+}
 
-ENV_MOUNT="$ENV_FILE:/app/.env:ro"
+MOUNT_OPTS=""
-DATA_MOUNT="$DATA_DIR:/data"
+if [ "$OS" = "Linux" ] && selinux_enforcing; then
-BACKUP_ARCHIVE_MOUNT="$BACKUP_ARCHIVE_DIR_HOST:/backups-archives"
+    MOUNT_OPTS="z"
-if [ -n "$SELINUX_SUFFIX" ]; then
-    ENV_MOUNT="$ENV_FILE:/app/.env:ro,z"
-    DATA_MOUNT="$DATA_DIR:/data,z"
-    BACKUP_ARCHIVE_MOUNT="$BACKUP_ARCHIVE_DIR_HOST:/backups-archives,z"
 fi
+# END:selinux-detection
+
+# MARK:mount-specs
+# A spec is `source:destination[:options]`. Options are comma separated and
+# only count in that third field, so each spec below either opens the field
+# with a colon or adds to one that is already there. A destination is
+# always a plain absolute path.
+ENV_MOUNT="$ENV_FILE:/app/.env:ro${MOUNT_OPTS:+,$MOUNT_OPTS}"
+DATA_MOUNT="$DATA_DIR:/data${MOUNT_OPTS:+:$MOUNT_OPTS}"
+BACKUP_ARCHIVE_MOUNT="$BACKUP_ARCHIVE_DIR_HOST:/backups-archives${MOUNT_OPTS:+:$MOUNT_OPTS}"
+# END:mount-specs
 
 # ---------------------------------------------------------------- helpers
 
@@ -527,10 +539,7 @@ backup_database() {
     mkdir -p "$BACKUP_DIR"
     say "Backing up the database"
 
-    local backup_mount="$data_source:/data"
+    local backup_mount="$data_source:/data${MOUNT_OPTS:+:$MOUNT_OPTS}"
-    if [ -n "$SELINUX_SUFFIX" ]; then
-        backup_mount="$backup_mount,z"
-    fi
 
     local image_id=""
     if container_running; then
@@ -575,6 +584,71 @@ backup_database() {
     fi
 }
 
+# Whether the container's /data is really the host's data directory.
+# Older installers wrote the SELinux option onto the destination
+# (`$DATA_DIR:/data,z`), so docker mounted a directory named "/data,z"
+# and the app kept its database in the container's own writable layer.
+# Such a container must have that data lifted out before it is replaced.
+data_mount_broken() {
+    container_exists || return 1
+    local mounted
+    mounted="$(docker inspect -f '{{range .Mounts}}{{if eq .Destination "/data"}}{{.Source}}{{end}}{{end}}' "$CONTAINER" 2>/dev/null || true)"
+    [ "$mounted" != "$DATA_DIR" ]
+}
+
+# Copies a container's unmounted /data back onto the host before the
+# container is recreated, so a broken mount is repaired without losing the
+# database, the media or the computers' home directories. Runs only when
+# the container's /data is not the host's directory, and refuses to
+# overwrite a host directory that already holds a database.
+rescue_container_data() {
+    data_mount_broken || return 0
+
+    # Stop the app first: the copy must see a database nothing is writing
+    # to, and the container is about to be replaced anyway.
+    if container_running; then
+        say "Stopping Cobble to copy its data out"
+        docker stop -t 30 "$CONTAINER" >/dev/null 2>&1 || true
+    fi
+
+    local staged
+    staged="$(mktemp -d)"
+
+    if ! docker cp "$CONTAINER:/data/." "$staged/" >/dev/null 2>&1; then
+        rm -rf "$staged"
+        fail "Could not read the container's data. Its database is still in $CONTAINER; copy it out before updating."
+    fi
+
+    # A container that never wrote anything has nothing to rescue: the app
+    # is about to start fresh on the host directory.
+    if [ -z "$(ls -A "$staged" 2>/dev/null)" ]; then
+        rm -rf "$staged"
+        return 0
+    fi
+
+    say "This install's data was inside its container (a broken mount); moving it to $DATA_DIR"
+
+    if [ -n "$(ls -A "$DATA_DIR" 2>/dev/null)" ]; then
+        local aside
+        aside="$DATA_DIR.before-rescue-$(date +%s)"
+        warn "$DATA_DIR was not empty; the copy is kept at $aside"
+        mv "$DATA_DIR" "$aside" || fail "Could not move $DATA_DIR aside"
+    fi
+
+    mkdir -p "$DATA_DIR"
+    cp -a "$staged/." "$DATA_DIR/" || {
+        rm -rf "$staged"
+        fail "Could not copy the rescued data into $DATA_DIR"
+    }
+    rm -rf "$staged"
+
+    if [ -f "$DATA_DIR/db/cobble" ]; then
+        ok "Rescued the database and data into $DATA_DIR"
+    else
+        warn "Rescued data has no database file; the install will start empty"
+    fi
+}
+
 safe_remove_dir() {
     local dir="$1"
     case "$dir" in
@@ -789,6 +863,9 @@ if [ "$FORCE" = 0 ] && container_matches; then
 fi
 
 if container_exists; then
+    # A container whose /data was never mounted keeps the database in its
+    # own layer, so lift it out before anything replaces the container.
+    rescue_container_data
     backup_database
     say "Replacing the Cobble container"
     docker stop -t 30 "$CONTAINER" >/dev/null 2>&1 || true

install.test.ts

@@ -0,0 +1,220 @@
+import { describe, expect, test } from "bun:test";
+import path from "path";
+
+const ROOT = import.meta.dir;
+const INSTALL_SCRIPT = path.join(ROOT, "install.sh");
+
+/**
+ * Lifts one marked block out of `install.sh`, so the tests run the real
+ * installer code instead of a copy of it that could drift.
+ */
+async function readMarkedBlock({ name }: { name: string }): Promise<string> {
+    const source = await Bun.file(INSTALL_SCRIPT).text();
+    const start = `# MARK:${name}\n`;
+    const end = `# END:${name}`;
+    const from = source.indexOf(start);
+    const to = source.indexOf(end);
+
+    expect(from).toBeGreaterThan(-1);
+    expect(to).toBeGreaterThan(from);
+
+    return source.slice(from + start.length, to);
+}
+
+/** Runs a bash snippet and returns its stdout, failing on any stderr. */
+async function runBash({ script }: { script: string }): Promise<string> {
+    const proc = Bun.spawn({
+        cmd: ["bash", "-c", script],
+        stdout: "pipe",
+        stderr: "pipe",
+    });
+
+    const [stdout, stderr, code] = await Promise.all([
+        new Response(proc.stdout).text(),
+        new Response(proc.stderr).text(),
+        proc.exited,
+    ]);
+
+    expect(stderr).toBe("");
+    expect(code).toBe(0);
+
+    return stdout.trim();
+}
+
+type MountSpecs = {
+    env: string;
+    data: string;
+    backup: string;
+};
+
+/** The three mount specs the installer builds for one SELinux state. */
+async function runMountSpecs({
+    selinux,
+}: {
+    selinux: boolean;
+}): Promise<MountSpecs> {
+    const block = await readMarkedBlock({ name: "mount-specs" });
+
+    const script = [
+        `ENV_FILE="/opt/cobble/.env"`,
+        `DATA_DIR="/opt/cobble/data"`,
+        `BACKUP_ARCHIVE_DIR_HOST="/opt/cobble/backups-archives"`,
+        `MOUNT_OPTS="${selinux ? "z" : ""}"`,
+        block,
+        `printf '%s\\n' "$ENV_MOUNT" "$DATA_MOUNT" "$BACKUP_ARCHIVE_MOUNT"`,
+    ].join("\n");
+
+    const [env = "", data = "", backup = ""] = (
+        await runBash({ script })
+    ).split("\n");
+
+    return { env, data, backup };
+}
+
+/**
+ * Splits a `source:destination[:options]` spec the way docker does, so a
+ * destination that accidentally swallowed an option is caught.
+ */
+function parseMountSpec({ spec }: { spec: string }) {
+    const [source = "", destination = "", ...rest] = spec.split(":");
+
+    return { source, destination, options: rest.join(":") };
+}
+
+describe("install.sh SELinux detection", () => {
+    test("asks for the mount option when SELinux is enforcing", async () => {
+        const block = await readMarkedBlock({
+            name: "selinux-detection",
+        });
+
+        // An enforcing `getenforce` answers before the /sys fallback, so
+        // this result does not depend on the machine running the test.
+        const script = [
+            `OS="Linux"`,
+            `getenforce() { printf '%s' "Enforcing"; }`,
+            block,
+            `printf '%s' "$MOUNT_OPTS"`,
+        ].join("\n");
+
+        expect(await runBash({ script })).toBe("z");
+    });
+
+    test("asks for no mount option when SELinux is disabled", async () => {
+        const block = await readMarkedBlock({
+            name: "selinux-detection",
+        });
+
+        // Answering `Disabled` makes the block fall through to the
+        // /sys/fs/selinux/enforce probe. Shadowing the path check keeps the
+        // result the same on an SELinux host and on a plain one.
+        const script = [
+            `OS="Linux"`,
+            `getenforce() { printf '%s' "Disabled"; }`,
+            `SELINUX_ENFORCE_FILE="/nonexistent/cobble-test"`,
+            block,
+            `printf '%s' "$MOUNT_OPTS"`,
+        ].join("\n");
+
+        expect(await runBash({ script })).toBe("");
+    });
+});
+
+describe("install.sh mount specs", () => {
+    for (const selinux of [false, true]) {
+        const label = selinux ? "with SELinux" : "without SELinux";
+
+        test(`mounts the data directory at /data ${label}`, async () => {
+            const { data } = await runMountSpecs({ selinux });
+            const { source, destination, options } = parseMountSpec({
+                spec: data,
+            });
+
+            // The regression: the SELinux option must be its own field.
+            // Gluing it onto the destination makes docker mount a
+            // directory named "/data,z", so the app keeps its database in
+            // its own container layer, every update wipes the install and
+            // computer provisioning fails.
+            expect(source).toBe("/opt/cobble/data");
+            expect(destination).toBe("/data");
+            expect(options).toBe(selinux ? "z" : "");
+        });
+
+        test(`mounts the backup archive directory ${label}`, async () => {
+            const { backup } = await runMountSpecs({ selinux });
+            const { source, destination, options } = parseMountSpec({
+                spec: backup,
+            });
+
+            expect(source).toBe("/opt/cobble/backups-archives");
+            expect(destination).toBe("/backups-archives");
+            expect(options).toBe(selinux ? "z" : "");
+        });
+
+        test(`mounts .env read-only ${label}`, async () => {
+            const { env } = await runMountSpecs({ selinux });
+            const { source, destination, options } = parseMountSpec({
+                spec: env,
+            });
+
+            expect(source).toBe("/opt/cobble/.env");
+            expect(destination).toBe("/app/.env");
+            // Docker splits options on commas, never on colons: `ro:z` is
+            // an unknown option, and the container refuses to start.
+            expect(options).toBe(selinux ? "ro,z" : "ro");
+        });
+
+        test(`keeps every option in its own field ${label}`, async () => {
+            const specs = await runMountSpecs({ selinux });
+
+            for (const spec of Object.values(specs)) {
+                const { destination, options } = parseMountSpec({ spec });
+
+                expect(destination).not.toContain(",");
+                expect(destination).not.toContain(":");
+                expect(destination.startsWith("/")).toBe(true);
+
+                // Only the options docker understands, comma separated.
+                for (const option of options.split(",")) {
+                    expect(["", "ro", "rw", "z", "Z"]).toContain(option);
+                }
+            }
+        });
+    }
+});
+
+describe("install.sh data rescue", () => {
+    test("rescues the container's data before replacing it", async () => {
+        const source = await Bun.file(INSTALL_SCRIPT).text();
+        const rescue = source.indexOf("\n    rescue_container_data\n");
+        const replace = source.indexOf(
+            `say "Replacing the Cobble container"`,
+        );
+
+        // A container whose /data was never mounted keeps its database in
+        // its own layer, so the copy has to happen before the container is
+        // removed.
+        expect(rescue).toBeGreaterThan(-1);
+        expect(replace).toBeGreaterThan(rescue);
+    });
+
+    test("the database backup uses the same mount options", async () => {
+        const source = await Bun.file(INSTALL_SCRIPT).text();
+
+        expect(source).toContain(`backup_mount="$data_source:/data`);
+        expect(source).toContain("${MOUNT_OPTS:+:$MOUNT_OPTS}");
+    });
+
+    test("no code line glues an option onto a destination", async () => {
+        const source = await Bun.file(INSTALL_SCRIPT).text();
+
+        // Comments may quote the old broken form; only real code counts.
+        const code = source
+            .split("\n")
+            .filter((line) => !line.trimStart().startsWith("#"))
+            .join("\n");
+
+        expect(code).not.toContain("/data,z");
+        expect(code).not.toContain("/app/.env:ro,z");
+        expect(code).not.toContain("/backups-archives,z");
+    });
+});