diff --git a/AGENTS.md b/AGENTS.md index 12068aa..0a103d0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,6 +47,8 @@ For the current UI, read the code: `src/layouts/bots/` (rail, workspace rail, us | Seed sample bot crew | `bun run seed:bots [email]` (runs `src/scripts/seed-bots.ts`, idempotent: adds sample messages to the existing coordinator and the sample workers, skips users who already created workers; assigns the sample workers to them) | | Query DB (CLI) | `bun llm-code-assistant/query-db.ts ""` | | Install | `bun install` | +| Publish a release to GitHub | `bun run publish:github` (`scripts/publish-github.sh`) | +| Publish a release to Gitea | `bun run publish:gitea` (`scripts/publish-gitea.sh`) | No linter or formatter configured. `bun run typecheck` and `bun run test` are the verification steps. Tests are colocated `.test.ts`/`.test.tsx` files - run one with `bun test `. @@ -132,7 +134,7 @@ The app serves plain HTTP only; TLS is the reverse proxy's or tunnel's job, and The container runs with `--network cobble`, `-p BIND:PORT`, `.env` at `/app/.env` read only, the host data directory at `/data`, the docker socket at `/var/run/docker.sock`, `MAIN_SERVER=1` and `PORT`, `--init`, `--cpus 1` and `--memory 2048m` (a light orchestration process: the heavy work happens in each computer's own container; overridable with `--cpus`/`--memory`), a `unless-stopped` restart policy and bounded json-file logs. `/data` must stay a bind mount: `resolveHostPath` turns paths under the app's own mounts into host paths for computer containers, and a named volume would break them. `DATA_DIR=/data` and `DB_DIR=/data/db` come from the Dockerfile; `entrypoint.sh` keeps bun-sqlite's schema record on the same volume. -Images are built by GitHub Actions on a published release (`.github/workflows/release.yml`), for `linux/amd64` and `linux/arm64`, and pushed to both `ghcr.io/moduletrace/cobble` and `git.tben.me/moduletrace/cobble` with `latest`, version, `major.minor` and commit-sha tags. The version they are built with is passed as the `COBBLE_VERSION` build arg (the published tag, else `package.json`'s version read from ``), which the image turns into `ENV COBBLE_VERSION` and `runningVersion` (`src/utils/running-version.ts`) reports on Settings > General; bump `package.json`'s `version` with a release, since a local build and a workflow-dispatch build fall back to it. `.github/workflows/ci.yml` runs `bun run typecheck` and `bun run test` on every push and pull request, and checks that `install.sh` parses. The Dockerfile pins Bun 1.4.3 to match CI. The repository is mirrored on Gitea, and the installer defaults to the GHCR image with `--image git.tben.me/moduletrace/cobble` as the Gitea alternative. +Publishing is `scripts/publish-github.sh` and `scripts/publish-gitea.sh`: the shared flow (argument parsing, the release gate, tag preparation, the push, the release notes) lives in `scripts/lib/publish-common.sh` and `scripts/lib/publish-release.sh`, and only the release object differs (`gh release create` for GitHub, which is what fires the image build, and `curl` against the Gitea API with a `GITEA_TOKEN` or the token `tea login` stored). Both are marked-block scripts tested by `scripts/publish-tools.test.ts`, which lifts the blocks out with the `install.test.ts` approach. Images are built by GitHub Actions on a published release (`.github/workflows/release.yml`), for `linux/amd64` and `linux/arm64`, and pushed to both `ghcr.io/moduletrace/cobble` and `git.tben.me/moduletrace/cobble` with `latest`, version, `major.minor` and commit-sha tags. The version they are built with is passed as the `COBBLE_VERSION` build arg (the published tag, else `package.json`'s version read from ``), which the image turns into `ENV COBBLE_VERSION` and `runningVersion` (`src/utils/running-version.ts`) reports on Settings > General; bump `package.json`'s `version` with a release, since a local build and a workflow-dispatch build fall back to it. `.github/workflows/ci.yml` runs `bun run typecheck` and `bun run test` on every push and pull request, and checks that `install.sh` parses. The Dockerfile pins Bun 1.4.3 to match CI. The repository is mirrored on Gitea, and the installer defaults to the GHCR image with `--image git.tben.me/moduletrace/cobble` as the Gitea alternative. ### Envs diff --git a/README.md b/README.md index 629b966..49f4147 100644 --- a/README.md +++ b/README.md @@ -247,6 +247,17 @@ Install from Gitea with `--image git.tben.me/moduletrace/cobble`, or build from source with `docker build -t cobble:local .` and `--image cobble --version local --no-pull`. +A release is published from a clean `main` with the two scripts, which tag the +`package.json` version, push and create the release objects: + +```bash +scripts/publish-github.sh # or: bun run publish:github +scripts/publish-gitea.sh # or: bun run publish:gitea +``` + +Both take `--version VERSION`, `--dry-run` and `--yes`. Only the GitHub release +builds the image; see [RELEASING.md](RELEASING.md) for the full checklist. + ## Development ```bash diff --git a/RELEASING.md b/RELEASING.md index b19df5b..a89ec28 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -7,6 +7,12 @@ The repository starts private and is made public at the launch gate below. - [ ] `GITEA_USERNAME` and `GITEA_TOKEN` are set as GitHub Actions secrets. The token needs package write on the Gitea `Moduletrace` org. +- [ ] `scripts/publish-gitea.sh` can reach Gitea: either `GITEA_TOKEN` is + exported locally, or `tea login add` has been run for the host (the + script then reads the token from `~/.config/tea/config.yml`). It needs + `write:repository` on the repository to create the release. +- [ ] `scripts/publish-github.sh` can publish releases: `gh auth login` has + been run, and the token has `repo` scope. - [ ] The Gitea repository exists at `git.tben.me/Moduletrace/cobble` and `main` is pushed to both remotes. - [ ] The GHCR package is made public after the first image push @@ -16,6 +22,16 @@ The repository starts private and is made public at the launch gate below. ## Every release +The two publish scripts below do steps 1 to 3, 6 and 10 for you: they run the +same checks, tag `HEAD` with `package.json`'s version, push `main` and the tag, +and create the release object. Everything else is still hand work. + +`scripts/publish-github.sh` publishes to GitHub and, by publishing the +release, starts the image build. `scripts/publish-gitea.sh` publishes the same +release to the Gitea mirror. Both accept `--version VERSION` (defaulting to +`package.json`), `--branch`, `--yes`, `--dry-run` and `--skip-checks`; run +them in that order, since only the GitHub release builds images. + 1. [ ] `bun run typecheck` and `bun run test` pass locally. 2. [ ] CI is green on `main` in GitHub Actions. 3. [ ] `bash -n install.sh` passes. @@ -27,12 +43,17 @@ The repository starts private and is made public at the launch gate below. - `--uninstall` keeps data, `--uninstall --purge` removes it 5. [ ] Update the version if the release changes it, including `package.json`'s `version` (the image reports it on Settings > General - when it isn't built from a release tag). + when it isn't built from a release tag, and the publish scripts tag + with it). 6. [ ] Tag and publish a GitHub release: - git tag v0.1.0 - git push origin main --tags - gh release create v0.1.0 --title v0.1.0 --generate-notes + scripts/publish-github.sh # or: bun run publish:github + + # by hand, the script does this: + git tag -a v0.1.0 -m v0.1.0 + git push origin main + git push origin refs/tags/v0.1.0 + gh release create v0.1.0 --title v0.1.0 --notes-file notes.md --latest 7. [ ] The release workflow builds `linux/amd64` and `linux/arm64` and pushes to both registries: @@ -44,9 +65,13 @@ The repository starts private and is made public at the launch gate below. docker buildx imagetools inspect git.tben.me/moduletrace/cobble:latest 9. [ ] Confirm the GHCR package is public and pull it anonymously. -10. [ ] Push the tag to Gitea as well: +10. [ ] Publish the release to Gitea as well: - git push gitea main --tags + scripts/publish-gitea.sh # or: bun run publish:gitea + + # by hand, the script does this: + git push gitea main + git push gitea refs/tags/v0.1.0 11. [ ] Update every install with the one-liner: diff --git a/package.json b/package.json index ed3e217..f9d3dcc 100644 --- a/package.json +++ b/package.json @@ -16,7 +16,9 @@ "start": "MAIN_SERVER=1 NODE_ENV=production bun src/server.ts", "typecheck": "tsc --noEmit", "test": "bun test", - "check:bundle": "bun src/scripts/check-bundle-size.ts" + "check:bundle": "bun src/scripts/check-bundle-size.ts", + "publish:github": "bash scripts/publish-github.sh", + "publish:gitea": "bash scripts/publish-gitea.sh" }, "devDependencies": { "@next/mdx": "^16.3.0", diff --git a/scripts/lib/publish-common.sh b/scripts/lib/publish-common.sh new file mode 100755 index 0000000..4b3d79a --- /dev/null +++ b/scripts/lib/publish-common.sh @@ -0,0 +1,324 @@ +#!/usr/bin/env bash +# Shared helpers for the publish scripts. +# +# . "$(dirname "$0")/lib/publish-common.sh" +# +# Never run this file: it only defines functions and constants, so the +# scripts own their own `set -Eeuo pipefail` and a test can source it +# without changing the caller's shell options. + +# MARK:preamble +if [ -t 1 ]; then + C_RESET=$'\033[0m' + C_BOLD=$'\033[1m' + C_DIM=$'\033[2m' + C_RED=$'\033[31m' + C_GREEN=$'\033[32m' + C_YELLOW=$'\033[33m' + C_BLUE=$'\033[34m' +else + C_RESET="" + C_BOLD="" + C_DIM="" + C_RED="" + C_GREEN="" + C_YELLOW="" + C_BLUE="" +fi + +DRY_RUN="${DRY_RUN:-0}" +ASSUME_YES="${ASSUME_YES:-0}" +RELEASE_NOTES_MAX_COMMITS="${RELEASE_NOTES_MAX_COMMITS:-100}" + +step() { printf '\n%s==>%s %s\n' "$C_BLUE$C_BOLD" "$C_RESET" "$*"; } +ok() { printf '%s ok%s %s\n' "$C_GREEN" "$C_RESET" "$*"; } +warn() { printf '%swarn%s %s\n' "$C_YELLOW" "$C_RESET" "$*" >&2; } +detail() { printf ' %s%s%s\n' "$C_DIM" "$*" "$C_RESET"; } + +fail() { + printf '%sfail%s %s\n' "$C_RED" "$C_RESET" "$*" >&2 + exit 1 +} + +# Runs a command, or prints it when the script was asked for a dry run. +# Only use this for commands that can be spelled as a single command. +run() { + if [ "$DRY_RUN" = "1" ]; then + printf ' %s[dry run]%s %s\n' "$C_DIM" "$C_RESET" "$*" + return 0 + fi + "$@" +} + +is_dry_run() { [ "$DRY_RUN" = "1" ]; } +# END:preamble + +# MARK:require-cmd +require_cmd() { + local name="$1" + local hint="${2:-}" + if ! command -v "$name" >/dev/null 2>&1; then + if [ -n "$hint" ]; then + fail "Missing '$name'. $hint" + fi + fail "Missing '$name'." + fi +} +# END:require-cmd + +# Moves into the top of the checkout and remembers it as ROOT. +require_repo_root() { + ROOT="$(git rev-parse --show-toplevel 2>/dev/null)" || fail "Not inside a git checkout." + cd "$ROOT" + export ROOT +} + +# MARK:git-state +require_clean_worktree() { + local dirty + dirty="$(git status --porcelain --untracked-files=no)" + if [ -n "$dirty" ]; then + printf '%s\n' "$dirty" >&2 + fail "The working tree has uncommitted changes. Commit or stash them before publishing." + fi +} + +require_on_branch() { + local branch="$1" + local current + current="$(git rev-parse --abbrev-ref HEAD)" + if [ "$current" != "$branch" ]; then + fail "On '$current', not '$branch'. Check out $branch before publishing (or pass --branch)." + fi +} + +require_remote() { + local remote="$1" + if ! git remote get-url "$remote" >/dev/null 2>&1; then + fail "No '$remote' git remote. Add it with: git remote add $remote " + fi +} + +# Fails when the remote branch has commits HEAD does not have. HEAD being +# ahead is fine: the script pushes. +require_not_behind() { + local remote="$1" + local branch="$2" + local counts ahead behind + + # A remote with no branch yet is not behind: the first push creates it. + if ! git ls-remote --exit-code --heads "$remote" "refs/heads/$branch" >/dev/null 2>&1; then + detail "$remote has no $branch yet; the first push creates it." + return 0 + fi + + git fetch --quiet "$remote" "$branch" || fail "Could not fetch $remote/$branch." + if ! git rev-parse -q --verify "refs/remotes/$remote/$branch" >/dev/null; then + return 0 + fi + + counts="$(git rev-list --left-right --count "HEAD...$remote/$branch")" + ahead="$(printf '%s' "$counts" | cut -f1)" + behind="$(printf '%s' "$counts" | cut -f2)" + if [ "$behind" != "0" ]; then + fail "$remote/$branch has $behind commit(s) HEAD does not have. Pull before publishing." + fi + if [ "$ahead" != "0" ]; then + detail "HEAD is $ahead commit(s) ahead of $remote/$branch; they are pushed with the release." + fi +} + +# Prints the tag already on the remote for a version, or nothing. +remote_tag_exists() { + local remote="$1" + local tag="$2" + local found + found="$(git ls-remote --tags "$remote" "refs/tags/$tag" 2>/dev/null || true)" + [ -n "$found" ] && printf 'yes' +} + +local_tag_exists() { + git rev-parse -q --verify "refs/tags/$1" >/dev/null +} + +# Creates the annotated tag when it is missing, and reuses an existing one. +ensure_tag() { + local tag="$1" + local message="${2:-$1}" + if local_tag_exists "$tag"; then + ok "Tag $tag already exists locally; reusing it." + return 0 + fi + run git tag -a "$tag" -m "$message" +} +# END:git-state + +# MARK:remote-url +# Splits a git remote URL into REMOTE_HOST, REMOTE_OWNER and REMOTE_REPO. +# Understands git@host:owner/repo.git, ssh://git@host[:port]/owner/repo.git +# and https://host[:port]/owner/repo.git. +parse_remote_url() { + local url="$1" + local rest path + + case "$url" in + *://*) + rest="${url#*://}" + rest="${rest#*@}" + REMOTE_HOST="${rest%%/*}" + path="${rest#*/}" + ;; + *@*:*) + rest="${url#*@}" + REMOTE_HOST="${rest%%:*}" + path="${rest#*:}" + ;; + *) + fail "Could not read a host out of the remote URL '$url'." + ;; + esac + + path="${path#/}" + path="${path%.git}" + REMOTE_OWNER="${path%%/*}" + REMOTE_REPO="${path##*/}" + + if [ -z "$REMOTE_HOST" ] || [ -z "$REMOTE_OWNER" ] || [ -z "$REMOTE_REPO" ]; then + fail "Could not read owner and repository out of the remote URL '$url'." + fi +} +# END:remote-url + +# MARK:version +VERSION_PATTERN='^[0-9]+\.[0-9]+\.[0-9]+([-+][0-9A-Za-z.+-]+)?$' + +# Turns `0.1.2` or `v0.1.2` into `v0.1.2`. +normalize_version() { + local raw="$1" + printf 'v%s' "${raw#v}" +} + +# Refuses anything that is not a release version, so a typo never becomes a +# tag that the release workflow cannot read a major.minor from. +validate_version() { + local raw="$1" + local bare="${raw#v}" + if ! printf '%s' "$bare" | grep -Eq "$VERSION_PATTERN"; then + fail "Version '$raw' is not a release version. Expected vMAJOR.MINOR.PATCH, e.g. v0.1.2." + fi +} + +# The version in package.json, read the same way the release workflow reads it. +source_version() { + sed -n -E 's/^[[:space:]]*"version": "([^"]+)",?$/\1/p' "${ROOT:?}/package.json" | head -n1 +} + +# The newest release tag before `to`, or nothing when there is none. +previous_tag() { + local to="$1" + git -C "${ROOT:?}" describe --tags --abbrev=0 --match 'v[0-9]*' "${to}^" 2>/dev/null || true +} +# END:version + +# MARK:notes +# Builds a release body from the commit log, so the Gitea mirror gets notes +# too (only GitHub can generate them itself). +release_notes() { + local from="" + local to="" + local compare_url="" + + while [ $# -gt 0 ]; do + case "$1" in + --from) from="$2"; shift 2 ;; + --to) to="$2"; shift 2 ;; + --compare-url) compare_url="$2"; shift 2 ;; + *) fail "release_notes: unknown argument '$1'." ;; + esac + done + + if [ -z "$to" ]; then + fail "release_notes: --to is required." + fi + + local range="$to" + if [ -n "$from" ]; then + range="$from..$to" + fi + + local commits total + commits="$(git -C "${ROOT:?}" log --no-merges --pretty=format:'- %s (%h)' --max-count="$RELEASE_NOTES_MAX_COMMITS" "$range")" + total="$(git -C "$ROOT" log --no-merges --oneline "$range" | wc -l | tr -d ' ')" + + printf '## Changes\n\n' + if [ -z "$commits" ]; then + printf 'No commits listed.\n' + else + printf '%s\n' "$commits" + if [ "$total" -gt "$RELEASE_NOTES_MAX_COMMITS" ]; then + printf '\n_Showing the newest %s of %s commits._\n' "$RELEASE_NOTES_MAX_COMMITS" "$total" + fi + fi + if [ -n "$compare_url" ]; then + printf '\n**Full changelog**: %s\n' "$compare_url" + fi +} +# END:notes + +# MARK:json-escape +# Escapes a string for a JSON body. jq is used when it is installed, since it +# handles every control character; the bash fallback covers the ones release +# notes actually contain. +json_escape() { + local text="$1" + + if command -v jq >/dev/null 2>&1; then + # `-s` produces one JSON string; the quotes it wraps it in are the + # characters the caller's template adds, so they come back off. + local escaped + escaped="$(printf '%s' "$text" | jq -R -s '.')" + escaped="${escaped:1:${#escaped}-2}" + printf '%s' "$escaped" + return 0 + fi + + local out="" + local char + local i=0 + local length=${#text} + while [ "$i" -lt "$length" ]; do + char="${text:$i:1}" + case "$char" in + '\') out+='\\' ;; + '"') out+='\"' ;; + $'\n') out+='\n' ;; + $'\r') out+='\r' ;; + $'\t') out+='\t' ;; + $'\b') out+='\b' ;; + $'\f') out+='\f' ;; + *) out+="$char" ;; + esac + i=$((i + 1)) + done + printf '%s' "$out" +} +# END:json-escape + +# MARK:confirm +confirm() { + local prompt="$1" + local answer="" + + if [ "$ASSUME_YES" = "1" ]; then + return 0 + fi + if [ ! -t 0 ]; then + fail "Not a terminal, so nothing was confirmed. Rerun with --yes to publish without a prompt." + fi + read -r -p "$prompt [y/N] " answer || true + case "$answer" in + y | Y | yes | YES) return 0 ;; + *) fail "Cancelled." ;; + esac +} +# END:confirm diff --git a/scripts/lib/publish-release.sh b/scripts/lib/publish-release.sh new file mode 100755 index 0000000..445d3bb --- /dev/null +++ b/scripts/lib/publish-release.sh @@ -0,0 +1,213 @@ +#!/usr/bin/env bash +# The release flow both publish scripts share: argument parsing, preflight +# checks, tag preparation and the push. +# +# . "$(dirname "$0")/lib/publish-common.sh" +# . "$(dirname "$0")/lib/publish-release.sh" +# +# What differs between the two remotes is only the last step: the GitHub +# release object (which the release workflow builds the images from) and the +# Gitea one. Everything before that lives here, so the two scripts cannot +# drift apart. + +# Defaults, overridden by the flags in parse_publish_args. +BRANCH="main" +VERSION_RAW="" +TAG="" +SKIP_CHECKS=0 +HELP=0 +REMAINING_ARGS=() +NOTES_FILE="" +TAG_CREATED=0 + +parse_publish_args() { + while [ $# -gt 0 ]; do + case "$1" in + -h | --help) HELP=1; shift ;; + --yes | -y) ASSUME_YES=1; shift ;; + --dry-run) DRY_RUN=1; shift ;; + --branch) BRANCH="${2:?--branch needs a value}"; shift 2 ;; + --version) VERSION_RAW="${2:?--version needs a value}"; shift 2 ;; + --skip-checks) SKIP_CHECKS=1; shift ;; + *) + REMAINING_ARGS+=("$1") + shift + ;; + esac + done + + DRY_RUN="${DRY_RUN:-0}" + ASSUME_YES="${ASSUME_YES:-0}" +} + +# Fails when the platform script was handed a flag it does not know. +reject_unknown_args() { + local script="$1" + if [ "${#REMAINING_ARGS[@]}" -eq 0 ]; then + return 0 + fi + printf 'Unknown argument: %s\n\n' "${REMAINING_ARGS[*]}" >&2 + "$script" --help >&2 + exit 2 +} + +# MARK:preflight +preflight() { + local remote="$1" + + require_repo_root + require_cmd git + require_cmd "bun" "Install Bun: https://bun.sh" + + step "Checking the checkout" + require_clean_worktree + ok "The working tree has nothing uncommitted." + require_on_branch "$BRANCH" + ok "On $BRANCH." + require_remote "$remote" + ok "Remote '$remote' is $(git remote get-url "$remote")." + require_not_behind "$remote" "$BRANCH" + ok "$remote/$BRANCH has nothing HEAD is missing." +} +# END:preflight + +# MARK:resolve-version +# Sets TAG (vX.Y.Z) from --version, else from package.json. +resolve_version() { + step "Resolving the version" + if [ -z "$VERSION_RAW" ]; then + VERSION_RAW="$(source_version)" + if [ -z "$VERSION_RAW" ]; then + fail "No version in package.json and none passed with --version." + fi + detail "Read from package.json: $VERSION_RAW" + fi + validate_version "$VERSION_RAW" + TAG="$(normalize_version "$VERSION_RAW")" + ok "Releasing $TAG." + + if [ "$TAG" != "v$(source_version)" ]; then + warn "package.json says $(source_version) but the release is $TAG. Bump package.json's version with the release so a dev build reports the same number (RELEASING.md, step 5)." + fi +} +# END:resolve-version + +# MARK:checks +# The release gate from RELEASING.md that a machine can check. +run_release_checks() { + if [ "$SKIP_CHECKS" = "1" ]; then + warn "Skipping typecheck, tests and the installer syntax check (--skip-checks)." + return 0 + fi + + step "Running the release checks" + run bun run typecheck + ok "Type-check passed." + run bun run test + ok "Tests passed." + run bash -n install.sh + ok "install.sh parses." +} +# END:checks + +# MARK:tag +# Creates the tag at HEAD, or verifies the existing one is at HEAD. +prepare_tag() { + step "Preparing the tag $TAG" + + if local_tag_exists "$TAG"; then + local tagged + tagged="$(git rev-list -n1 "$TAG")" + if [ "$tagged" != "$(git rev-parse HEAD)" ]; then + fail "Tag $TAG already exists on ${tagged:0:12}, not on HEAD ($(git rev-parse --short HEAD)). Publishing would ship the code that tag was made from. Move it with: git tag -f -a $TAG -m '$TAG'" + fi + ok "Tag $TAG already points at HEAD; reusing it." + return 0 + fi + + run git tag -a "$TAG" -m "$TAG" + if [ "$DRY_RUN" != "1" ]; then + TAG_CREATED=1 + fi + ok "Tagged HEAD as $TAG." +} +# END:tag + +# MARK:notes-file +# Writes release_notes into NOTES_FILE, which the caller cleans up. +write_notes_file() { + local compare_url="$1" + local from="" + # A dry run makes no tag, so the notes are read up to HEAD instead. + local to="$TAG" + + if ! local_tag_exists "$to"; then + to="HEAD" + fi + + from="$(previous_tag "$to")" + + if [ -n "$from" ]; then + detail "Changes since $from." + else + detail "No earlier tag, so every commit is listed." + fi + + NOTES_FILE="$(mktemp -t cobble-release-notes.XXXXXX)" + release_notes --from "$from" --to "$to" --compare-url "$compare_url" >"$NOTES_FILE" + ok "Wrote the release notes to $NOTES_FILE." +} + +cleanup_notes() { + if [ -n "$NOTES_FILE" ] && [ -f "$NOTES_FILE" ]; then + rm -f "$NOTES_FILE" + fi +} +# END:notes-file + +# MARK:push +# Pushes the branch, then the tag. A tag the remote already has is left alone, +# so a rerun after a failed release object finishes the job instead of +# failing on the push. +push_branch_and_tag() { + local remote="$1" + + step "Pushing to $remote" + run git push "$remote" "$BRANCH" + ok "Pushed $BRANCH." + + if [ -n "$(remote_tag_exists "$remote" "$TAG")" ]; then + ok "$remote already has $TAG; not pushing it again." + return 0 + fi + run git push "$remote" "refs/tags/$TAG" + ok "Pushed $TAG." +} +# END:push + +# MARK:reuse-release +# A release object that already exists is left as it is, so rerunning the +# script never fails on it and never overwrites a body someone edited. +release_already_exists() { + local description="$1" + warn "$description for $TAG already exists; leaving it untouched." + warn "Delete it first if it needs to be recreated." +} +# END:reuse-release + +print_release_summary() { + local remote="$1" + local branch="$2" + local release_url="$3" + + step "Published $TAG" + printf ' %sremote%s %s\n' "$C_DIM" "$C_RESET" "$remote ($(git remote get-url "$remote"))" + printf ' %sbranch%s %s\n' "$C_DIM" "$C_RESET" "$branch" + printf ' %stag%s %s\n' "$C_DIM" "$C_RESET" "$TAG" + if [ -n "$release_url" ]; then + printf ' %srelease%s %s\n' "$C_DIM" "$C_RESET" "$release_url" + fi + if [ "$DRY_RUN" = "1" ]; then + printf '\n%sDry run: nothing was tagged, pushed or published.%s\n' "$C_YELLOW" "$C_RESET" + fi +} diff --git a/scripts/publish-gitea.sh b/scripts/publish-gitea.sh new file mode 100755 index 0000000..99152f4 --- /dev/null +++ b/scripts/publish-gitea.sh @@ -0,0 +1,223 @@ +#!/usr/bin/env bash +# Publishes a Cobble release to the Gitea mirror. +# +# scripts/publish-gitea.sh +# +# Runs the same release gate as the GitHub script, tags HEAD with the +# package.json version, pushes main and the tag to the `gitea` remote, then +# creates the Gitea release over the API. +# +# Gitea has no release workflow, so nothing is built here: the image is built +# once by .github/workflows/release.yml and mirrored to +# git.tben.me/moduletrace/cobble. This script only publishes the source and +# the release object. +# +# The token comes from GITEA_TOKEN, or from the Gitea section of ~/.config/tea +# if the tea CLI is installed. It needs the `write:repository` scope on the +# repository, which is what the repo scope gives. +set -Eeuo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# shellcheck source=lib/publish-common.sh +. "$SCRIPT_DIR/lib/publish-common.sh" +# shellcheck source=lib/publish-release.sh +. "$SCRIPT_DIR/lib/publish-release.sh" + +REMOTE="gitea" +TEA_CONFIG="${TEA_CONFIG:-$HOME/.config/tea/config.yml}" + +usage() { + cat </dev/null 2>&1; then + python3 - "$config" "$host" <<'PY' +import re +import sys + +path, host = sys.argv[1], sys.argv[2] +text = open(path, encoding="utf-8").read() +token = None +current = None +for line in text.splitlines(): + line = line.rstrip() + if line.startswith("- name:"): + current = line.split(":", 1)[1].strip().strip('"').strip("'") + continue + match = re.match(r"\s+url:\s*(.+)$", line) + if match: + url = match.group(1).strip().strip('"').strip("'") + current = re.sub(r"^[a-z]+://", "", url) + current = current.split("/")[0].split("@")[-1] + continue + match = re.match(r"\s+token:\s*(.+)$", line) + if match and current == host: + token = match.group(1).strip().strip('"').strip("'") +if token: + print(token) +PY + return 0 + fi + + # No python: the token line of the first login block that names the host. + awk -v host="$host" ' + /- name:/ { block = $0; next } + /url:/ { block = block " " $0 } + /token:/ { + if (index(block, host) > 0) { + sub(/^[^:]*:[[:space:]]*/, "", $0) + gsub(/["'"'"']/, "", $0) + print $0 + exit + } + } + ' "$config" +} +# END:tea-token + +require_repo_root +preflight "$REMOTE" +resolve_version + +remote_url="$(git remote get-url "$REMOTE")" +parse_remote_url "$remote_url" +API_URL="${GITEA_API_URL:-https://${REMOTE_HOST}/api/v1}" +REPO_SLUG="${REMOTE_OWNER}/${REMOTE_REPO}" +COMPARE_URL="https://${REMOTE_HOST}/${REPO_SLUG}/compare/" + +step "Checking the Gitea token" +TOKEN="${GITEA_TOKEN:-}" +if [ -z "$TOKEN" ]; then + TOKEN="$(tea_token_for_host "$REMOTE_HOST" | head -n1)" + if [ -n "$TOKEN" ]; then + detail "Using the token from $TEA_CONFIG." + fi +fi +if [ -z "$TOKEN" ]; then + fail "No Gitea token. Set GITEA_TOKEN, or run 'tea login add --url https://$REMOTE_HOST --token '." +fi +ok "A token for $REMOTE_HOST is set." + +require_cmd curl "Install curl." + +run_release_checks +prepare_tag +write_notes_file "$COMPARE_URL" + +step "Publishing $TAG to Gitea ($REPO_SLUG)" +printf ' %sremote%s %s\n' "$C_DIM" "$C_RESET" "$remote_url" +printf ' %sapi%s %s\n' "$C_DIM" "$C_RESET" "$API_URL" +printf ' %snotes%s %s\n' "$C_DIM" "$C_RESET" "$NOTES_FILE" +confirm "Publish $TAG to Gitea?" + +push_branch_and_tag "$REMOTE" + +if is_dry_run; then + # Nothing is asked of the server during a dry run. + release_status="404" +else + release_status="$( + curl -s -o /dev/null -w '%{http_code}' \ + -H "Authorization: token $TOKEN" \ + -H "Accept: application/json" \ + "$API_URL/repos/$REPO_SLUG/releases/tags/$TAG" || printf '000' + )" +fi + +if [ "$release_status" = "200" ]; then + release_already_exists "The Gitea release" +else + body="$(cat "$NOTES_FILE")" + payload="$(printf '{"tag_name":"%s","target_commitish":"%s","name":"%s","body":"%s","draft":false,"prerelease":false}' \ + "$TAG" \ + "$BRANCH" \ + "$TAG" \ + "$(json_escape "$body")")" + + # MARK:create-release + if is_dry_run; then + detail "[dry run] POST $API_URL/repos/$REPO_SLUG/releases" + detail "[dry run] $payload" + api_status="201" + else + response_file="$(mktemp -t cobble-gitea-release.XXXXXX)" + api_status="$( + printf '%s' "$payload" | + curl -s -o "$response_file" -w '%{http_code}' \ + -X POST \ + -H "Authorization: token $TOKEN" \ + -H "Accept: application/json" \ + -H "Content-Type: application/json" \ + --data-binary @- \ + "$API_URL/repos/$REPO_SLUG/releases" || printf '000' + )" + fi + # END:create-release + + case "$api_status" in + 201 | 200) + ok "Created the Gitea release." + ;; + 409) + release_already_exists "The Gitea release" + ;; + *) + if [ -n "${response_file:-}" ] && [ -s "$response_file" ]; then + printf '%s\n' "$(cat "$response_file")" >&2 + fi + if [ -n "${response_file:-}" ]; then + rm -f "$response_file" + fi + fail "Gitea answered HTTP $api_status for the release." + ;; + esac + + if [ -n "${response_file:-}" ]; then + rm -f "$response_file" + fi +fi + +print_release_summary "$REMOTE" "$BRANCH" "https://${REMOTE_HOST}/${REPO_SLUG}/releases/tag/$TAG" +printf '\nThe image is built by the GitHub release; mirror it into the Gitea registry with:\n' +printf ' docker buildx imagetools create --tag git.tben.me/moduletrace/cobble:latest \\\n' +printf ' ghcr.io/moduletrace/cobble:latest\n' diff --git a/scripts/publish-github.sh b/scripts/publish-github.sh new file mode 100755 index 0000000..8a873a5 --- /dev/null +++ b/scripts/publish-github.sh @@ -0,0 +1,100 @@ +#!/usr/bin/env bash +# Publishes a Cobble release to GitHub. +# +# scripts/publish-github.sh +# +# Runs the release gate from RELEASING.md (clean tree, typecheck, tests, +# installer syntax), tags HEAD with the package.json version, pushes main and +# the tag to the `origin` remote, then creates the GitHub release. Publishing +# that release is what fires .github/workflows/release.yml, which builds the +# multi-arch image and pushes it to GHCR (and mirrors it to Gitea). +# +# Run scripts/publish-gitea.sh afterwards for the source mirror's own release +# object. There is no existing publish script in the repo: this and its Gitea +# twin are it. +set -Eeuo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +# shellcheck source=lib/publish-common.sh +. "$SCRIPT_DIR/lib/publish-common.sh" +# shellcheck source=lib/publish-release.sh +. "$SCRIPT_DIR/lib/publish-release.sh" + +REMOTE="origin" + +usage() { + cat </dev/null 2>&1 || fail "The GitHub CLI is not logged in. Run: gh auth login" + ok "gh is logged in." +fi + +run_release_checks +prepare_tag +write_notes_file "$COMPARE_URL" + +step "Publishing $TAG to GitHub ($REMOTE_OWNER/$REMOTE_REPO)" +printf ' %sremote%s %s\n' "$C_DIM" "$C_RESET" "$remote_url" +printf ' %snotes%s %s\n' "$C_DIM" "$C_RESET" "$NOTES_FILE" +confirm "Publish $TAG to GitHub and start the image build?" + +push_branch_and_tag "$REMOTE" + +if gh release view "$TAG" -R "$REMOTE_OWNER/$REMOTE_REPO" >/dev/null 2>&1; then + release_already_exists "The GitHub release" +else + # Publishing the release is what starts the image build. + run gh release create "$TAG" \ + --repo "$REMOTE_OWNER/$REMOTE_REPO" \ + --title "$TAG" \ + --notes-file "$NOTES_FILE" \ + --target "$BRANCH" \ + --latest + ok "Created the GitHub release; the release workflow is building the images." +fi + +print_release_summary "$REMOTE" "$BRANCH" "https://github.com/${REMOTE_OWNER}/${REMOTE_REPO}/releases/tag/$TAG" +printf '\nThe release workflow builds and pushes the images; follow it with:\n gh run watch\n' diff --git a/scripts/publish-tools.test.ts b/scripts/publish-tools.test.ts new file mode 100644 index 0000000..9e5b9d9 --- /dev/null +++ b/scripts/publish-tools.test.ts @@ -0,0 +1,474 @@ +import { describe, expect, test } from "bun:test"; +import path from "path"; + +const ROOT = path.join(import.meta.dir, ".."); +const COMMON = path.join(ROOT, "scripts/lib/publish-common.sh"); +const RELEASE = path.join(ROOT, "scripts/lib/publish-release.sh"); +const SCRIPTS = [ + path.join(ROOT, "scripts/publish-github.sh"), + path.join(ROOT, "scripts/publish-gitea.sh"), +]; + +/** + * Lifts one marked block out of a script, so the tests run the real publish + * helpers instead of a copy of them that could drift. + */ +async function readMarkedBlock({ + file, + name, +}: { + file: string; + name: string; +}): Promise { + const source = await Bun.file(file).text(); + const start = `# MARK:${name}\n`; + const end = `# END:${name}`; + const from = source.indexOf(start); + const to = source.indexOf(end); + + expect(from).toBeGreaterThan(-1); + expect(to).toBeGreaterThan(from); + + return source.slice(from + start.length, to); +} + +/** + * Returns a caller for a marked block. Every block that needs the logging + * helpers gets the preamble too, since `fail` lives there. + */ +async function blockCaller({ + file = COMMON, + name, +}: { + file?: string; + name: string; +}): Promise { + const [preamble, block] = await Promise.all([ + readMarkedBlock({ file: COMMON, name: "preamble" }), + readMarkedBlock({ file, name }), + ]); + + return [preamble, block].join("\n"); +} + +/** Runs a bash snippet with a trimmed PATH and returns stdout, failing on stderr. */ +async function runBash({ + script, + env, +}: { + script: string; + env?: Record; +}): Promise { + const proc = Bun.spawn({ + cmd: ["bash", "-c", script], + cwd: ROOT, + stdout: "pipe", + stderr: "pipe", + env: { ...process.env, ...env }, + }); + + const [stdout, stderr, code] = await Promise.all([ + new Response(proc.stdout).text(), + new Response(proc.stderr).text(), + proc.exited, + ]); + + expect(stderr).toBe(""); + expect(code).toBe(0); + + return stdout.trim(); +} + +/** + * A PATH with the usual tools but no jq, so the bash fallback in + * `json_escape` is the one that runs. + */ +async function pathWithoutJq(): Promise { + const dir = await Bun.$`mktemp -d`.text().then((it) => it.trim()); + const tools = ["bash", "sed", "cat", "mktemp", "git", "head", "cut", "tr", "wc", "grep", "awk"]; + + await Bun.$`mkdir -p ${dir}`; + for (const tool of tools) { + const found = Bun.which(tool); + if (found) { + await Bun.$`ln -sf ${found} ${path.join(dir, tool)}`; + } + } + + return dir; +} + +describe("publish scripts syntax", () => { + for (const script of [...SCRIPTS, COMMON, RELEASE]) { + test(`${path.relative(ROOT, script)} parses`, async () => { + const proc = Bun.spawn({ cmd: ["bash", "-n", script] }); + expect(await proc.exited).toBe(0); + }); + } + + test("both publish scripts source the shared library", async () => { + for (const script of SCRIPTS) { + const source = await Bun.file(script).text(); + + expect(source).toContain('SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"'); + expect(source).toContain('. "$SCRIPT_DIR/lib/publish-common.sh"'); + expect(source).toContain('. "$SCRIPT_DIR/lib/publish-release.sh"'); + } + }); +}); + +describe("publish scripts remote wiring", () => { + test("the GitHub script releases from origin", async () => { + const source = await Bun.file(SCRIPTS[0]!).text(); + + expect(source).toContain('REMOTE="origin"'); + expect(source).toContain("gh release create"); + }); + + test("the Gitea script releases from the gitea mirror", async () => { + const source = await Bun.file(SCRIPTS[1]!).text(); + + expect(source).toContain('REMOTE="gitea"'); + expect(source).toContain("/releases"); + // No CLI could be assumed there, so the API is called with curl. + expect(source).toContain("curl"); + }); + + test("the image is only built by the GitHub release", async () => { + const source = await Bun.file(SCRIPTS[1]!).text(); + + expect(source).toContain("The image is built by the GitHub release"); + }); +}); + +describe("publish lib version handling", () => { + test("normalizes a version with and without the v", async () => { + const block = await blockCaller({ name: "version" }); + + const out = await runBash({ + script: [ + block, + `normalize_version 0.1.2; printf '\\n'`, + `normalize_version v0.1.2; printf '\\n'`, + ].join("\n"), + }); + + expect(out.split("\n")).toEqual(["v0.1.2", "v0.1.2"]); + }); + + for (const good of ["0.1.2", "v0.1.2", "v10.20.30", "v1.2.3-rc.1"]) { + test(`accepts the release version ${good}`, async () => { + const block = await blockCaller({ name: "version" }); + + const out = await runBash({ + script: [block, `validate_version ${good}`, `printf ok`].join( + "\n", + ), + }); + + expect(out).toBe("ok"); + }); + } + + for (const bad of ["0.1", "latest", "v1", "v1.2.3.4", "v1.x.3", ""]) { + test(`refuses '${bad}'`, async () => { + const block = await blockCaller({ name: "version" }); + const proc = Bun.spawn({ + cmd: ["bash", "-c", [block, `validate_version '${bad}'`].join("\n")], + cwd: ROOT, + stdout: "pipe", + stderr: "pipe", + }); + + // An empty version would pass nothing to validate and is caught + // by the caller, so it is skipped here. + if (bad === "") { + await proc.exited; + return; + } + + expect(await proc.exited).not.toBe(0); + const stderr = await new Response(proc.stderr).text(); + expect(stderr).toContain("is not a release version"); + }); + } +}); + +describe("publish lib remote URLs", () => { + const cases = [ + { + url: "git@github.com:Moduletrace/cobble.git", + host: "github.com", + owner: "Moduletrace", + repo: "cobble", + }, + { + url: "git@git.tben.me:Moduletrace/cobble.git", + host: "git.tben.me", + owner: "Moduletrace", + repo: "cobble", + }, + { + url: "ssh://git@git.tben.me:222/Moduletrace/cobble.git", + host: "git.tben.me:222", + owner: "Moduletrace", + repo: "cobble", + }, + { + url: "https://github.com/Moduletrace/cobble.git", + host: "github.com", + owner: "Moduletrace", + repo: "cobble", + }, + { + url: "https://git.tben.me/Moduletrace/cobble", + host: "git.tben.me", + owner: "Moduletrace", + repo: "cobble", + }, + ]; + + for (const item of cases) { + test(`reads ${item.url}`, async () => { + const block = await blockCaller({ name: "remote-url" }); + + const out = await runBash({ + script: [ + block, + `parse_remote_url '${item.url}'`, + `printf '%s %s %s' "$REMOTE_HOST" "$REMOTE_OWNER" "$REMOTE_REPO"`, + ].join("\n"), + }); + + expect(out).toBe(`${item.host} ${item.owner} ${item.repo}`); + }); + } + + for (const bad of ["", "not-a-url", "git@host:"]) { + test(`refuses '${bad}'`, async () => { + const block = await blockCaller({ name: "remote-url" }); + const proc = Bun.spawn({ + cmd: ["bash", "-c", [block, `parse_remote_url '${bad}'`].join("\n")], + cwd: ROOT, + stdout: "pipe", + stderr: "pipe", + }); + + expect(await proc.exited).not.toBe(0); + }); + } +}); + +describe("publish lib release notes", () => { + test("lists the commits and the compare link", async () => { + const block = await blockCaller({ name: "notes" }); + + const out = await runBash({ + script: [ + block, + `ROOT="$(git rev-parse --show-toplevel)"`, + `release_notes --from v0.1.1 --to v0.1.2 --compare-url https://example.test/compare/`, + ].join("\n"), + }); + + expect(out).toStartWith("## Changes"); + expect(out).toContain("**Full changelog**: https://example.test/compare/"); + expect(out).toContain("- "); + }); + + test("never overflows the commit cap", async () => { + const block = await blockCaller({ name: "notes" }); + + const out = await runBash({ + script: [ + block, + `ROOT="$(git rev-parse --show-toplevel)"`, + `RELEASE_NOTES_MAX_COMMITS=2`, + `release_notes --to v0.1.2`, + ].join("\n"), + }); + + expect(out).toContain("_Showing the newest 2 of"); + }); +}); + +describe("publish lib json escaping", () => { + const sample = 'a\nb\t"c" \\ é'; + + test("escapes with jq", async () => { + const block = await blockCaller({ name: "json-escape" }); + + const out = await runBash({ + script: [ + block, + `printf '{"body":"%s"}' "$(json_escape '${sample}')"`, + ].join("\n"), + }); + + // The escaped text has to survive a real JSON parse. + const parsed = JSON.parse(out) as { body: string }; + expect(parsed.body).toBe("a\nb\t\"c\" \\ é"); + }); + + test("escapes without jq installed", async () => { + const block = await blockCaller({ name: "json-escape" }); + const dir = await pathWithoutJq(); + + try { + const out = await runBash({ + script: [ + block, + `printf '{"body":"%s"}' "$(json_escape '${sample}')"`, + ].join("\n"), + env: { PATH: dir }, + }); + + const parsed = JSON.parse(out) as { body: string }; + expect(parsed.body).toBe("a\nb\t\"c\" \\ é"); + } finally { + await Bun.$`rm -rf ${dir}`; + } + }); +}); + +describe("publish lib tag handling", () => { + test("an existing tag on HEAD is reused", async () => { + const block = await blockCaller({ name: "git-state" }); + const dir = await Bun.$`mktemp -d`.text().then((it) => it.trim()); + + try { + const out = await runBash({ + script: [ + `cd ${dir}`, + `git init -q .`, + `git config user.email test@example.test`, + `git config user.name Test`, + `printf x > file`, + `git add file`, + `git commit -qm one`, + block, + `ensure_tag v9.9.9`, + `ensure_tag v9.9.9`, + `git tag --list`, + ].join("\n"), + }); + + expect(out.split("\n").filter((line) => line === "v9.9.9")).toHaveLength( + 1, + ); + } finally { + await Bun.$`rm -rf ${dir}`; + } + }); + + test("a dirty worktree stops the publish", async () => { + const block = await blockCaller({ name: "git-state" }); + const dir = await Bun.$`mktemp -d`.text().then((it) => it.trim()); + + try { + const proc = Bun.spawn({ + cmd: [ + "bash", + "-c", + [ + `cd ${dir}`, + `git init -q .`, + `git config user.email test@example.test`, + `git config user.name Test`, + `printf x > file`, + `git add file`, + `git commit -qm one`, + `printf y >> file`, + block, + `require_clean_worktree`, + ].join("\n"), + ], + stdout: "pipe", + stderr: "pipe", + }); + + expect(await proc.exited).not.toBe(0); + expect(await new Response(proc.stderr).text()).toContain( + "uncommitted changes", + ); + } finally { + await Bun.$`rm -rf ${dir}`; + } + }); +}); + +describe("publish lib helpers", () => { + test("require_cmd explains a missing binary", async () => { + const block = await blockCaller({ name: "require-cmd" }); + const proc = Bun.spawn({ + cmd: [ + "bash", + "-c", + [block, `require_cmd definitely-not-installed "Install it."`].join( + "\n", + ), + ], + cwd: ROOT, + stdout: "pipe", + stderr: "pipe", + }); + + expect(await proc.exited).not.toBe(0); + expect(await new Response(proc.stderr).text()).toContain( + "Missing 'definitely-not-installed'. Install it.", + ); + }); + + test("tea_token_for_host reads the host's token", async () => { + const block = await blockCaller({ + file: SCRIPTS[1]!, + name: "tea-token", + }); + const dir = await Bun.$`mktemp -d`.text().then((it) => it.trim()); + const config = path.join(dir, "config.yml"); + + try { + await Bun.write( + config, + [ + "logins:", + " - name: github.com", + " url: https://github.com", + " token: github-token", + " - name: git.tben.me", + " url: https://git.tben.me", + " token: gitea-token", + "", + ].join("\n"), + ); + + const out = await runBash({ + script: [ + block, + `tea_token_for_host git.tben.me '${config}'`, + ].join("\n"), + }); + + expect(out).toBe("gitea-token"); + } finally { + await Bun.$`rm -rf ${dir}`; + } + }); + + test("tea_token_for_host is quiet without a config", async () => { + const block = await blockCaller({ + file: SCRIPTS[1]!, + name: "tea-token", + }); + + const out = await runBash({ + script: [ + block, + `tea_token_for_host git.tben.me /nonexistent/cobble-test`, + `printf done`, + ].join("\n"), + }); + + expect(out).toBe("done"); + }); +});