feat(workflows): add Railway preview environment and teardown workflows (#766)
* feat(workflows): add Railway preview environment and teardown workflows * fix(workflows): enhance security by binding branch names to env variables in Railway workflows * fix(workflows): ensure preview URLs are live by blocking on service healthchecks during deployment * temp: auto-run railway-preview.yml on push for demo recording workflow_dispatch requires the file on the default branch before "Run workflow" is available; add a scoped push trigger + input fallbacks so this branch can produce a real run to record. Revert before merge. * fix(workflows): pass --workspace to railway link for non-interactive CI Railway CLI now requires --workspace when running `railway link` non-interactively; without it the preview workflow fails at the link step. Defaults to the "cio projects" workspace the staging project lives in, overridable via a RAILWAY_WORKSPACE repo variable. * fix(workflows): read RAILWAY_PROJECT_ID from secrets, not vars It's configured as a repository secret (has been for weeks), but the workflow read it via the vars context, which is a separate namespace and always resolved empty -> railway link failed with Project "" not found. Read it via secrets to match how it's actually stored. * fix(workflows): pass --workspace to environment new/delete, stop masking failures railway environment new needed --workspace in non-interactive mode too (same as railway link), but its failure was unconditionally swallowed by `|| echo`, so the workflow proceeded to "Link" as if the preview environment existed when it never got created. Add --workspace to both environment new and environment delete, and only swallow the "already exists" case -- any other failure now fails the run where it actually happened instead of surfacing as a confusing error two steps later. * fix(workflows): stop passing --project/--workspace to railway environment new/delete Verified against the actual CLI (railway environment new --help / delete --help): those subcommands take NO --project/--workspace/--environment flags at all, and were silently rejecting them, which is why environment creation kept failing and getting masked by the fallback echo. Only `railway link` accepts --project/--workspace/--environment, so link to the base project first, then create/delete environments by name, then re-link to switch into the new one for deploy. * fix(workflows): force path-style addressing for mc against Railway's MinIO domain mc alias set defaults to --path auto, which can pick virtual-host-style bucket addressing (bucket.host/key). Railway's edge only routes the exact configured domain, not wildcard bucket subdomains, so mb then fails with a misleading "specified bucket does not exist" even though the alias/auth check (a GET) succeeds. Force --path on, matching the app's own OBJECT_STORAGE_FORCE_PATH_STYLE=true for this same endpoint. * chore: retrigger preview workflow after fixing cio-minio start command staging's cio-minio now has the correct Custom Start Command, and the stale pr-chore-staging-setup preview env (duplicated before that fix) has been deleted. Empty commit to trigger a fresh deploy run. * fix(workflows): generate cio-minio's public domain in the preview env Confirmed staging's cio-minio has its own domain, but environment duplication doesn't carry it to the new preview environment, so RAILWAY_PUBLIC_DOMAIN came back empty there and produced an invalid "https://" alias URL. Generate one in CI, same as the existing cio-dashboard domain step, pinned to port 9000 since the minio/minio image exposes both the S3 API (9000) and console (9001). * chore(workflows): revert temp push-trigger testing scaffold, trim comments Remove the push trigger and || fallback expressions added solely to get a run before this file exists on main (workflow_dispatch requires that for the "Run workflow" UI). Back to workflow_dispatch-only, with comments trimmed from a debugging narrative down to the durable facts future readers need. All real fixes found during testing (link --workspace, secrets vs vars, environment new/delete flag handling, mc --path on, cio-minio domain generation) are unchanged. * fix(workflows): fix teardown link/secrets bugs, harden domain reads railway-preview-teardown.yml had the same class of bugs already found and fixed in railway-preview.yml during testing: - RAILWAY_PROJECT_ID read from vars.* while the deploy workflow (and the actual GitHub config) uses secrets.* — always resolved empty. - environment delete was called without linking first, and passed an unsupported --project flag; the resulting CLI error was swallowed by `|| echo`, so teardown silently no-opped on every run and every preview environment leaked indefinitely. Also hardens railway-preview.yml against a repeat of the cio-minio domain bug: two other "generate domain, then read it back" spots (cio-minio bucket seeding, cio-dashboard's final URL report) had the same unguarded `|| true` swallow with no check that the resulting host is non-empty. Both now fail loudly instead of silently proceeding — the dashboard one matters most since it's the final step and would otherwise report success with a broken link. Resolves the now-verified SPIKE comment on environment duplication, and trims a few comments that restated code instead of explaining non-obvious behavior. * fix(workflows): address CodeRabbit findings on preview lifecycle - Unify concurrency groups between deploy and teardown (was railway-preview-<branch> vs railway-preview-teardown-<branch>, so they never coordinated). A PR-close teardown and a manual deploy for the same branch could otherwise race, letting a deploy recreate an environment right after teardown deleted it. - Pin the Railway CLI to 5.30.1 (the version verified against all session) in both workflows, instead of an unpinned `npm install -g` that could silently pick up different undocumented CLI behavior later, same class of bug this session hit repeatedly. - Append a short hash of the original branch string to the derived environment name in both workflows. The slug alone is lossy (/, _, and . all collapse to the same separator), so e.g. "foo/bar" and "foo_bar" previously collided onto the same environment name. * fix(workflows): scope teardown job to same-repo PRs only The header comment already documented that this only applies to same-repo branches (secrets are withheld from fork PR runs), but nothing enforced it -- the job ran on every closed PR including forks, where the empty RAILWAY_API_TOKEN/RAILWAY_PROJECT_ID made railway link fail with an auth error and show a red run for a PR that never had a preview environment to clean up. * docs: migrate RAILWAY_PREVIEW.md to PREVIEW_ENV.md Replace docker/docs/RAILWAY_PREVIEW.md with PREVIEW_ENV.md at the repo root, restructured per PR review feedback around the two things a reader actually needs: how to deploy to staging, and how to deploy a PR to a preview environment. Drops the one-time Railway project setup section (not needed on this page, per review). --------- Co-authored-by: Rotimi Best <[email protected]>
This commit is contained in:
Abdulmuizz Hamzat
co-authored by
Rotimi Best
Rotimi Best
parent
c01b8dea3c
commit
73f39da54a
@@ -0,0 +1,61 @@
|
||||
name: Railway Preview Teardown
|
||||
|
||||
# Auto-delete a preview environment created by railway-preview.yml when its PR closes,
|
||||
# so manually-spun previews don't linger and accrue Railway cost. (Native Railway PR
|
||||
# environments clean themselves up; this covers the pr-<branch> envs made by the manual
|
||||
# workflow.) Deletion is best-effort — if no matching env exists, the run is a no-op.
|
||||
#
|
||||
# Uses the same RAILWAY_API_TOKEN + RAILWAY_PROJECT_ID secrets as the deploy workflow.
|
||||
# Note: secrets are unavailable to PRs from forks, so this only tears down previews for
|
||||
# same-repo branches (which is where manual previews come from anyway).
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [closed]
|
||||
|
||||
concurrency:
|
||||
group: railway-preview-${{ github.event.pull_request.head.ref }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
teardown:
|
||||
runs-on: ubuntu-latest
|
||||
if: github.event.pull_request.head.repo.full_name == github.repository
|
||||
env:
|
||||
RAILWAY_API_TOKEN: ${{ secrets.RAILWAY_API_TOKEN }}
|
||||
RAILWAY_PROJECT_ID: ${{ secrets.RAILWAY_PROJECT_ID }}
|
||||
RAILWAY_WORKSPACE: ${{ vars.RAILWAY_WORKSPACE || 'cio projects' }}
|
||||
BASE_ENV: ${{ vars.RAILWAY_BASE_ENV || 'staging' }}
|
||||
steps:
|
||||
- name: Install Railway CLI
|
||||
run: npm install -g @railway/[email protected]
|
||||
|
||||
- name: Link to base project
|
||||
# `railway environment delete` takes no --project/--workspace/--environment
|
||||
# flags — only `railway link` does, so it must run first to establish that
|
||||
# context (same requirement as railway-preview.yml).
|
||||
run: |
|
||||
railway link --project "$RAILWAY_PROJECT_ID" --workspace "$RAILWAY_WORKSPACE" --environment "$BASE_ENV"
|
||||
|
||||
- name: Delete the preview environment
|
||||
# Mirror the name derivation in railway-preview.yml so the env name matches.
|
||||
# HEAD_REF is bound via env (not interpolated into the script) so a malicious
|
||||
# branch name can never be executed as shell — Actions expands ${{ }} as text
|
||||
# BEFORE bash parses it, so a raw `${{ ...head.ref }}` here is a script-injection hole.
|
||||
env:
|
||||
HEAD_REF: ${{ github.event.pull_request.head.ref }}
|
||||
run: |
|
||||
slug=$(echo "$HEAD_REF" | tr '[:upper:]/_.' '[:lower:]---' | sed 's/[^a-z0-9-]//g' | sed 's/--*/-/g; s/^-//; s/-$//')
|
||||
hash=$(echo -n "$HEAD_REF" | sha256sum | cut -c1-8)
|
||||
env_name="pr-${slug}-${hash}"
|
||||
echo "Attempting to delete environment: ${env_name}"
|
||||
if ! output=$(railway environment delete "${env_name}" --yes 2>&1); then
|
||||
echo "$output"
|
||||
if echo "$output" | grep -qi "not found"; then
|
||||
echo "No matching preview environment (or already deleted) — nothing to do."
|
||||
else
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "$output"
|
||||
fi
|
||||
@@ -0,0 +1,213 @@
|
||||
name: Railway Preview Environment
|
||||
|
||||
# Manually spin up (or tear down) a short-lived Railway preview environment for a
|
||||
# branch/PR, without waiting for a merge. Railway builds each app service FROM SOURCE
|
||||
# (docker/Dockerfile.*) so the preview runs the branch's exact code — this is the
|
||||
# controlled counterpart to Railway's native PR environments.
|
||||
#
|
||||
# Prerequisites (one-time):
|
||||
# - A Railway project (e.g. "classroomio-staging") whose `staging` environment is
|
||||
# fully configured (Postgres, Redis, cio-minio, cio-api, cio-dashboard, cio-jobs).
|
||||
# - Repo/Org secret RAILWAY_API_TOKEN -> a Railway ACCOUNT or TEAM token (NOT a
|
||||
# project token; env-scoped project tokens can't create environments).
|
||||
# - Repo/Org secret RAILWAY_PROJECT_ID -> the target project's id.
|
||||
# - Repo/Org variable RAILWAY_WORKSPACE -> the Railway workspace the project lives in
|
||||
# (default: "cio projects"); required by `railway link` in non-interactive mode.
|
||||
# - Repo/Org variable RAILWAY_BASE_ENV -> the env to duplicate from (default: staging).
|
||||
#
|
||||
# The DB needs no explicit migration step here: the cio-api image self-migrates on boot
|
||||
# (docker/entrypoint-api.sh -> `pnpm --filter @cio/db db:setup`, which runs drizzle
|
||||
# migrations + essential seed). The optional `seed` input adds the full demo dataset.
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
branch:
|
||||
description: "Branch (or PR head ref) to deploy into the preview env"
|
||||
required: true
|
||||
type: string
|
||||
action:
|
||||
description: "deploy = create/update the preview; destroy = delete it"
|
||||
required: true
|
||||
type: choice
|
||||
default: deploy
|
||||
options:
|
||||
- deploy
|
||||
- destroy
|
||||
seed:
|
||||
description: "Seed the full demo dataset ([email protected] etc.) after deploy"
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
|
||||
# One run per branch at a time; a newer run supersedes an in-flight one.
|
||||
concurrency:
|
||||
group: railway-preview-${{ inputs.branch }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
RAILWAY_API_TOKEN: ${{ secrets.RAILWAY_API_TOKEN }}
|
||||
RAILWAY_PROJECT_ID: ${{ secrets.RAILWAY_PROJECT_ID }}
|
||||
RAILWAY_WORKSPACE: ${{ vars.RAILWAY_WORKSPACE || 'cio projects' }}
|
||||
BASE_ENV: ${{ vars.RAILWAY_BASE_ENV || 'staging' }}
|
||||
# The three app services that build from source. Postgres/Redis/cio-minio come along
|
||||
# via environment duplication and need no `railway up`.
|
||||
APP_SERVICES: "cio-api cio-dashboard cio-jobs"
|
||||
|
||||
jobs:
|
||||
preview:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout target branch
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.branch }}
|
||||
|
||||
- name: Install Railway CLI
|
||||
run: npm install -g @railway/[email protected]
|
||||
|
||||
- name: Derive a safe environment name
|
||||
id: env
|
||||
# BRANCH is bound via env (not interpolated into the script) so an attacker-supplied
|
||||
# value can't be executed as shell. "feat/foo_bar" -> "pr-feat-foo-bar-<hash>" — the
|
||||
# slug alone is lossy (e.g. "foo/bar" and "foo_bar" both slug to "foo-bar"), so a
|
||||
# hash of the original branch string disambiguates collisions.
|
||||
env:
|
||||
BRANCH: ${{ inputs.branch }}
|
||||
run: |
|
||||
slug=$(echo "$BRANCH" | tr '[:upper:]/_.' '[:lower:]---' | sed 's/[^a-z0-9-]//g' | sed 's/--*/-/g; s/^-//; s/-$//')
|
||||
hash=$(echo -n "$BRANCH" | sha256sum | cut -c1-8)
|
||||
echo "name=pr-${slug}-${hash}" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Link to base project
|
||||
# `railway environment new`/`delete` take no --project/--workspace/--environment
|
||||
# flags — only `railway link` does, so it must run first to establish that
|
||||
# context for every other command in this job.
|
||||
run: |
|
||||
railway link --project "$RAILWAY_PROJECT_ID" --workspace "$RAILWAY_WORKSPACE" --environment "$BASE_ENV"
|
||||
|
||||
# ---------------------------------------------------------------- destroy
|
||||
- name: Destroy preview environment
|
||||
if: ${{ inputs.action == 'destroy' }}
|
||||
env:
|
||||
ENV_NAME: ${{ steps.env.outputs.name }}
|
||||
run: |
|
||||
railway environment delete "$ENV_NAME" --yes
|
||||
echo "Deleted environment $ENV_NAME." >> "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
# ---------------------------------------------------------------- deploy
|
||||
- name: Create preview environment (duplicate base)
|
||||
if: ${{ inputs.action == 'deploy' }}
|
||||
# `--duplicate` copies service config and variables but not public domains (see
|
||||
# domain-generation steps below). Only "already exists" is swallowed here — any
|
||||
# other failure must surface, not silently continue into a "Link" step that can't
|
||||
# find an environment that was never created.
|
||||
env:
|
||||
ENV_NAME: ${{ steps.env.outputs.name }}
|
||||
run: |
|
||||
if ! output=$(railway environment new "$ENV_NAME" --duplicate "$BASE_ENV" 2>&1); then
|
||||
echo "$output"
|
||||
if echo "$output" | grep -qi "already exists"; then
|
||||
echo "Environment may already exist; continuing."
|
||||
else
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "$output"
|
||||
fi
|
||||
|
||||
- name: Link project + preview environment
|
||||
if: ${{ inputs.action == 'deploy' }}
|
||||
env:
|
||||
ENV_NAME: ${{ steps.env.outputs.name }}
|
||||
run: |
|
||||
railway link --project "$RAILWAY_PROJECT_ID" --workspace "$RAILWAY_WORKSPACE" --environment "$ENV_NAME"
|
||||
|
||||
- name: Pin the Dockerfile per service
|
||||
if: ${{ inputs.action == 'deploy' }}
|
||||
# `railway up` uploads local source but does NOT carry the dashboard's "Dockerfile
|
||||
# Path" build setting, so without this Railway falls back to Railpack and builds
|
||||
# the whole monorepo as a generic Node app. (Same gotcha documented in the setup guide.)
|
||||
run: |
|
||||
railway variables --service cio-api --set "RAILWAY_DOCKERFILE_PATH=docker/Dockerfile.api"
|
||||
railway variables --service cio-dashboard --set "RAILWAY_DOCKERFILE_PATH=docker/Dockerfile.dashboard"
|
||||
railway variables --service cio-jobs --set "RAILWAY_DOCKERFILE_PATH=docker/Dockerfile.jobs"
|
||||
|
||||
- name: Seed MinIO buckets (fresh volume starts empty)
|
||||
if: ${{ inputs.action == 'deploy' }}
|
||||
# Railway has no minio-init sidecar and a duplicated env's MinIO volume is empty,
|
||||
# so create the buckets the app expects. Root creds carry over via duplication,
|
||||
# but the public domain doesn't, so generate one here (same as "Report the preview
|
||||
# URL" does for cio-dashboard). `--ignore-existing` keeps re-deploys safe.
|
||||
run: |
|
||||
# --port 9000: the minio/minio image exposes both 9000 (S3 API) and 9001
|
||||
# (console); pin it so domain generation doesn't land on the wrong one.
|
||||
railway domain --service cio-minio --port 9000 || true
|
||||
# `cut -d= -f2-` (not `awk -F= '{print $2}'`) so a value containing '=' — common in
|
||||
# base64/auto-generated secrets like MINIO_ROOT_PASSWORD — isn't silently truncated.
|
||||
MINIO_HOST=$(railway variables --service cio-minio --kv | grep '^RAILWAY_PUBLIC_DOMAIN=' | cut -d= -f2-)
|
||||
MINIO_USER=$(railway variables --service cio-minio --kv | grep '^MINIO_ROOT_USER=' | cut -d= -f2-)
|
||||
MINIO_PASS=$(railway variables --service cio-minio --kv | grep '^MINIO_ROOT_PASSWORD=' | cut -d= -f2-)
|
||||
if [ -z "$MINIO_HOST" ]; then
|
||||
echo "::error::cio-minio has no public domain — 'railway domain' may have failed."
|
||||
exit 1
|
||||
fi
|
||||
# Verify mc against MinIO's published SHA256 before executing it.
|
||||
curl -sSL https://dl.min.io/client/mc/release/linux-amd64/mc -o mc
|
||||
curl -sSL https://dl.min.io/client/mc/release/linux-amd64/mc.sha256sum -o mc.sha256sum
|
||||
echo "$(cut -d' ' -f1 mc.sha256sum) mc" | sha256sum -c -
|
||||
chmod +x mc
|
||||
# --path on forces path-style bucket addressing (host/bucket/key), same as the
|
||||
# app's own OBJECT_STORAGE_FORCE_PATH_STYLE=true — this Railway domain doesn't
|
||||
# route wildcard bucket subdomains, so virtual-host style (mc's "auto" default)
|
||||
# fails bucket creation with a misleading "does not exist" error.
|
||||
./mc alias set cio "https://${MINIO_HOST}" "$MINIO_USER" "$MINIO_PASS" --path on
|
||||
./mc mb cio/videos cio/documents cio/media --ignore-existing
|
||||
./mc anonymous set download cio/media
|
||||
|
||||
- name: Deploy each app service from this branch
|
||||
if: ${{ inputs.action == 'deploy' }}
|
||||
# `--ci` (not `--detach`) blocks until each service's deploy/healthcheck succeeds,
|
||||
# so the URL printed later is actually live. Trade-off: builds run sequentially.
|
||||
run: |
|
||||
for svc in $APP_SERVICES; do
|
||||
echo "::group::railway up $svc"
|
||||
railway up --service "$svc" --ci
|
||||
echo "::endgroup::"
|
||||
done
|
||||
|
||||
- name: Seed the full demo dataset (optional)
|
||||
if: ${{ inputs.action == 'deploy' && inputs.seed }}
|
||||
# SPIKE: `railway run` injects the service's env vars and runs the command ON THIS
|
||||
# RUNNER, so it needs a DB URL reachable from GitHub (Railway's public TCP proxy,
|
||||
# DATABASE_PUBLIC_URL) — NOT the private railway.internal one. If cio-api only
|
||||
# exposes a private DATABASE_URL, add DATABASE_PUBLIC_URL to this step, or drop
|
||||
# this step and rely on the api boot-time db:setup (migrations + essential seed).
|
||||
run: |
|
||||
corepack enable
|
||||
railway run --service cio-api -- pnpm --filter @cio/db db:setup:seed || \
|
||||
echo "Demo seed failed (likely DB not reachable from runner) — migrations + essential seed still ran on api boot."
|
||||
|
||||
- name: Report the preview URL
|
||||
if: ${{ inputs.action == 'deploy' }}
|
||||
# BRANCH / ENV_NAME bound via env (never interpolated into the script).
|
||||
env:
|
||||
BRANCH: ${{ inputs.branch }}
|
||||
ENV_NAME: ${{ steps.env.outputs.name }}
|
||||
run: |
|
||||
railway domain --service cio-dashboard || true
|
||||
URL=$(railway variables --service cio-dashboard --kv | grep '^RAILWAY_PUBLIC_DOMAIN=' | cut -d= -f2-)
|
||||
if [ -z "$URL" ]; then
|
||||
echo "::error::cio-dashboard has no public domain — 'railway domain' may have failed."
|
||||
exit 1
|
||||
fi
|
||||
{
|
||||
echo "### Railway preview ready :rocket:"
|
||||
echo ""
|
||||
echo "- Environment: \`$ENV_NAME\` (branch \`$BRANCH\`)"
|
||||
echo "- Dashboard: https://${URL}"
|
||||
echo ""
|
||||
echo "> All services passed their Railway deploy/healthcheck before this link was printed, so it should be live now."
|
||||
echo ""
|
||||
echo "Tear down with this workflow using **action = destroy** and the same branch."
|
||||
} >> "$GITHUB_STEP_SUMMARY"
|
||||
Reference in New Issue
Block a user