bbf5eb24de885ab68d6f2d0fa88eb16e1327aca0
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Abdulmuizz Hamzat
|
8af0675895 |
fix: minio bucket seed race (#820)
* fix(workflows): retry MinIO bucket creation to survive post-boot format window A freshly duplicated cio-minio's HTTP listener accepts connections before it finishes formatting its storage backend (confirmed via its own boot logs: "Formatting 1st pool, 1 set(s), 1 drives per set."), so a bucket-creation call landing in that window can transiently fail with a misleading "specified bucket does not exist" response even though auth and connectivity (mc alias set) already succeeded. Retry bucket creation up to 10 times with a 5s backoff instead of failing the whole run on what's usually just a not-ready-yet response. * fix(workflows): truncate env name slug to fit Railway's length limit railway environment new failed with "Error in name - Invalid input" for a 48-char name (pr-cursor-notes-workspace-feedback-6d09-0e6aa7c9) derived from a long auto-generated branch name. Railway docs cite a 32-char limit for service names; environment names hit the same wall. The collision-resistant hash suffix (added for the earlier CodeRabbit finding) is what actually guarantees uniqueness -- the slug is only there for readability -- so cap the slug at 18 chars instead of dropping the hash. pr-<slug><=18>-<hash8> stays at or under 30 chars. * fix(workflows): retry mc anonymous set for the same boot-race window The mb retry loop worked as intended in the latest test run (attempt 1 failed on all three buckets during cio-minio's post-boot format window, attempt 2 succeeded), but the very next call -- anonymous set download on cio/media -- hit the identical "specified bucket does not exist" error right after mb had just succeeded. Same underlying race, one command later. Wrap it in the same retry pattern. * update workflow * fix: silen error on db:seed * update workflow * update workflow * update workflow |
||
|
Abdulmuizz HamzatandRotimi Best
|
73f39da54a |
feat(workflows): add Railway preview environment and teardown workflows (#766)
* feat(workflows): add Railway preview environment and teardown workflows * fix(workflows): enhance security by binding branch names to env variables in Railway workflows * fix(workflows): ensure preview URLs are live by blocking on service healthchecks during deployment * temp: auto-run railway-preview.yml on push for demo recording workflow_dispatch requires the file on the default branch before "Run workflow" is available; add a scoped push trigger + input fallbacks so this branch can produce a real run to record. Revert before merge. * fix(workflows): pass --workspace to railway link for non-interactive CI Railway CLI now requires --workspace when running `railway link` non-interactively; without it the preview workflow fails at the link step. Defaults to the "cio projects" workspace the staging project lives in, overridable via a RAILWAY_WORKSPACE repo variable. * fix(workflows): read RAILWAY_PROJECT_ID from secrets, not vars It's configured as a repository secret (has been for weeks), but the workflow read it via the vars context, which is a separate namespace and always resolved empty -> railway link failed with Project "" not found. Read it via secrets to match how it's actually stored. * fix(workflows): pass --workspace to environment new/delete, stop masking failures railway environment new needed --workspace in non-interactive mode too (same as railway link), but its failure was unconditionally swallowed by `|| echo`, so the workflow proceeded to "Link" as if the preview environment existed when it never got created. Add --workspace to both environment new and environment delete, and only swallow the "already exists" case -- any other failure now fails the run where it actually happened instead of surfacing as a confusing error two steps later. * fix(workflows): stop passing --project/--workspace to railway environment new/delete Verified against the actual CLI (railway environment new --help / delete --help): those subcommands take NO --project/--workspace/--environment flags at all, and were silently rejecting them, which is why environment creation kept failing and getting masked by the fallback echo. Only `railway link` accepts --project/--workspace/--environment, so link to the base project first, then create/delete environments by name, then re-link to switch into the new one for deploy. * fix(workflows): force path-style addressing for mc against Railway's MinIO domain mc alias set defaults to --path auto, which can pick virtual-host-style bucket addressing (bucket.host/key). Railway's edge only routes the exact configured domain, not wildcard bucket subdomains, so mb then fails with a misleading "specified bucket does not exist" even though the alias/auth check (a GET) succeeds. Force --path on, matching the app's own OBJECT_STORAGE_FORCE_PATH_STYLE=true for this same endpoint. * chore: retrigger preview workflow after fixing cio-minio start command staging's cio-minio now has the correct Custom Start Command, and the stale pr-chore-staging-setup preview env (duplicated before that fix) has been deleted. Empty commit to trigger a fresh deploy run. * fix(workflows): generate cio-minio's public domain in the preview env Confirmed staging's cio-minio has its own domain, but environment duplication doesn't carry it to the new preview environment, so RAILWAY_PUBLIC_DOMAIN came back empty there and produced an invalid "https://" alias URL. Generate one in CI, same as the existing cio-dashboard domain step, pinned to port 9000 since the minio/minio image exposes both the S3 API (9000) and console (9001). * chore(workflows): revert temp push-trigger testing scaffold, trim comments Remove the push trigger and || fallback expressions added solely to get a run before this file exists on main (workflow_dispatch requires that for the "Run workflow" UI). Back to workflow_dispatch-only, with comments trimmed from a debugging narrative down to the durable facts future readers need. All real fixes found during testing (link --workspace, secrets vs vars, environment new/delete flag handling, mc --path on, cio-minio domain generation) are unchanged. * fix(workflows): fix teardown link/secrets bugs, harden domain reads railway-preview-teardown.yml had the same class of bugs already found and fixed in railway-preview.yml during testing: - RAILWAY_PROJECT_ID read from vars.* while the deploy workflow (and the actual GitHub config) uses secrets.* — always resolved empty. - environment delete was called without linking first, and passed an unsupported --project flag; the resulting CLI error was swallowed by `|| echo`, so teardown silently no-opped on every run and every preview environment leaked indefinitely. Also hardens railway-preview.yml against a repeat of the cio-minio domain bug: two other "generate domain, then read it back" spots (cio-minio bucket seeding, cio-dashboard's final URL report) had the same unguarded `|| true` swallow with no check that the resulting host is non-empty. Both now fail loudly instead of silently proceeding — the dashboard one matters most since it's the final step and would otherwise report success with a broken link. Resolves the now-verified SPIKE comment on environment duplication, and trims a few comments that restated code instead of explaining non-obvious behavior. * fix(workflows): address CodeRabbit findings on preview lifecycle - Unify concurrency groups between deploy and teardown (was railway-preview-<branch> vs railway-preview-teardown-<branch>, so they never coordinated). A PR-close teardown and a manual deploy for the same branch could otherwise race, letting a deploy recreate an environment right after teardown deleted it. - Pin the Railway CLI to 5.30.1 (the version verified against all session) in both workflows, instead of an unpinned `npm install -g` that could silently pick up different undocumented CLI behavior later, same class of bug this session hit repeatedly. - Append a short hash of the original branch string to the derived environment name in both workflows. The slug alone is lossy (/, _, and . all collapse to the same separator), so e.g. "foo/bar" and "foo_bar" previously collided onto the same environment name. * fix(workflows): scope teardown job to same-repo PRs only The header comment already documented that this only applies to same-repo branches (secrets are withheld from fork PR runs), but nothing enforced it -- the job ran on every closed PR including forks, where the empty RAILWAY_API_TOKEN/RAILWAY_PROJECT_ID made railway link fail with an auth error and show a red run for a PR that never had a preview environment to clean up. * docs: migrate RAILWAY_PREVIEW.md to PREVIEW_ENV.md Replace docker/docs/RAILWAY_PREVIEW.md with PREVIEW_ENV.md at the repo root, restructured per PR review feedback around the two things a reader actually needs: how to deploy to staging, and how to deploy a PR to a preview environment. Drops the one-time Railway project setup section (not needed on this page, per review). --------- Co-authored-by: Rotimi Best <[email protected]> |