E2E lifecycle reports
Where to open reports (GitHub Actions)
- Open the lifecycle workflow run (private Hub lifecycle (Stripe test) or public Hub lifecycle dispatch).
- Summary tab — markdown table of tests, durations, Playwright annotations (site id, plan, Phase B notes, checkout fallback, etc.), and when configured an R2 stable report link.
- Artifacts (private Hub lifecycle (Stripe test) only) — download
playwright-report-{plan}-{run_id}:- Unzip and open
playwright-report/index.htmlfor the full HTML report. test-results/holds traces, screenshots, and video on failure (retain-on-failure).
- Unzip and open
The private workflow uploads artifacts on every run (if: always()) and keeps them 30 days, including green runs. The public dispatch uploads no GitHub artifact (#1169). Anyone with a GitHub account can download a public repo's artifacts, and traces hold request headers, session cookies and sign-in codes. Its report, with traces, goes only to private R2 (Tier 2 below).
Public repo: after changing the template here, sync:
node scripts/sync-public-lifecycle-workflow.mjs --repo marklovely/lovely-hub-provisionTier 2 — R2 stable report URLs
After each lifecycle job, CI uploads playwright-report/ (and test-results/results.json when present) to platform R2:
- Prefix:
e2e-reports/{github_run_id}/{plan}/(includesindex.html) - Bucket:
lovely-home-e2e-reports(Terraformcloudflare_r2_bucket.e2e_reportsinterraform/modules/platform_admin/r2.tf, production platform stack). It holds only test reports and failed-deploy logs. Hub backups live inlovely-home-hub-archives, which has no web address (#1564). There is no override: every script usesE2E_REPORTS_R2_BUCKET_NAMEfromscripts/lib/platform-archive-storage.mjs. Reports and logs written before that move were copied across by the one-off Copy old e2e reports Action (scripts/copy-old-e2e-reports-r2.mjs, #1577), same keys, so old links still open. Only the last 30 days were copied. Calls are paced to 3 a second to stay under Cloudflare's ~1,200 calls per 5 minutes (#1579); a re-run skips what is already copied. The originals were then deleted from the backups bucket with the same Action's delete from backups option (#1581), which only touches those two folders and refuses while anything recent is uncopied. - Retention: 30 days. A lifecycle rule on the bucket (Terraform
cloudflare_r2_bucket_lifecycle.e2e_reports) deletes every object 30 days after upload, so nothing has to run a prune.scripts/prune-e2e-reports-r2.mjsis still there for a manual clear-out. - Address:
e2e-reports.ops.lovely-home.co.ukis the bucket's custom domain (Terraformcloudflare_r2_custom_domain.e2e_reports). A Cloudflare Access app on that hostname asks for sign-in first. - Upload:
scripts/upload-e2e-playwright-report-r2.mjsvia Wrangler (CLOUDFLARE_ACCOUNT_ID,CLOUDFLARE_API_TOKEN)
Viewing HTML in R2 (auth)
Reports may contain hub screenshots — treat as ops-only.
| Approach | Notes |
|---|---|
| R2 custom domain + Cloudflare Access | Set repo secret E2E_REPORTS_PUBLIC_BASE_URL (HTTPS origin, no trailing slash). Summary and Grafana artifact_url use {base}/e2e-reports/{run_id}/{plan}/index.html. Protect the hostname with Zero Trust (operator or service token), same pattern as platform admin APIs. |
| Cloudflare dashboard | Open r2://{bucket}/e2e-reports/... without a public URL — artifact_url falls back to r2://… when E2E_REPORTS_PUBLIC_BASE_URL is unset. |
| Signed URLs | Not generated in CI today; add a small Worker in front of the bucket if you need time-limited links without Access. |
GitHub Actions also writes the R2 prefix and stable URL (when configured) to GITHUB_STEP_SUMMARY and passes artifact_url to the Tier 3 POST step.
Runs index (browse all uploads)
CI rebuilds a static HTML index after each R2 upload (and after prune deletes):
- Object key:
e2e-reports/index.html - Public URL (when configured):
{E2E_REPORTS_PUBLIC_BASE_URL}/e2e-reports/index.html— same Cloudflare Access policy as individual reports - Contents: Playwright logo as the tab icon (inline SVG from
assets/playwright-logo.svg, copied from playwright-core, Apache-2.0), Lovely Home wordmark (inline PNG, same asset as hub/api/branding/logo), GitHubrun_id, plan(s), latest R2 upload time (maxlast_modifiedunder that run, same heuristic as prune), links to each plan’sindex.html, optional passed/failed fromresults.json/test-results/results.jsonin R2, and a GitHub Actions run link (default repomarklovely/lovely-hub-provision; override withE2E_LIFECYCLE_GITHUB_REPOSITORY)
Manual rebuild from a machine with R2 list/read + write:
CLOUDFLARE_ACCOUNT_ID=… CLOUDFLARE_API_TOKEN=… \
npm run rebuild:e2e-reports-indexImplementation: scripts/rebuild-e2e-reports-r2-index.mjs (also invoked from upload and prune scripts).
Retention (R2 prune)
Private-repo GitHub Artifacts expire after 30 days; R2 uploads are not removed automatically. Prune only the e2e-reports/ prefix in the archive bucket (never hub {site_id}/site-backup-* keys):
# Preview (default — no deletes)
CLOUDFLARE_ACCOUNT_ID=… CLOUDFLARE_API_TOKEN=… \
node scripts/prune-e2e-reports-r2.mjs
# Delete runs whose newest object is older than 30 days (default retention)
CLOUDFLARE_ACCOUNT_ID=… CLOUDFLARE_API_TOKEN=… \
node scripts/prune-e2e-reports-r2.mjs --apply
# Custom retention
E2E_REPORTS_R2_RETENTION_DAYS=14 node scripts/prune-e2e-reports-r2.mjs --apply --retention-days 14Age heuristic: folder names use GitHub run_id (numeric, not a timestamp). The script groups objects under e2e-reports/{run_id}/ and compares the newest R2 last_modified / uploaded timestamp in that run to the retention cutoff. Runs without timestamps are skipped with a warning.
Token needs R2 read + delete on lovely-home-e2e-reports.
Tier 3 — Grafana (metrics & history)
Grafana on the ops VPS tracks run health over time (not full Playwright traces).
- D1 table
e2e_lifecycle_runsonSOCIAL_INSIGHTS_DB(social/migrations/0016_e2e_lifecycle_runs.sql— apply withnpm run social:migrate). - CI
POST /api/platform/e2e-lifecycle-runson the production platform (E2E_LIFECYCLE_REPORT_ORIGIN, defaulthttps://platform.lovely-home.co.uk) after tests. That holds even when the run tested sandbox, since Grafana reads production (#1003).E2E_LIFECYCLE_CF_ACCESS_*must therefore be the production service token. (scripts/post-e2e-lifecycle-run.mjs) with the same Grafana Infinity Cloudflare Access service token headers as social insights. GET /api/platform/e2e-lifecycle-runs?from=&to=— JSON for Grafana Infinity (stat, duration time series, last 20 runs table with GitHub / R2 links).
Dashboard: Lovely Home — E2E lifecycle (social/grafana/lovely-home-e2e-lifecycle-dashboard.json). See grafana-dashboards.md.
Full HTML/traces remain in R2 (and, for the private workflow, GitHub Artifacts).
Recording is best effort (#945). post-e2e-lifecycle-run.mjs prints a ::warning:: and exits 0 when the platform can't store the run, so it never turns a green lifecycle job red:
503 SOCIAL_INSIGHTS_DB_NOT_CONFIGURED: the report platform's Pages project has noSOCIAL_INSIGHTS_DBbinding. Sandbox has none, which is why runs used to be lost before #1003 sent them to production.401 UNAUTHORIZED:E2E_LIFECYCLE_CF_ACCESS_*doesn't match production'sSOCIAL_INSIGHTS_CF_ACCESS_*. Set the secrets from the production Terraform outputs on both repos.- An HTML page instead of JSON — Cloudflare Access answered with its login page because the
Lovely Home — E2E lifecycle runs APIbypass app is missing on that platform. It'splatform_e2e_lifecycle_runsinterraform/modules/platform_admin/access.tf; on sandbox apply it to the sandbox state (see platform-terraform.md).
GitHub secrets (lifecycle workflows)
| Secret | Used for |
|---|---|
CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_API_TOKEN | R2 upload (R2 Object Write on the reports bucket) |
E2E_REPORTS_PUBLIC_BASE_URL | Optional HTTPS base for bookmarkable HTML URLs |
E2E_LIFECYCLE_CF_ACCESS_CLIENT_ID, E2E_LIFECYCLE_CF_ACCESS_CLIENT_SECRET | Platform POST/GET — same values as Grafana social_insights_service_token_* Terraform outputs |
On the public orchestrator repo, add the same secrets (plus existing Stripe/IMAP secrets). Sync the workflow template after merging home-dashboard changes.
One-shot sync (from a machine with hub.tfvars, Terraform R2 backend, and gh auth login):
# Full Cloudflare token for provision + R2 (do this first — do not skip on the public repo)
node scripts/sync-hub-tfvars-to-github-secrets.mjs --repo marklovely/lovely-hub-provision --profile provision --from-terraform-state --skip-missing
# E2E Access + R2 bucket name only (does not overwrite CLOUDFLARE_API_TOKEN)
node scripts/sync-e2e-lifecycle-github-secrets.mjsThe E2E sync always reads the production platform state (it runs tf-platform.sh production init first) and prints the token ID. That ID must match SOCIAL_INSIGHTS_CF_ACCESS_CLIENT_ID from bash scripts/verify-grafana-platform-api.sh. Before #1003 it read whichever state terraform/ was last set up for, which put the sandbox token in these secrets. The CI runner queue metrics workflow uses the same secrets.
Optional: E2E_REPORTS_PUBLIC_BASE_URL=… in env before the second command.
Related
- Project planning: agent store e2e-reporting-and-grafana.md
- public-lifecycle-dispatch.md