Skip to content

E2E lifecycle reports ​

Where to open reports (GitHub Actions) ​

  1. Open the lifecycle workflow run (private Hub lifecycle (Stripe test) or public Hub lifecycle dispatch).
  2. Summary tab — markdown table of tests, durations, Playwright annotations (site id, plan, Phase B notes, checkout fallback, etc.), and when configured an R2 stable report link.
  3. Artifacts (private Hub lifecycle (Stripe test) only) — download playwright-report-{plan}-{run_id}:
    • Unzip and open playwright-report/index.html for the full HTML report.
    • test-results/ holds traces, screenshots, and video on failure (retain-on-failure).

The private workflow uploads artifacts on every run (if: always()) and keeps them 30 days, including green runs. The public dispatch uploads no GitHub artifact (#1169). Anyone with a GitHub account can download a public repo's artifacts, and traces hold request headers, session cookies and sign-in codes. Its report, with traces, goes only to private R2 (Tier 2 below).

Public repo: after changing the template here, sync:

bash
node scripts/sync-public-lifecycle-workflow.mjs --repo marklovely/lovely-hub-provision

Tier 2 — R2 stable report URLs ​

After each lifecycle job, CI uploads playwright-report/ (and test-results/results.json when present) to platform R2:

  • Prefix: e2e-reports/{github_run_id}/{plan}/ (includes index.html)
  • Bucket: lovely-home-e2e-reports (Terraform cloudflare_r2_bucket.e2e_reports in terraform/modules/platform_admin/r2.tf, production platform stack). It holds only test reports and failed-deploy logs. Hub backups live in lovely-home-hub-archives, which has no web address (#1564). There is no override: every script uses E2E_REPORTS_R2_BUCKET_NAME from scripts/lib/platform-archive-storage.mjs. Reports and logs written before that move were copied across by the one-off Copy old e2e reports Action (scripts/copy-old-e2e-reports-r2.mjs, #1577), same keys, so old links still open. Only the last 30 days were copied. Calls are paced to 3 a second to stay under Cloudflare's ~1,200 calls per 5 minutes (#1579); a re-run skips what is already copied. The originals were then deleted from the backups bucket with the same Action's delete from backups option (#1581), which only touches those two folders and refuses while anything recent is uncopied.
  • Retention: 30 days. A lifecycle rule on the bucket (Terraform cloudflare_r2_bucket_lifecycle.e2e_reports) deletes every object 30 days after upload, so nothing has to run a prune. scripts/prune-e2e-reports-r2.mjs is still there for a manual clear-out.
  • Address: e2e-reports.ops.lovely-home.co.uk is the bucket's custom domain (Terraform cloudflare_r2_custom_domain.e2e_reports). A Cloudflare Access app on that hostname asks for sign-in first.
  • Upload: scripts/upload-e2e-playwright-report-r2.mjs via Wrangler (CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_API_TOKEN)

Viewing HTML in R2 (auth) ​

Reports may contain hub screenshots — treat as ops-only.

ApproachNotes
R2 custom domain + Cloudflare AccessSet repo secret E2E_REPORTS_PUBLIC_BASE_URL (HTTPS origin, no trailing slash). Summary and Grafana artifact_url use {base}/e2e-reports/{run_id}/{plan}/index.html. Protect the hostname with Zero Trust (operator or service token), same pattern as platform admin APIs.
Cloudflare dashboardOpen r2://{bucket}/e2e-reports/... without a public URL — artifact_url falls back to r2://… when E2E_REPORTS_PUBLIC_BASE_URL is unset.
Signed URLsNot generated in CI today; add a small Worker in front of the bucket if you need time-limited links without Access.

GitHub Actions also writes the R2 prefix and stable URL (when configured) to GITHUB_STEP_SUMMARY and passes artifact_url to the Tier 3 POST step.

Runs index (browse all uploads) ​

CI rebuilds a static HTML index after each R2 upload (and after prune deletes):

  • Object key: e2e-reports/index.html
  • Public URL (when configured): {E2E_REPORTS_PUBLIC_BASE_URL}/e2e-reports/index.html — same Cloudflare Access policy as individual reports
  • Contents: Playwright logo as the tab icon (inline SVG from assets/playwright-logo.svg, copied from playwright-core, Apache-2.0), Lovely Home wordmark (inline PNG, same asset as hub /api/branding/logo), GitHub run_id, plan(s), latest R2 upload time (max last_modified under that run, same heuristic as prune), links to each plan’s index.html, optional passed/failed from results.json / test-results/results.json in R2, and a GitHub Actions run link (default repo marklovely/lovely-hub-provision; override with E2E_LIFECYCLE_GITHUB_REPOSITORY)

Manual rebuild from a machine with R2 list/read + write:

bash
CLOUDFLARE_ACCOUNT_ID=… CLOUDFLARE_API_TOKEN=… \
  npm run rebuild:e2e-reports-index

Implementation: scripts/rebuild-e2e-reports-r2-index.mjs (also invoked from upload and prune scripts).

Retention (R2 prune) ​

Private-repo GitHub Artifacts expire after 30 days; R2 uploads are not removed automatically. Prune only the e2e-reports/ prefix in the archive bucket (never hub {site_id}/site-backup-* keys):

bash
# Preview (default — no deletes)
CLOUDFLARE_ACCOUNT_ID=… CLOUDFLARE_API_TOKEN=… \
  node scripts/prune-e2e-reports-r2.mjs

# Delete runs whose newest object is older than 30 days (default retention)
CLOUDFLARE_ACCOUNT_ID=… CLOUDFLARE_API_TOKEN=… \
  node scripts/prune-e2e-reports-r2.mjs --apply

# Custom retention
E2E_REPORTS_R2_RETENTION_DAYS=14 node scripts/prune-e2e-reports-r2.mjs --apply --retention-days 14

Age heuristic: folder names use GitHub run_id (numeric, not a timestamp). The script groups objects under e2e-reports/{run_id}/ and compares the newest R2 last_modified / uploaded timestamp in that run to the retention cutoff. Runs without timestamps are skipped with a warning.

Token needs R2 read + delete on lovely-home-e2e-reports.

Tier 3 — Grafana (metrics & history) ​

Grafana on the ops VPS tracks run health over time (not full Playwright traces).

  1. D1 table e2e_lifecycle_runs on SOCIAL_INSIGHTS_DB (social/migrations/0016_e2e_lifecycle_runs.sql — apply with npm run social:migrate).
  2. CI POST /api/platform/e2e-lifecycle-runs on the production platform (E2E_LIFECYCLE_REPORT_ORIGIN, default https://platform.lovely-home.co.uk) after tests. That holds even when the run tested sandbox, since Grafana reads production (#1003). E2E_LIFECYCLE_CF_ACCESS_* must therefore be the production service token. (scripts/post-e2e-lifecycle-run.mjs) with the same Grafana Infinity Cloudflare Access service token headers as social insights.
  3. GET /api/platform/e2e-lifecycle-runs?from=&to= — JSON for Grafana Infinity (stat, duration time series, last 20 runs table with GitHub / R2 links).

Dashboard: Lovely Home — E2E lifecycle (social/grafana/lovely-home-e2e-lifecycle-dashboard.json). See grafana-dashboards.md.

Full HTML/traces remain in R2 (and, for the private workflow, GitHub Artifacts).

Recording is best effort (#945). post-e2e-lifecycle-run.mjs prints a ::warning:: and exits 0 when the platform can't store the run, so it never turns a green lifecycle job red:

  • 503 SOCIAL_INSIGHTS_DB_NOT_CONFIGURED: the report platform's Pages project has no SOCIAL_INSIGHTS_DB binding. Sandbox has none, which is why runs used to be lost before #1003 sent them to production.
  • 401 UNAUTHORIZED: E2E_LIFECYCLE_CF_ACCESS_* doesn't match production's SOCIAL_INSIGHTS_CF_ACCESS_*. Set the secrets from the production Terraform outputs on both repos.
  • An HTML page instead of JSON — Cloudflare Access answered with its login page because the Lovely Home — E2E lifecycle runs API bypass app is missing on that platform. It's platform_e2e_lifecycle_runs in terraform/modules/platform_admin/access.tf; on sandbox apply it to the sandbox state (see platform-terraform.md).

GitHub secrets (lifecycle workflows) ​

SecretUsed for
CLOUDFLARE_ACCOUNT_ID, CLOUDFLARE_API_TOKENR2 upload (R2 Object Write on the reports bucket)
E2E_REPORTS_PUBLIC_BASE_URLOptional HTTPS base for bookmarkable HTML URLs
E2E_LIFECYCLE_CF_ACCESS_CLIENT_ID, E2E_LIFECYCLE_CF_ACCESS_CLIENT_SECRETPlatform POST/GET — same values as Grafana social_insights_service_token_* Terraform outputs

On the public orchestrator repo, add the same secrets (plus existing Stripe/IMAP secrets). Sync the workflow template after merging home-dashboard changes.

One-shot sync (from a machine with hub.tfvars, Terraform R2 backend, and gh auth login):

bash
# Full Cloudflare token for provision + R2 (do this first — do not skip on the public repo)
node scripts/sync-hub-tfvars-to-github-secrets.mjs --repo marklovely/lovely-hub-provision --profile provision --from-terraform-state --skip-missing

# E2E Access + R2 bucket name only (does not overwrite CLOUDFLARE_API_TOKEN)
node scripts/sync-e2e-lifecycle-github-secrets.mjs

The E2E sync always reads the production platform state (it runs tf-platform.sh production init first) and prints the token ID. That ID must match SOCIAL_INSIGHTS_CF_ACCESS_CLIENT_ID from bash scripts/verify-grafana-platform-api.sh. Before #1003 it read whichever state terraform/ was last set up for, which put the sandbox token in these secrets. The CI runner queue metrics workflow uses the same secrets.

Optional: E2E_REPORTS_PUBLIC_BASE_URL=… in env before the second command.