Skip to content

test(e2e): add Playwright diagnostics harness - #143

Open
s4piens wants to merge 12 commits into
mainfrom
ci/playwright-e2e-diagnostics
Open

test(e2e): add Playwright diagnostics harness#143
s4piens wants to merge 12 commits into
mainfrom
ci/playwright-e2e-diagnostics

Conversation

@s4piens

@s4piens s4piens commented May 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add a Playwright-based E2E regression harness with screenshots/video/trace on failure
  • Preserve a versioned Home Assistant fixture so entity-to-area-to-floor mappings stay stable
  • Add a dedicated CI workflow that uploads Playwright artifacts on failure
  • Document how to run and update the fixture

Commits

  • 5827f2d test(e2e): add Playwright HA regression harness
  • ed9db88 ci(e2e): upload Playwright failure artifacts
  • e471db6 docs(testing): document the Playwright E2E setup

Validation

  • npm run test:e2e
  • npm run type-check ⚠️ existing repo issue in src/cards/SwipeCard.ts / src/types/lovelace-mushroom/cards/swipe-card-config.ts
  • npm run build ⚠️ local rspack bus error on Node 20.19.2; CI uses Node 24 in this repo

Notes

  • Playwright captures screenshots, video, and trace artifacts on failure
  • The fixture currently models 28 entities / 5 areas / 2 floors / 12 devices

@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

🔍 CI Check Results

Check Status Details
Python Lint (Ruff) Passed
TypeScript & ESLint Passed (non-blocking)
Build & Smoke Tests Passed
Playwright E2E Passed

✅ All Required Checks Passed!

This PR is ready for review and can be merged.


Automated CI check • View workflow run

s4piens added 6 commits May 13, 2026 10:30
…ce capture on failure

- Install @playwright/test as dev dependency
- Add playwright.config.ts with failure diagnostics (screenshot, video, trace)
- Add npm scripts: test:e2e, test:e2e:ui, test:e2e:server
Add comprehensive Home Assistant fixture for E2E testing:
- fixture-version.json: Metadata and versioning
- floor_registry.json: 2 floors (ground, first)
- area_registry.json: 5 areas with floor mappings
- label_registry.json: 3 labels (utility, outside, security)
- device_registry.json: 12 devices with area assignments
- entity_registry.json: 28 entities with device/area relationships
- states.json: Representative entity states including unavailable
- icons.json: Frontend icons for entity_component and services
- linus_config.json: Linus Dashboard configuration

Preserves entity↔device↔area↔floor hierarchy for reproducible tests
- add a versioned Home Assistant fixture that preserves entity↔area↔floor mappings
- boot a lightweight local test server and Playwright harness
- capture screenshots, video, and traces on failure
- add scripts for local E2E execution
- add a dedicated E2E workflow on PRs, push, and manual dispatch
- upload Playwright report and trace/video artifacts on failure
- keep the workflow isolated from the feature MR
- explain the versioned Home Assistant fixture strategy
- document the screenshot/video/trace artifacts on failure
- document local and CI execution
@s4piens
s4piens force-pushed the ci/playwright-e2e-diagnostics branch from 4ab7ed7 to e26c446 Compare May 13, 2026 10:31
@github-actions

Copy link
Copy Markdown
Contributor

🎥 Playwright E2E artifacts

These tests exercise the dashboard generation against the versioned HA fixture.

What is covered

  • dashboard generation completes without errors
  • entity ↔ area ↔ floor mappings stay valid
  • generated views exist for every area and floor
  • the visible summary matches the fixture counts

Artifacts

What to inspect

  • playwright-report: HTML report for the run
  • playwright-failure-artifacts: screenshots, videos, and traces when a failure occurs

Workflow run

@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

🎥 Playwright E2E report

Open the report directly in the browser: Playwright HTML report

What is covered

  • dashboard generation completes without errors
  • entity ↔ area ↔ floor mappings stay valid
  • generated views exist for every area and floor
  • the visible summary matches the fixture counts

Debug artifacts

  • screenshots, videos, and traces are uploaded in the GitHub Actions run when the test fails

Workflow run

@s4piens s4piens left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review indépendante — verdict : approve

The Playwright E2E harness is integrated across CI, fixtures, and reporting, and the additional workflow/docs changes line up with the new diagnostics flow.

🔴 Blocking : 0 · 🟡 Important : 0 · 🟢 Suggestion : 0

Points positifs

  • The fixture bundle is explicit and versioned, which should make regression runs reproducible.
  • The PR posts a PR-side status summary so failures are easier to diagnose without opening the raw logs.

Risques résiduels

  • The workflow depends on GitHub Pages deployment and Playwright artifact publication, so any permissions regression would surface at runtime rather than in static diff review.

Postée automatiquement. Aucun merge, aucune approbation : la décision reste humaine.

@s4piens s4piens left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review indépendante — verdict : approve

The PR adds a Playwright E2E diagnostics harness, a versioned Home Assistant fixture bundle, and CI publishing for failure artifacts. The change set is coherent and the collected GitHub checks are green.

🔴 Blocking : 0 · 🟡 Important : 0 · 🟢 Suggestion : 0

Points positifs

  • The PR keeps the diagnostics logic self-contained under tests/e2e and tests/ha-fixture.
  • CI failure artifacts should materially improve triage for regressions.
  • The fixture versioning makes the E2E data set auditable and reproducible.

Risques résiduels

  • Generated bundle and source map churn is large; future fixture or harness changes may need careful regeneration discipline.
  • The new workflow depends on GitHub Pages/deploy-pages behavior remaining stable.

Postée automatiquement. Aucun merge, aucune approbation : la décision reste humaine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant