Skip to content

Updates for PF6 unit tests#126

Open
mshriver wants to merge 1 commit into
RedHatQE:mainfrom
mshriver:pf6-test-fixes-hatch
Open

Updates for PF6 unit tests#126
mshriver wants to merge 1 commit into
RedHatQE:mainfrom
mshriver:pf6-test-fixes-hatch

Conversation

@mshriver
Copy link
Copy Markdown
Collaborator

@mshriver mshriver commented Jun 3, 2026

Summary by Sourcery

Update Playwright-based testing and widget interactions to be more robust and compatible with PatternFly 5 and 6, and standardize test execution via Hatch environments.

Bug Fixes:

  • Fix menu item selection for always-open menus and ensure checkbox state is reliably toggled with retries.
  • Correct radio widgets to reflect and change selection via the underlying input element with proper DOM events for PF6 behavior.
  • Make dropdown open/close, select create-item, navigation selection, chip closing, and slider value changes more reliable by using Playwright-friendly locators, event dispatching, and short wait/retry loops.
  • Adjust alert and dropdown test locators to avoid PF6-specific markup changes and transient site-wide alerts that interfered with assertions.

Enhancements:

  • Introduce Hatch-based test and lint environments with convenient scripts for PF5/PF6 test runs and debugging.
  • Configure pytest defaults (test paths and timeouts) and Playwright browser context timeouts for more predictable and maintainable test behavior.

Build:

  • Add Playwright as a development dependency and define Hatch envs and scripts in pyproject.toml for running tests and linting.

CI:

  • Refactor the GitHub Actions test workflow to use Hatch-managed environments and scripts for installing Playwright browsers and running the PF5/PF6 test matrix.

Tests:

  • Update UI tests and shared fixtures to work against PatternFly 6 markup and improve stability under Playwright by scoping locators and setting browser context timeouts.

Copilot AI review requested due to automatic review settings June 3, 2026 09:23
@sourcery-ai
Copy link
Copy Markdown

sourcery-ai Bot commented Jun 3, 2026

Reviewer's Guide

Refactors CI to use Hatch-based test environments and Playwright management, adds Hatch test/lint environments and pytest config, and introduces several Playwright/PF6‑oriented robustness fixes across widgets and tests (menus, radios, selects, navigation, sliders, chips, and alert tests).

Sequence diagram for PF6 Radio.fill interaction and event dispatch

sequenceDiagram
    actor Test
    participant Radio
    participant Browser
    participant Element

    Test->>Radio: fill(values)
    Radio->>Radio: selected
    Radio->>Browser: is_checked(RADIO_LOC)
    Browser-->>Radio: checked_state
    alt [values == selected]
        Radio-->>Test: return False
    else [values is truthy and != selected]
        Radio->>Browser: element(RADIO_LOC)
        Browser-->>Radio: Element
        Radio->>Element: evaluate(js_set_checked_and_dispatch_events)
        Radio-->>Test: return True
    end
Loading

File-Level Changes

Change Details Files
CI workflow now uses Hatch-managed test environment and Playwright installation/caching for PF5/PF6 test matrix.
  • Replace direct pip installation of Playwright with installing Hatch, creating a test env, and using it to detect Playwright version and run tests
  • Add caching for the Hatch environment in setup and matrix test jobs keyed by OS, Python version, and pyproject.toml hash
  • Switch Playwright browser installation step to hatch run test:install-browsers and adjust browser cache key usage
  • Update matrix pf-version values from v5/v6 to 5/6 and adjust job naming and coverage include entry accordingly
  • Run pytest via hatch run test:pf{pf-version} with appropriate flags, removing explicit dev dependency installation in the workflow
.github/workflows/tests.yaml
Introduce Hatch environments and test helpers, including Playwright dependency and pytest configuration.
  • Add playwright to dev dependencies
  • Define tool.hatch.envs.test with dev features and scripts for installing browsers, generic/headless tests, PF5/PF6 specific runs, and a debug mode
  • Add lint hatch env with pre-commit dependency and script
  • Configure pytest defaults via [tool.pytest.ini_options] to use testing as test root and a default timeout
pyproject.toml
Improve PF5 Menu widget behaviour for always-open menus, selection reliability, and state reading.
  • Special-case item_element when IS_ALWAYS_OPEN to search both browser and root_browser items, returning either the child input or the element itself and raising MenuItemNotFound with available items when not found
  • Wrap item_select logic in a wait loop that repeatedly clicks until the item is selected, with a bounded timeout
  • Wrap item_deselect logic in a similar wait loop ensuring items become unselected
  • Simplify read to get the child input once and use its is_checked state, defaulting to False when input is absent
src/widgetastic_patternfly5/components/menus/menu.py
Adjust radios, select, navigation, slider, and chip components for PF6/Playwright interaction and reliability.
  • Change Radio.selected to use browser.is_checked(RADIO_LOC) and implement fill to idempotently set checked state via JS that uses the native checked setter and dispatches click/input/change events
  • Update Select.fill to wait for the create-item option to appear using Playwright page.locator and then click it, instead of clicking via root browser locator immediately
  • Modify Navigation.select expansion click to target the child anchor/button via dispatch_event('click') and make missing sub-items raise NoSuchElementException explicitly
  • Improve InputSlider.fill by focusing the input, filling the new value, dispatching a change event, pressing Enter, and waiting until the displayed text matches the desired value
  • Change CategoryChip.close to dispatch a click on CATEGORY_CLOSE via the browser rather than using the close_button property directly
src/widgetastic_patternfly5/components/forms/radio.py
src/widgetastic_patternfly5/components/menus/select.py
src/widgetastic_patternfly5/components/navigation.py
src/widgetastic_patternfly5/components/slider.py
src/widgetastic_patternfly5/components/chip.py
Stabilize dropdown/menu-related and alert tests for PF6 rendering differences and Playwright timeouts.
  • Adjust dropdown disabled test to use a PF5/PF6-agnostic locator that finds the first menu toggle button with text 'Dropdown' and moves to its parent container
  • Make dropdown open use wait_for around a click on BUTTON_LOCATOR until is_open is true
  • Scope alert tests to the 'Alert variants' section to avoid interference from the PF6 site-wide info notification
  • Set a default 10s timeout on the Playwright browser context for tests
testing/components/menus/test_dropdown_disabled.py
src/widgetastic_patternfly5/components/menus/dropdown.py
testing/components/test_alert.py
testing/conftest.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link
Copy Markdown

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 3 issues, and left some high level feedback:

  • In ChipCategory.close (or the relevant chip category class), switching from self.close_button.click() to self.browser.element(CATEGORY_CLOSE) removes scoping to the widget root and may click the wrong close button when multiple chip categories are present; consider resolving CATEGORY_CLOSE relative to the component root instead of via the global browser.
  • In Select.fill, deriving a CSS selector by string-splitting CREATE_ITEM_LOCATOR to extract an @id assumes a very specific XPath shape and will break if the locator changes format; it would be more robust either to use the existing locator directly with Playwright or to store the bare ID/name separately rather than parsing it from the string.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `ChipCategory.close` (or the relevant chip category class), switching from `self.close_button.click()` to `self.browser.element(CATEGORY_CLOSE)` removes scoping to the widget root and may click the wrong close button when multiple chip categories are present; consider resolving `CATEGORY_CLOSE` relative to the component root instead of via the global browser.
- In `Select.fill`, deriving a CSS selector by string-splitting `CREATE_ITEM_LOCATOR` to extract an `@id` assumes a very specific XPath shape and will break if the locator changes format; it would be more robust either to use the existing locator directly with Playwright or to store the bare ID/name separately rather than parsing it from the string.

## Individual Comments

### Comment 1
<location path="src/widgetastic_patternfly5/components/forms/radio.py" line_range="52-61" />
<code_context>
     def fill(self, values):
         """Can only handle `True` to check the radio, nature of individual radio button"""
-        return self.radio.fill(values)
+        if values == self.selected:
+            return False
+        if values:
+            el = self.browser.element(self.RADIO_LOC)
+            el.evaluate(
+                "e => {"
+                "  const nativeSetter = Object.getOwnPropertyDescriptor("
+                "    window.HTMLInputElement.prototype, 'checked'"
+                "  ).set;"
+                "  nativeSetter.call(e, true);"
+                "  e.dispatchEvent(new Event('click', {bubbles: true}));"
+                "  e.dispatchEvent(new Event('input', {bubbles: true}));"
+                "  e.dispatchEvent(new Event('change', {bubbles: true}));"
+                "}"
+            )
+        return True
</code_context>
<issue_to_address>
**issue (bug_risk):** Clarify behavior when `values` is `False` for an already-checked radio.

With the current logic, calling `fill(False)` on a selected radio skips the early return, skips the `if values:` block, and still returns `True`, suggesting a change occurred when nothing actually changed.

Consider either treating `False` as a no-op that returns `False`, or explicitly disallowing `False` (e.g., by raising) and documenting that `fill` only supports `True`. Otherwise, callers may misinterpret the return value and assume the radio was updated when it was not.
</issue_to_address>

### Comment 2
<location path="src/widgetastic_patternfly5/components/menus/select.py" line_range="225-233" />
<code_context>
-            self.root_browser.click(self.CREATE_ITEM_LOCATOR)
+            from wait_for import wait_for as _wait_for
+
+            _id_attr = self.CREATE_ITEM_LOCATOR.split("@id='")[1].rstrip("']")
+            create_css = "#" + _id_attr
+            page = self.browser.element(".").page
+            _wait_for(
+                lambda: page.locator(create_css).count() > 0,
+                timeout=3,
+                delay=0.2,
+            )
+            page.locator(create_css).click()
             return True
         else:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Avoid brittle string parsing of `CREATE_ITEM_LOCATOR` to derive the CSS selector.

This assumes `CREATE_ITEM_LOCATOR` is always an XPath with `@id='…'` in a specific format. Any change (quotes, extra predicates, non-XPath locator) will either raise `IndexError` or yield a wrong selector.

Instead, either:
- Use the locator directly via the browser abstraction (e.g. `self.root_browser.click(self.CREATE_ITEM_LOCATOR)` with waiting), or
- Locate the element with the existing locator and call `get_attribute('id')` to derive `#id`.

That avoids coupling this code to a specific string representation of the locator.
</issue_to_address>

### Comment 3
<location path="src/widgetastic_patternfly5/components/chip.py" line_range="249-250" />
<code_context>

     def close(self):
-        self.close_button.click()
+        close_el = self.browser.element(CATEGORY_CLOSE)
+        close_el.dispatch_event("click")

     @classmethod
</code_context>
<issue_to_address>
**issue (bug_risk):** Use the class attribute for `CATEGORY_CLOSE` and leverage the existing `close_button` widget.

`CATEGORY_CLOSE` is not defined in this scope and should be referenced as the class attribute (e.g. `self.CATEGORY_CLOSE`), otherwise this will raise a `NameError`.

Also, the previous version used `self.close_button`, which likely wraps the correct locator and behavior. To stay consistent with the existing API and avoid regressions, consider either:
- `close_el = self.browser.element(self.CATEGORY_CLOSE)`, or
- Preferably, `self.close_button.dispatch_event("click")` to reuse the widget abstraction.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +52 to +61
if values == self.selected:
return False
if values:
el = self.browser.element(self.RADIO_LOC)
el.evaluate(
"e => {"
" const nativeSetter = Object.getOwnPropertyDescriptor("
" window.HTMLInputElement.prototype, 'checked'"
" ).set;"
" nativeSetter.call(e, true);"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Clarify behavior when values is False for an already-checked radio.

With the current logic, calling fill(False) on a selected radio skips the early return, skips the if values: block, and still returns True, suggesting a change occurred when nothing actually changed.

Consider either treating False as a no-op that returns False, or explicitly disallowing False (e.g., by raising) and documenting that fill only supports True. Otherwise, callers may misinterpret the return value and assume the radio was updated when it was not.

Comment on lines +225 to +233
_id_attr = self.CREATE_ITEM_LOCATOR.split("@id='")[1].rstrip("']")
create_css = "#" + _id_attr
page = self.browser.element(".").page
_wait_for(
lambda: page.locator(create_css).count() > 0,
timeout=3,
delay=0.2,
)
page.locator(create_css).click()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (bug_risk): Avoid brittle string parsing of CREATE_ITEM_LOCATOR to derive the CSS selector.

This assumes CREATE_ITEM_LOCATOR is always an XPath with @id='…' in a specific format. Any change (quotes, extra predicates, non-XPath locator) will either raise IndexError or yield a wrong selector.

Instead, either:

  • Use the locator directly via the browser abstraction (e.g. self.root_browser.click(self.CREATE_ITEM_LOCATOR) with waiting), or
  • Locate the element with the existing locator and call get_attribute('id') to derive #id.

That avoids coupling this code to a specific string representation of the locator.

Comment on lines +249 to +250
close_el = self.browser.element(CATEGORY_CLOSE)
close_el.dispatch_event("click")
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (bug_risk): Use the class attribute for CATEGORY_CLOSE and leverage the existing close_button widget.

CATEGORY_CLOSE is not defined in this scope and should be referenced as the class attribute (e.g. self.CATEGORY_CLOSE), otherwise this will raise a NameError.

Also, the previous version used self.close_button, which likely wraps the correct locator and behavior. To stay consistent with the existing API and avoid regressions, consider either:

  • close_el = self.browser.element(self.CATEGORY_CLOSE), or
  • Preferably, self.close_button.dispatch_event("click") to reuse the widget abstraction.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request updates the library’s PatternFly widget interactions and the accompanying Playwright-based test suite to be more reliable across PatternFly v5 and v6, while also standardizing local/CI test execution through Hatch environments.

Changes:

  • Improve widget interaction robustness (dropdown/menu selection, radio selection, slider input updates, chip-group close behavior) to better align with PF6 DOM/event behavior.
  • Stabilize UI tests by scoping locators to PF demo sections and using PF5/PF6-safe locators.
  • Add Hatch-managed test/lint environments and refactor GitHub Actions CI to run the PF5/PF6 matrix via Hatch scripts.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
testing/conftest.py Sets Playwright context default timeout to improve test reliability.
testing/components/test_alert.py Scopes alert selection to the demo section to avoid site-wide PF6 alerts.
testing/components/menus/test_dropdown_disabled.py Updates dropdown root locator to work across PF5/PF6 markup differences.
src/widgetastic_patternfly5/components/slider.py Adjusts InputSlider filling to be more event-accurate and adds a post-fill wait.
src/widgetastic_patternfly5/components/navigation.py Changes navigation expansion to click the actionable element via dispatched click.
src/widgetastic_patternfly5/components/menus/select.py Makes typeahead “create item” selection more Playwright-friendly with explicit waiting.
src/widgetastic_patternfly5/components/menus/menu.py Improves always-open menu item lookup and adds retry loops for checkbox select/deselect.
src/widgetastic_patternfly5/components/menus/dropdown.py Adds retry/wait logic to dropdown open to reduce flakiness.
src/widgetastic_patternfly5/components/forms/radio.py Reworks radio selection to operate on the underlying input and dispatch DOM events.
src/widgetastic_patternfly5/components/chip.py Changes category chip-group close to use an event dispatch approach.
pyproject.toml Adds Playwright dev dependency; introduces Hatch envs/scripts; configures pytest defaults.
.github/workflows/tests.yaml Migrates CI to Hatch-managed envs and scripts; updates PF version matrix handling.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines 248 to +250
def close(self):
self.close_button.click()
close_el = self.browser.element(CATEGORY_CLOSE)
close_el.dispatch_event("click")
Comment on lines 50 to +54
def fill(self, values):
"""Can only handle `True` to check the radio, nature of individual radio button"""
return self.radio.fill(values)
if values == self.selected:
return False
if values:
Comment on lines 100 to 102
if self.text == value:
return False
el = self.browser.element(self.INPUT)
Comment on lines +223 to +233
from wait_for import wait_for as _wait_for

_id_attr = self.CREATE_ITEM_LOCATOR.split("@id='")[1].rstrip("']")
create_css = "#" + _id_attr
page = self.browser.element(".").page
_wait_for(
lambda: page.locator(create_css).count() > 0,
timeout=3,
delay=0.2,
)
page.locator(create_css).click()
@mshriver mshriver force-pushed the pf6-test-fixes-hatch branch from 9f0fd29 to 84fc0eb Compare June 3, 2026 09:27
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 3, 2026

Codecov Report

❌ Patch coverage is 67.79661% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.55%. Comparing base (4fb14fe) to head (540f821).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...c/widgetastic_patternfly5/components/menus/menu.py 65.51% 10 Missing ⚠️
...c/widgetastic_patternfly5/components/navigation.py 0.00% 6 Missing ⚠️
.../widgetastic_patternfly5/components/forms/radio.py 71.42% 2 Missing ⚠️
src/widgetastic_patternfly5/components/slider.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #126      +/-   ##
==========================================
- Coverage   91.19%   90.55%   -0.64%     
==========================================
  Files          38       38              
  Lines        2191     2234      +43     
==========================================
+ Hits         1998     2023      +25     
- Misses        193      211      +18     
Flag Coverage Δ
unittests 90.55% <67.79%> (-0.64%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-authored-by: Claude <noreply@anthropic.com>
@mshriver mshriver force-pushed the pf6-test-fixes-hatch branch from 84fc0eb to 540f821 Compare June 3, 2026 14:05
@mshriver mshriver requested a review from Copilot June 4, 2026 06:02
@mshriver mshriver added the bug Something isn't working label Jun 4, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Comment on lines 50 to +54
def fill(self, values):
"""Can only handle `True` to check the radio, nature of individual radio button"""
return self.radio.fill(values)
if values == self.selected:
return False
if values:
Comment on lines +124 to +127
try:
current_item = self.browser.element(self.SUB_ITEMS_ROOT, parent=li)
except NoSuchElementException:
raise
Comment on lines +224 to +226
_id_attr = self.CREATE_ITEM_LOCATOR.split("@id='")[1].rstrip("']")
create_css = "#" + _id_attr
page = self.browser.element(".").page
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants