Make test_positive_task_status faster and more robust by adamlazik1 · Pull Request #21685 · SatelliteQE/robottelo

adamlazik1 · 2026-05-28T12:20:06Z

Switched the repository URL from 'www.non_existent_repo_url.org' to
'doesnotexist.invalid'. The .invalid TLD (RFC 2606) is guaranteed to
return NXDOMAIN immediately from any DNS resolver, eliminating DNS
timeout variance and thus making the task fail faster.

Removed the LatestFailedTasks navigation and task details read that
followed it. The TaskDetails navigator always re-navigates to the tasks
list via the menu and searches for the task independently, so the call
provided no coverage for whether the dashboard link itself worked
correctly. The task result assertion it enabled was also deterministic.
'warning' and 'error' were achieved respectively in PRT and local
environment, making it a source of flakiness rather than meaningful
coverage.

Made with Cursor.

Depends on: SatelliteQE/airgun#2441

Summary by Sourcery

Improve the reliability and speed of the dashboard task status UI test when syncing an invalid repository.

Bug Fixes:

Use a reserved .invalid TLD for the test repository URL to avoid DNS timeout variability and make the failing sync complete faster.
Adjust the dashboard task status assertion to rely on the task result state rather than brittle backend error message text.

sourcery-ai · 2026-05-28T12:20:14Z

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Makes the dashboard task status UI test faster and less brittle by using a guaranteed-invalid hostname for the repo sync and loosening the assertion to focus on task result instead of specific backend error text.

File-Level Changes

Change	Details	Files
Use a guaranteed NXDOMAIN URL to make the failing repository sync faster and more deterministic.	Replace the repository URL host with the RFC 2606-reserved doesnotexist.invalid domain so DNS fails immediately. Remove the intermediate url variable and inline the URL string directly in the Repository creation.	`tests/foreman/ui/test_dashboard.py`
Relax task status assertions to focus on UI-relevant state instead of backend error text.	Keep the assertion on the task result but change the expected value from warning to error to match the new failure behavior. Remove brittle assertions on the exact task error messages that depended on specific HTTP or connection error strings.	`tests/foreman/ui/test_dashboard.py`

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey - I've found 1 issue, and left some high level feedback:

The test still expects tasks['table'][0]['Result'] == 'warning' but now asserts values['task']['result'] == 'error'; please confirm this mismatch is intentional and, if not, align the expected task result with the dashboard status to avoid flaky or confusing behavior.
Consider extracting the repository URL (http://doesnotexist.invalid) into a local variable or constant and reusing it for both creation and any related assertions so that future URL changes don’t silently diverge across the test.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The test still expects `tasks['table'][0]['Result'] == 'warning'` but now asserts `values['task']['result'] == 'error'`; please confirm this mismatch is intentional and, if not, align the expected task result with the dashboard status to avoid flaky or confusing behavior.
- Consider extracting the repository URL (`http://doesnotexist.invalid`) into a local variable or constant and reusing it for both creation and any related assertions so that future URL changes don’t silently diverge across the test.

## Individual Comments

### Comment 1
<location path="tests/foreman/ui/test_dashboard.py" line_range="136-139" />
<code_context>
     product = target_sat.api.Product(organization=org).create()
     repo = target_sat.api.Repository(
-        url=f'http://{url}', product=product, content_type='yum'
+        url='http://doesnotexist.invalid', product=product, content_type='yum'
     ).create()
     with pytest.raises(TaskFailedError):
</code_context>
<issue_to_address>
**suggestion:** Consider documenting the rationale for using the `.invalid` TLD in this test

Because this behavior depends on RFC 2606’s guarantee that `.invalid` always returns NXDOMAIN, please add a brief comment near this line noting that this hostname is intentionally chosen to avoid DNS timeouts and resolver-dependent behavior, so future changes don’t “simplify” it back to a real domain.

```suggestion
    product = target_sat.api.Product(organization=org).create()
    # Use a .invalid TLD as specified in RFC 2606 to guarantee NXDOMAIN and avoid
    # DNS timeouts or resolver-dependent behavior in this test.
    repo = target_sat.api.Repository(
        url='http://doesnotexist.invalid', product=product, content_type='yum'
    ).create()
```
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

adamlazik1 · 2026-05-28T13:32:16Z

trigger: test-robottelo
pytest: -k 'test_positive_task_status' tests/foreman/ui/test_dashboard.py
airgun: 2441

adamruzicka · 2026-05-28T13:48:37Z

-            f"500, message='Internal Server Error', url='http://{url}'",
-            f'Cannot connect to host {url}:80 ssl:default [Domain name not found]',
-        ]
+        assert values['task']['result'] == 'error'


Why the switch from warning to error? I would expect that switching one non-resolvable hostname for another shouldn't have this kind of effect

As far as I understand .invalid domain is different in the fact that the resolver does not even try to resolve it but returns NXDOMAIN immediately. The previous domain probably caused resolver to return different error type after attempting to resolve it.

PRT seems to agree that this is odd

> assert values['task']['result'] == 'error' E AssertionError: assert 'warning' == 'error' E E - error E + warning tests/foreman/ui/test_dashboard.py:161: AssertionError

In my local environment, the task result did change to error, but in PRT it remained 'warning'. I need to investigate further to see if this can be made more robust.

It seems that the result of that assertion can be different based on environment while the assertion itself does not bring much value. I think removing the assertion completely is the best option.

Satellite-QE · 2026-05-28T14:07:04Z

PRT Result

Build Number: 15643
Build Status: UNSTABLE
PRT Comment: pytest -k test_positive_task_status tests/foreman/ui/test_dashboard.py --external-logging
Test Result : =========== 1 failed, 4 deselected, 5 warnings in 1193.15s (0:19:53) ===========

Switched the repository URL from 'www.non_existent_repo_url.org' to 'doesnotexist.invalid'. The .invalid TLD (RFC 2606) is guaranteed to return NXDOMAIN immediately from any DNS resolver, eliminating DNS timeout variance and thus making the task fail faster. Removed the LatestFailedTasks navigation and task details read that followed it. The TaskDetails navigator always re-navigates to the tasks list via the menu and searches for the task independently, so the call provided no coverage for whether the dashboard link itself worked correctly. The task result assertion it enabled was also deterministic. 'warning' and 'error' were achieved respectively in PRT and local environment, making it a source of flakiness rather than meaningful coverage. Made with Cursor.

adamlazik1 · 2026-05-28T14:56:38Z

trigger: test-robottelo
pytest: -k 'test_positive_task_status' tests/foreman/ui/test_dashboard.py
airgun: 2441

Satellite-QE · 2026-05-28T15:45:15Z

PRT Result

Build Number: 15645
Build Status: UNSTABLE
PRT Comment: pytest -k test_positive_task_status tests/foreman/ui/test_dashboard.py --external-logging
Test Result : ====== 1 passed, 4 deselected, 7 warnings, 1 error in 2292.46s (0:38:12) =======

adamlazik1 · 2026-05-28T15:50:48Z

trigger: test-robottelo
pytest: -k 'test_positive_task_status' tests/foreman/ui/test_dashboard.py
airgun: 2441

Satellite-QE · 2026-05-28T16:19:09Z

PRT Result

Build Number: 15647
Build Status: SUCCESS
PRT Comment: pytest -k test_positive_task_status tests/foreman/ui/test_dashboard.py --external-logging
Test Result : =========== 1 passed, 4 deselected, 5 warnings in 1058.08s (0:17:38) ===========

adamlazik1 · 2026-05-29T09:55:13Z

Dependency merged, this is ready for re-review.

adamlazik1 requested a review from a team as a code owner May 28, 2026 12:20

adamlazik1 added the No-CherryPick PR doesnt need CherryPick to previous branches label May 28, 2026

github-actions Bot assigned adamlazik1 May 28, 2026

sourcery-ai Bot reviewed May 28, 2026

View reviewed changes

Comment thread tests/foreman/ui/test_dashboard.py

adamruzicka reviewed May 28, 2026

View reviewed changes

Satellite-QE added the PRT-Failed Indicates that latest PRT run is failed for the PR label May 28, 2026

adamlazik1 force-pushed the update-test_positive_task_status branch from 6834ae7 to 745fdfd Compare May 28, 2026 14:55

adamlazik1 force-pushed the update-test_positive_task_status branch from 745fdfd to b964bca Compare May 28, 2026 14:56

Satellite-QE added PRT-Passed Indicates that latest PRT run is passed for the PR and removed PRT-Failed Indicates that latest PRT run is failed for the PR labels May 28, 2026

adamlazik1 mentioned this pull request May 28, 2026

Update TasksDetailsView to match the PF5 update in tasks SatelliteQE/airgun#2441

Merged

adamlazik1 requested a review from adamruzicka May 29, 2026 09:55

Conversation

adamlazik1 commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by Sourcery

Uh oh!

sourcery-ai Bot commented May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

File-Level Changes

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

adamlazik1 commented May 28, 2026

Uh oh!

adamruzicka May 28, 2026

Choose a reason for hiding this comment

Uh oh!

adamlazik1 May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamruzicka May 28, 2026

Choose a reason for hiding this comment

Uh oh!

adamlazik1 May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

adamlazik1 May 28, 2026

Choose a reason for hiding this comment

Uh oh!

Satellite-QE commented May 28, 2026

Uh oh!

adamlazik1 commented May 28, 2026

Uh oh!

Satellite-QE commented May 28, 2026

Uh oh!

adamlazik1 commented May 28, 2026

Uh oh!

Satellite-QE commented May 28, 2026

Uh oh!

adamlazik1 commented May 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

adamlazik1 commented May 28, 2026 •

edited

Loading

sourcery-ai Bot commented May 28, 2026 •

edited

Loading

adamlazik1 May 28, 2026 •

edited

Loading

adamlazik1 May 28, 2026 •

edited

Loading