Image refresh for rhel-10-2

[cockpituous]

Image refresh for rhel-10-2

• image-refresh rhel-10-2

[cockpituous]

image-refresh rhel-10-2 done: https://github.com/cockpit-project/bots/commits/image-refresh-rhel-10-2-20251127-225434

[cockpituous]

Success. Log: https://logs-cockpit.apps.ocp.cloud.ci.centos.org/image-refresh-rhel-10-2-d572245a-20251127-224149/log.html

[Venefilyn]

Unsure if relevant. Will retry

From https://logs-cockpit.apps.ocp.cloud.ci.centos.org/pull-8491-678daabb-20251128-104322-rhel-10-2-storage-cockpit-project-cockpit/log.html

# testBtrfs (__main__.TestStorageAnaconda.testBtrfs)
# Result testBtrfs (__main__.TestStorageAnaconda.testBtrfs) skipped: rhel-10-2: no btrfs support
ok 3 test/verify/check-storage-anaconda TestStorageAnaconda.testBtrfs [ND@2] # SKIP testBtrfs (__main__.TestStorageAnaconda.testBtrfs) rhel-10-2: no btrfs support
# ----------------------------------------------------------------------
# testDegradedBtrfs (__main__.TestStorageAnaconda.testDegradedBtrfs)
# Result testDegradedBtrfs (__main__.TestStorageAnaconda.testDegradedBtrfs) skipped: rhel-10-2: no btrfs support
ok 4 test/verify/check-storage-anaconda TestStorageAnaconda.testDegradedBtrfs [ND@3] # SKIP testDegradedBtrfs (__main__.TestStorageAnaconda.testDegradedBtrfs) rhel-10-2: no btrfs support
# ----------------------------------------------------------------------
# testBasic (__main__.TestStorageAnaconda.testBasic)
Starting ChromeDriver 142.0.7444.162 (c076baf266c3ed5efb225de664cfa7b183668ad6-refs/branch-heads/7444@{#2498}) on port 39957
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully on port 39957.
> info: failed to fetch lastlog2: Object(5)
> info: ANACONDA Object(2)
> warn: Can't enable storaged btrfs module Error initializing module 'btrfs': /usr/lib64/udisks2/modules/libudisks2_btrfs.so: cannot open shared object file: No such file or directory
WARNING: Waiting for ph_count_check("[data-test-card-title='Storage'] tbody tr",1) took 8.2 seconds, which is 54% of the timeout.
Traceback (most recent call last):
  File "/work/make-checkout-workdir/test/verify/check-storage-anaconda", line 73, in testBasic
    self.click_dropdown(self.card_row("Storage", name="vgroup0"), "Create new logical volume")
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/make-checkout-workdir/test/common/storagelib.py", line 645, in click_dropdown
    self.browser.click(self.dropdown_toggle(parent))
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/make-checkout-workdir/test/common/testlib.py", line 617, in click
    self.mouse(selector + ":not([disabled]):not([aria-disabled=true])", "click")
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/make-checkout-workdir/test/common/testlib.py", line 539, in mouse
    self.wait_visible(selector)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/work/make-checkout-workdir/test/common/testlib.py", line 877, in wait_visible
    self._wait_present(selector)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/work/make-checkout-workdir/test/common/testlib.py", line 868, in _wait_present
    self.wait_js_func('ph_is_present', selector)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/make-checkout-workdir/test/common/testlib.py", line 862, in wait_js_func
    self.wait_js_cond("%s(%s)" % (func, ','.join(map(jsquote, args))))
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/work/make-checkout-workdir/test/common/testlib.py", line 859, in wait_js_cond
    raise Error(f"timeout\nwait_js_cond({cond}): {last_error.msg}") from None
testlib.Error: timeout
wait_js_cond(ph_is_present("[data-test-card-title='Storage'] table:nth-of-type(1) tbody [data-test-row-name='vgroup0'] .pf-v6-c-menu-toggle:not([disabled]):not([aria-disabled=true])")): Error: condition did not become true

[jelly]

This PR cockpit-project/cockpit#22626 (review) should make things better

[martinpitt]

Thanks! Landed, and rebased.

[martinpitt]

Retrying failure once to compare

[martinpitt]

Nope, this is broken → pilot board.

[tomasmatus]

Locally I can't even run this test to the point where it fails in CI. I think this is another case of "not enough RAM" on rhel-10-2. Big problem because RAM is so expensive these days.

Running it with 1024 MB makes it pass.
Too bad in this case the usual culprits are already disabled in self.setup_provisioned_hosts()

[jelly]

Locally I can't even run this test to the point where it fails in CI. I think this is another case of "not enough RAM" on rhel-10-2. Big problem because RAM is so expensive these days.

Running it with 1024 MB makes it pass. Too bad in this case the usual culprits are already disabled in self.setup_provisioned_hosts()

Any insights on what is consuming all the ram?

[tomasmatus]

machine 1, I cant even ssh into it...

machine 2

[root@machine2 ~]# free -h
               total        used        free      shared  buff/cache   available
Mem:           451Mi       300Mi       7.4Mi       1.5Mi       156Mi       151Mi
Swap:             0B          0B          0B

Seems like subscription manager ate our RAM!

[root@machine2 ~]# ps aux --sort=-%mem | head -n 5                        
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root        1455  0.2 14.6 399940 67848 ?        Ssl  10:51   0:00 /usr/bin/python3 /usr/libexec/rhsm-service
root           1  0.4  9.1  50752 42304 ?        Ss   10:50   0:01 /usr/lib/systemd/systemd --switched-root --system --deserialize=51
root         744  0.0  8.8 352740 41056 ?        Ssl  10:50   0:00 /usr/bin/python3 -sP /usr/sbin/firewalld --nofork --nopid
admin       1387  0.2  7.1 634248 32908 ?        Ssl  10:51   0:00 /usr/bin/python3 /usr/bin/cockpit-bridge

[tomasmatus]

Wohoo