Set up OMERO server test infrastructure for tests

[jo-mueller]

Relevant for #95

This PR will expand the tests of the plugins by adding a testing OMERO instance to the test CI. WIP since trial and error more or less happens on github (not sure if it's possible to do that locally).

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 36.21%. Comparing base (83ae41a) to head (5fba538).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #112   +/-   ##
=======================================
  Coverage   36.21%   36.21%           
=======================================
  Files          14       14           
  Lines         947      947           
=======================================
  Hits          343      343           
  Misses        604      604           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[psobolewskiPhD]

I think in theory https://github.com/nektos/act is supposed to let you test actions locally? I've never tried it though.
I tend to work in a fork. A trick i've used before is to change my fork main branch to the PR branch. Then can test a PR against that branch without merging or use manual dispatch.

[psobolewskiPhD]

Based on the hints from Josh and Will, I think we could use something like this:
https://github.com/ome/omero-gallery/blob/master/.github/workflows/omero_plugin.yml

but then in the run: use .omero/compose up -d (honestly not sure about the -d)

Still not sure it's easier to have it in it's own section or as part of the rest of the tests.
Adv. of separate would be probably not needing any of the Qt libs.
But it's two files to maintain shrug

[joshmoore]

I think in theory nektos/act is supposed to let you test actions locally? I've never tried it though.

I've had mixed results with akt. Happy to try to help if it's of interest.

The design of omero-test-infra is intended to make (local) testing a two liner:

git clone ... .omero
.omero/docker "PHASE" # (cli, app, lib, etc.)

Where PHASE is the type of thing you want to happen with your code base. Do you want it installed in the OMERO.web container? in the OMERO.server container? Or somewhere else?

There are then callbacks that runs the pytests within the right container (or externally to the container if necessary) and gives feedback.

[jo-mueller]

but then in the run: use .omero/compose up -d (honestly not sure about the -d)

Without the -d the compose call never finishes and the pytest workflow stops in the OMERO set up step. I had just taken the instructions from the omero-test-infra Readme but copying directly from the CI workflow where it's implemented is probably better. Will do that next 👍

Adv. of separate would be probably not needing any of the Qt libs.

Not sure about that, tbh. A big part of the test imho would be the OMERO browser widget, the ROI uploader widget (the interactive functionality in general), all of which would require both Qt and a remote OMERO instance.

[psobolewskiPhD]

Not sure about that, tbh. A big part of the test imho would be the OMERO browser widget, the ROI uploader widget (the interactive functionality in general), all of which would require both Qt and a remote OMERO instance.

hehe, I was thinking about the ROI uploader specifically -- the widget is just a button that executes something. That something uses OMERO and can be tested independently of the button. Tests of the button can test that the button calls what it's supposed to, but don't need to actually execute anything.
I think the browser is also pretty segmented? But i'd need to take a closer look.

[jo-mueller]

Yosh, by the time my train has passed through the tunnels and my internet is back up I find the commits already reviewed ^^ Thanks a lot for looking over it but please don't feel pressured to invest much time in my WIP at this stage :)

[psobolewskiPhD]

Hehe, I saw failed CI so my detective sense started tingling.
BTW, I'm happy to try to pair on this or do a zoom/video call to discuss.
napari zulip might be the easiest way to coordinate something.

[psobolewskiPhD]

@jo-mueller I started looking at this and I think it's very close!
I'm pushing some changes to this branch, so please pull before doing anything.

[psobolewskiPhD]

@jo-mueller
I think this is pretty much ready to go -- as a starter anyway.
We now have tests/local for things that will not involve the actual OMERO server, so just unit tests and mocks as needed and then there is tests/server which can access the OMERO machinery.
Right now we just test the basic connection process, but I think the machinery is now in place to be able to build out other tests.

Note: I've not done anything locally, just dry-code and CI. So we may need to follow with the contributor guide on testing. That may be act or just having a local docker or whatever, but we'll need to provide some more info.

Edit: there is no real rush on this PR, we should finish the stuff we want for the next release, do the release, and then merge this and then over time build out tests +/- ai code assist.

[psobolewskiPhD]

I don't really get why we need to specify this and didn't have much to go on other than the issue, but without echoing the env vars, the ports are unreachable.

[joshmoore]

The ports are by default randomly generated with the docker-compose set up in order to avoid (local) collisions.

[psobolewskiPhD]

I think it's simpler to have server tests and local tests -- we may end up not even running the server ones on every linux case.

[psobolewskiPhD]

I kinda wonder if we should do the OMERO server tests in a separate CI job? 🤔
It would be easier to control the job matrix...

[psobolewskiPhD]

@jo-mueller I made the change I suggested above: putting the server tests in their own job. Plus some cleanups.
I think this is ready to go. We can then use ezomero on the conftest/fxiture side to handle the boiler plate of getting some images, etc. and then use the server tests to verify integration works as expected for images, roi, etc.