Streamline input output constraints

[jo-mueller]

Fixes ome/ngff#523
Fixes #159

• removes some duplicate text and adds more descriptive tables to the transformations section to make the whole paragraph a bit easier to read.
• synchronizes the constraints introduced for transformation input/output fields across the spec so that they are progressively narrowed down from broad to context-dependend tight
• relaxes restriction on directionality in ms > coordinateTransformations, which makes the text cleaner and has no practical implication since we are only allowing identity/scale/translate/sequence here.

cc @btbest

[github-actions]

Automated Review URLs

• Readthedocs

[jo-mueller]

Also, I noticed that in #117 , I introduced a constraint that would require all transformations under multiscale > coordinateTransformations to have the same directionality, i.e., "intrinsic" should be referenced by input, and the other should be the output, which is unnecessarily strict. I am removing this constraint here, which makes things easier.

cc @btbest @will-moore @clbarnes @tlambert03 @dstansby @kevinyamauchi

[btbest]

Happy with loosening the directionality, but the current MAY phrasing accidentally reintroduces the "multiscale transforms can point anywhere" problem:

• One of input or output MUST reference the "intrinsic" coordinate system by name (path MAY be omitted or null).
• The other MAY reference a named coordinate system in the same multiscales group (by name, path MAY be omitted or null), or in a child labels group (by name and path).
• When referencing a child labels group, the transformation MUST be identity, scale, or translation.

The MAY in the second bullet point is the catch. Maybe I just have mischief baked in, but I immediately read this as "Well, you never told me I can't reference some other multiscale or a scene".

This should say "The other MUST reference a coordinate system either in... or in..." (the same goes for line 1328)

[jo-mueller]

This should say "The other MUST reference a coordinate system either in... or in..." (the same goes for line 1328)

Right!

[jo-mueller]

@tlambert03 would you mind you have a quick look at this branch and see if you find the wording around the constraints around the input and output fields and where they apply clearer here? 🙂

[btbest]

Looks good as is, I just added some minor phrasing suggestions.

Remembering that any multiscales group can technically define multiple multiscales actually presents a semantic problem: the name + path referencing isn't ambiguous if the multiscales group at the path defines multiple multiscales whose intrinsic systems are identically named. I doubt this would be an issue in practice because nobody uses multiple entries in the multiscales array, but it's theoretically there.

I think to have version 0.6 being clean, we'd need the "The multiscales array MUST contain exactly one multiscale object` constraint that has been discussed before (but I think got delayed because it was bundled with flattening the multiscales-list to a single object, making it a breaking change for almost 0 gain).

[btbest]

Need to check again but aren't labels required to be the same shape as the multiscale; which means this basically always has to be identity? Allowing scale and translation isn't a problem of course, just presents more opportunities for confusion I suppose.

[jo-mueller]

That's the part where we are entering the grey zone between collections and transforms. The relationship between labels and images is first and foremost semantic in nature, not spatial. Hence 0.6 will not force you to add spatial information to your label/image connection, but if you want to, you can.

Hence, the shaping of the labels has never been written out in previous versions and 0.6 won't force this because only collections will express this relationship properly.

[btbest]

The relationship between labels and images is first and foremost semantic in nature, not spatial

Subtle disagreement from the spec itself there:

Some applications–notably image segmentation–produce a new image that is in the same coordinate system as a corresponding multiscale image (usually having the same dimensions and coordinate transformations).

Sounds very much spatial to me :)

Just to be clear, I'm not actually asking for anything to be changed here. Just indulging.

[jo-mueller]

Oh, I see what you mean. The loophole here is "usually"; There was a PR where thought of changing that (possibly #117) and require this transformation to be explicit. Ultimately we decided against it:

• If that transformation were to live in the parent multiscale image, then the addition of a child labels image would effectively invalidate the parent image because that transform would then be missing
• If the transform were to live in /labels/segmentation_image/zarr.json, its output would have to be ../../zarr.json, which would also be problematic because moving the label image elsewhere would make it invalid and thus no longer self-describing.

The clean option would be to put the label image into a transformation graph and not use the /labels/... structure at all. Or wait for collections that will make this kind of linkage much more explicit from the point of view of an apex zarr.json in the hierarchy, which would be my preferred solution :)

[btbest]

I also don't see any point to requiring the metadata to enumerate all contents. At most I'd go to "SHOULD" level - just like the labels metadata only "SHOULD" reference all label multiscales available.

[jo-mueller]

Remembering that any multiscales group can technically define multiple multiscales actually presents a semantic problem: the name + path referencing isn't ambiguous if the multiscales group at the path defines multiple multiscales whose intrinsic systems are identically named. I doubt this would be an issue in practice because nobody uses multiple entries in the multiscales array, but it's theoretically there.

Gulp...there's an entire RFC around this (with a complicated history, I think ^^"). Essentially, it was decided to also defer to this to collections, because collections will arrange this bit of metadata anyway. As you say, so far there haven't really been any datasets in the wild that would make use of the array structure under the multiscales field so it's something I think we can live with in the meantime.

[jo-mueller]

Follow-up: Since we are using the term multiscale group very extensively throughout the spec I'll avoid changing this terminology too much here, but this doesn't mean that it couldn't be changed in a follow-up PR (even in an 0.6.1 version, since this is really just phrasing, not semantics).

[jo-mueller]

I agree that this statement would be the cleanliest way out of that issue:

The multiscales array MUST contain exactly one multiscale object

But given the discussion around it, I think it's safer to just let it rest as it is for the moment and save the cleaning up for 1.0 😬