Allow nesting HCS and Scenes?

Relevant for #441 , see also https://forum.image.sc/t/how-to-build-hcs-zarrs-with-multiple-image-types-per-fov/119329.

It was asked/stated in a few places that people would like to store additional image data (i.e., a derived image, co-registered images, etc) inside their HCS storage layouts. I think this usecase lies at the intersection of two features of the spec, which are blocked from co-existence by small wording choices.

How a scene can help here

If one "only" wants to store multiple copies (i.e., raw image, prediction, processed image, etc) in a defined layout, the "Scene" lets you do this; All of these images would share the same pixel space, so transformations between such images would likely be identity transforms. Such a scene metadata could look like this:

{
  "ome": {
    "scene": {
      "coordinateTransformations": [
        {
          "type": "identity",
          "input": {"path": "imageA", "name": "physical"},
          "output": {"path": "imageA_blurred", "name": "physical"}
        }
      ]
    }
  }
}

and the corresponding storage layout would look like this:

root
├── imageA
|    ├──zarr.json
|    ├──s0
|    └──s1
├── imageA_blurred
|    ├──zarr.json
|    ├──s0
|    └──s1
└  zarr.json  # scene metadata is here

How a scene inside a well could look like

I think there are two scenarios for practical realization of a "Scene" storage layout inside a "Well layout:

Option 1: Nesting

Nesting the two structure could look something like this:

HCS_root
├── A/1/
|   ├── zarr.json  # Well metadata is here
|   ├── imageA_scene  # would usually be a multiscale, now a scene with two images
|   |   ├── imageA
|   |   |    ├──zarr.json
|   |   |    ├──s0
|   |   |    └──s1
|   |   ├── imageA_blurred
|   |   |    ├──zarr.json
|   |   |    ├──s0
|   |   |    └──s1
|   |   └  zarr.json  # scene metadata is here
|   └── imageB_scene  # would usually be a multiscale, now a scene with two images
|       ├── imageB
|       |    ├──zarr.json
|       |    ├──s0
|       |    └──s1
|       ├── imageB_blurred
|       |    ├──zarr.json
|       |    ├──s0
|       |    └──s1
|       └  zarr.json  # scene metadata is here
├── A/2/
...

For this we would need a wording change. The spec currently defines the layout of the HCS structure like this:

the group above the images defines the well and MUST implement the well specification. All images contained in a well are fields of view of the same well

This is obviously in disagreement because the group above the images would define a Scene here and not a well. So a suggestion for change could be (top of my head):

the group above the images defines the well or an intermediary Scene and MUST implement the well specification in the corresponding parent folder

Option 2: Coexistence

Technically, one could argue that "All images contained in a well are fields of view of the same well" is true for derived copies of images inside a well. This interpretation would give way to the following layout:

HCS_root
├── A/1/
|   ├── zarr.json  # Well & scene metadata live here
|   ├── imageA
|   |   ├──zarr.json
|   |   ├──s0
|   |   └──s1
|   ├── imageA_blurred
|   |   ├──zarr.json
|   |   ├──s0
|   |   └──s1
|   ├── imageB
|   |    ├──zarr.json
|   |    ├──s0
|   |    └──s1
|   └── imageB_blurred
|        ├──zarr.json
|        ├──s0
|        └──s1
├── A/2/
...

The corresponding metadata would contain both the Well and the Scene metadata:

{
  "ome": {
    "well": {
      "images": [
        {
          "acquisition": 1,
          "path": "imageA"
        },
        {
          "acquisition": 1,
          "path": "imageA_blurred"
        },
        {
          "acquisition": 2,
          "path": "imageB"
        },
        {
          "acquisition": 2,
          "path": "imageB_blurred"
        }
      ]
    },
    "scene": {
      "coordinateTransformations": [
        {
          "type": "identity",
          "input": {"path": "imageA", "name": "physical"},
          "output": {"path": "imageA_blurred", "name": "physical"}
        },
        {
          "type": "identity",
          "input": {"path": "imageB", "name": "physical"},
          "output": {"path": "imageB_blurred", "name": "physical"}
        },
        {
          "type": "translation",
          "input": {"path": "imageA", "name": "physical"},
          "output": {"name": "wellA1_world"},
          "translation": [...]
        },
        {
          "type": "translation",
          "input": {"path": "imageB", "name": "physical"},
          "output": {"name": "wellA1_world"},
          "translation": [...]
        }
      ],
      "coordinateSystems" : [
        {
          "name": "wellA1_world",
          "axes": [...]
        }

      ]
    }
  }
}

The requisite for allowing such a layout would be store the spatial relationship between acquisitions/fields of view as translations in the metadata. I think this is valid because multiple acquisitions/fields of view inside the same well are typically acquired at different spatial locations and are thus allowed inside the scene metadata. The hardcore part here would ofc be the feature of having multiple kinds of metadata definitions in the same zarr.json, which would be absolutely unheard of. Still, I wanted to have it written out somewhere.

Preference

My personal choice would be to enable Option 1 (nesting) because I imagine it massively simpler from an implementation perspective. A tool that's traversing the HCS layout would just check whether it's looking at an image or a scene; If it foesn't want to bother with parsing the scene, it could simply squash that group internally and treat everything as separate images in a single Well. At the very least, it wouldn't necessarily need to bother with complex transforms to know that the images in a Scene share some sort of relationship.

I'll stop my Essay here - let me know what you think of the idea. Too breaking? Undesirable? Too complex? Out of scope because Collections will enable this anyway?

LMKWYT :)

cc @mat10d @jluethi

Start with the linked HCS and Well specification sections, then compare the proposed nesting and coexistence layouts with their metadata examples. Determine which approach should be specified and how well and scene metadata should interact. Done means an agreed specification change with updated layout and metadata examples.