AllenNeuralDynamics / AllenNeuralDynamics/aind-smartspim-data-transformation
Allow de-activated tiles
- Linguagem predominante
- Python
- Estrelas
- 1
- Forks
- 0
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
## Motivation
Acquisitions can only be set up as rectangular grids of tiles, but some acquisitions have few to many tiles that do not contain tissue.
Examples are the corners of regular horizontally oriented mouse brain and most of the tiles in a "brain + full spinal cord."
Since the acquisition software supports deactivating these tiles, it is forward looking to support it here as well.
Deactivated tiles manifest as folders with exactly one image (the 000000.png file).
This would have saved hours of acquisition time for the spinal cord brains, saves data size on disk via better compression, and keeps up with the acquisition software scope more generally.
## Solution (proposed)
**Describe the solution you'd like**
Include a condition in the OME-Zarr writer that writes single image tiles as either 0 filled blocks of 110 valued (or 105? whatever our dark tile is) arrays.
Being single valued arrays should make them highly compressible, and so still a lightweight part of the dataset.
I think that the only wrinkle would be that we would need to add an input parameter to `_write_stacks` that is the expected shape in the depth dimension, as determined by examining non-singleton tiles.
## Outline
Something like:
1. Additional function `find_tile_shape(dataset | channel)` returns `(tile["images"][0].shape, ) + (len(tile["images"],)`. Looks at all tiles until a `>1 images` one is found.
2. `def _write_stacks(self, stacks_to_process: List, expected_shape: List)`
a. in the `for stack in stacks` loop, `image_files = list(stack.glob("*")); if len(image_files) > 1: active_tile = True`.
b. `delayed_stack = PngTiffReader().as_dask_array() if active_tile else delayed_stack = 110 * da.zeros(expected_shape)`
Open to comments on this @camilolaiton, it is definitely not a priority but might not be that difficult to implement
Guia de contribuição
Nenhum guia de contribuição indexado para este repositório
Direção de pesquisa
Start by locating the OME-Zarr writer and its _write_stacks entry point, then trace how tile image folders become stacks. Compare folders containing only 000000.png with multi-image tiles and determine the expected depth shape from a non-singleton tile. Done means inactive tiles produce correctly shaped constant dark arrays while active tiles retain the current output.
Escrita pelo modelo de indexação a partir do texto da issue.
Avaliação
- Stack de tecnologia
- python
- Domínio
- data-engineering
- Tipo de issue
- Funcionalidade
- Dificuldade
- 3/5
- Tempo estimado
- 1-2 dias
- Status de atividade
- Pouca atividade
- Clareza
- Razoavelmente clara
- Facilidade para iniciantes
- 56/100