AllenNeuralDynamics / AllenNeuralDynamics/aind-smartspim-data-transformation

Allow de-activated tiles

Aberta
#31 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
1
Forks
0
Métricas de merge de PRs
Nenhum PR com merge em 30d

Descrição

## Motivation
Acquisitions can only be set up as rectangular grids of tiles, but some acquisitions have few to many tiles that do not contain tissue.
Examples are the corners of regular horizontally oriented mouse brain and most of the tiles in a "brain + full spinal cord."
Since the acquisition software supports deactivating these tiles, it is forward looking to support it here as well.
Deactivated tiles manifest as folders with exactly one image (the 000000.png file).

This would have saved hours of acquisition time for the spinal cord brains, saves data size on disk via better compression, and keeps up with the acquisition software scope more generally.

## Solution (proposed)
**Describe the solution you'd like**
Include a condition in the OME-Zarr writer that writes single image tiles as either 0 filled blocks of 110 valued (or 105? whatever our dark tile is) arrays.
Being single valued arrays should make them highly compressible, and so still a lightweight part of the dataset.

I think that the only wrinkle would be that we would need to add an input parameter to `_write_stacks` that is the expected shape in the depth dimension, as determined by examining non-singleton tiles.

## Outline
Something like:
1. Additional function `find_tile_shape(dataset | channel)` returns `(tile["images"][0].shape, ) + (len(tile["images"],)`. Looks at all tiles until a `>1 images` one is found.
2. `def _write_stacks(self, stacks_to_process: List, expected_shape: List)`
a. in the `for stack in stacks` loop, `image_files = list(stack.glob("*")); if len(image_files) > 1: active_tile = True`.
b. `delayed_stack = PngTiffReader().as_dask_array() if active_tile else delayed_stack = 110 * da.zeros(expected_shape)`

Open to comments on this @camilolaiton, it is definitely not a priority but might not be that difficult to implement

Guia de contribuição

Nenhum guia de contribuição indexado para este repositório

Direção de pesquisa

Start by locating the OME-Zarr writer and its _write_stacks entry point, then trace how tile image folders become stacks. Compare folders containing only 000000.png with multi-image tiles and determine the expected depth shape from a non-singleton tile. Done means inactive tiles produce correctly shaped constant dark arrays while active tiles retain the current output.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
python
Domínio
data-engineering
Tipo de issue
Funcionalidade
Dificuldade
3/5
Tempo estimado
1-2 dias
Status de atividade
Pouca atividade
Clareza
Razoavelmente clara
Facilidade para iniciantes
56/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.