Image Functions

Nobody has claimed this yet.

As discussed with @LucaMarconato @melonora and @josenimo, I think it would be great to extend the functionality and ease of use of spatialdata by adding a few functions. When I first started using spatialdata I ran into a few issues like napari crashing due to image size, difficult image loading and overall very laborious image handling. The functions I suggest to implement are as follows:

Image Loader for universal file formats such as JPG, PNG, TIFF, Ome-TIFF
-> from my testing skimage.imread is able to handle all 4 of these formats, maybe a simple addition to Image2DModel or another simple function that loads any image into a dask array?
Image size/available GPU memory check
-> warn the user if the image size exceeds VRAM since big images may cause napari to stutter/crash
-> @melonora suggested the vispy.gloo.gl package
A Check/hook to verify that the image has been written to zarr before opening in napari

example for cpu check

def estimate_memory_requirements(dask_array):
    # Calculate total number of elements in the array
    num_elements = dask_array.size
    # Determine the size of each element in bytes
    element_size = dask_array.dtype.itemsize
    # Total memory requirement in bytes
    total_memory = num_elements * element_size
    
    return total_memory

def check_system_resources(memory_required):
    # Check available RAM
    available_ram = psutil.virtual_memory().available
    
    # Assuming that we will need approximately the same amount of GPU memory
    try:
        import pynvml
        pynvml.nvmlInit()
        handle = pynvml.nvmlDeviceGetHandleByIndex(0)
        available_gpu_memory = pynvml.nvmlDeviceGetMemoryInfo(handle).free
        pynvml.nvmlShutdown()
    except ImportError:
        available_gpu_memory = None  # pynvml is not installed or GPU is not available
    
    ram_sufficient = memory_required <= available_ram
    gpu_sufficient = available_gpu_memory is None or memory_required <= available_gpu_memory
    
    return ram_sufficient, gpu_sufficient, available_ram, available_gpu_memory

def load_and_check_image(image_path):
    dask_array = dask_image.imread.imread(image_path)
    memory_required = estimate_memory_requirements(dask_array)
    
    ram_sufficient, gpu_sufficient, available_ram, available_gpu_memory = check_system_resources(memory_required)
    
    if not ram_sufficient:
        print(f"\U00002757 Warning: Not enough RAM. Required: {memory_required / (1024**3):.2f} GB, Available: {available_ram / (1024**3):.2f} GB")
    if gpu_sufficient is False:
        print(f"\U00002757 Warning: Not enough GPU memory. Required: {memory_required / (1024**3):.2f} GB, Available: {available_gpu_memory / (1024**3):.2f} GB")
    
    if ram_sufficient and (gpu_sufficient is None or gpu_sufficient):
        print("\U00002705 System resources are sufficient to handle the image load.")
    else:
        print("\U0000274C System resources are insufficient to handle the image load. Downscaling recommended.")
    
    return dask_array

Start by reviewing Image2DModel and the current image-loading flow, then assess the proposed skimage.imread loader, vispy.gloo.gl resource check, and zarr-write verification. Before implementation, define acceptance criteria for supported formats, memory warnings, and napari-opening behavior.

Written by the indexing model from the issue text.

Description

Contributor guide

First steps

Research direction

Assessment