facebookresearch / facebookresearch/detectron2

Support reading TIFF images using imageio and int16 value types with imgaug

Open
#698 5 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature

imageio can read .tif files whereas Pillow cannot support all value types within TIFF files: https://imageio.readthedocs.io/en/latest/formats.html

Can we use imageio rather than (or along with) Pillow in this detection_utils imread function?https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/detection_utils.py#L36 ?

Edit: In order to support data types like Int16, Pillow's operations that are used in https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/transforms/transform.py would also need to be replaced by another library that supports data types beyond 8-bit integers. scikit image has functions for rotation, rescaling, etc. https://scikit-image.org/docs/dev/api/skimage.transform.html

## Motivation

TIFF is an image format that supported multichannel float or int16 images. the Pillow library does not and it looks like it doesn't intend to, which is a bummer for science domains that mostly use TIFF https://github.com/python-pillow/Pillow/issues/1888)

TIFF is the main format used to store satellite and aerial imagery and it is also used in microscopy (to name a few cases). There are many applications for detectron2 in geospatial ML. As an example, a popular building segmentation challenge, Spacenet, distributes satellite imagery in TIFF format (the geotiff variant of TIFF can be read by imageio as well) https://spacenetchallenge.github.io/AOI_Lists/AOI_2_Vegas.html

## Pitch
Since it is still convenient to have Pillow's ability to detect BGR or RGB format, I propose a simple check for the .tif or .tiff extension before an image is read, and if it exists, imageio is used instead Pillow. I haven't encountered any other image formats used in computer vision besides PNG and JPEG (most common for natural imagery) and TIFF (most common for geospatial imagery) so I think this catch would be sufficient (for image reading, replacing Pillow's transforms with scikit-image would be another task). Curious to hear if this seems like the best way or not. and thanks for considering!

```
def read_image(file_name, format=None):
"""
Read an image into the given format. Supports PNG, JPEG or TIFF
Will apply rotation and flipping if the image has such exif information.

Args:
file_name (str): image file path
format (str): one of the supported image modes in PIL, or "BGR"

Returns:
image (np.ndarray): an HWC image
"""

if file_name.lower().endswith(('.tiff', '.tif')):
image = imageio.imread(file_name)
return image
else:
with PathManager.open(file_name, "rb") as f:
image = Image.open(f)

# capture and ignore this bug: https://github.com/python-pillow/Pillow/issues/3973
try:
image = ImageOps.exif_transpose(image)
except Exception:
pass

if format is not None:
# PIL only supports RGB, so convert to RGB and flip channels over below
conversion_format = format
if format == "BGR":
conversion_format = "RGB"
image = image.convert(conversion_format)
image = np.asarray(image)
if format == "BGR":
# flip channels if needed
image = image[:, :, ::-1]
# PIL squeezes out the channel dimension for "L", so make it HWC
if format == "L":
image = np.expand_dims(image, -1)
return image
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.