facebookresearch / facebookresearch/detectron2
Support reading TIFF images using imageio and int16 value types with imgaug
- Dominant language
- Python
- Stars
- 34.7k
- Forks
- 7.9k
- PR merge metrics
- No merged PRs in 30d
Description
## 🚀 Feature
imageio can read .tif files whereas Pillow cannot support all value types within TIFF files: https://imageio.readthedocs.io/en/latest/formats.html
Can we use imageio rather than (or along with) Pillow in this detection_utils imread function?https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/detection_utils.py#L36 ?
Edit: In order to support data types like Int16, Pillow's operations that are used in https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/transforms/transform.py would also need to be replaced by another library that supports data types beyond 8-bit integers. scikit image has functions for rotation, rescaling, etc. https://scikit-image.org/docs/dev/api/skimage.transform.html
## Motivation
TIFF is an image format that supported multichannel float or int16 images. the Pillow library does not and it looks like it doesn't intend to, which is a bummer for science domains that mostly use TIFF https://github.com/python-pillow/Pillow/issues/1888)
TIFF is the main format used to store satellite and aerial imagery and it is also used in microscopy (to name a few cases). There are many applications for detectron2 in geospatial ML. As an example, a popular building segmentation challenge, Spacenet, distributes satellite imagery in TIFF format (the geotiff variant of TIFF can be read by imageio as well) https://spacenetchallenge.github.io/AOI_Lists/AOI_2_Vegas.html
## Pitch
Since it is still convenient to have Pillow's ability to detect BGR or RGB format, I propose a simple check for the .tif or .tiff extension before an image is read, and if it exists, imageio is used instead Pillow. I haven't encountered any other image formats used in computer vision besides PNG and JPEG (most common for natural imagery) and TIFF (most common for geospatial imagery) so I think this catch would be sufficient (for image reading, replacing Pillow's transforms with scikit-image would be another task). Curious to hear if this seems like the best way or not. and thanks for considering!
```
def read_image(file_name, format=None):
"""
Read an image into the given format. Supports PNG, JPEG or TIFF
Will apply rotation and flipping if the image has such exif information.
Args:
file_name (str): image file path
format (str): one of the supported image modes in PIL, or "BGR"
Returns:
image (np.ndarray): an HWC image
"""
if file_name.lower().endswith(('.tiff', '.tif')):
image = imageio.imread(file_name)
return image
else:
with PathManager.open(file_name, "rb") as f:
image = Image.open(f)
# capture and ignore this bug: https://github.com/python-pillow/Pillow/issues/3973
try:
image = ImageOps.exif_transpose(image)
except Exception:
pass
if format is not None:
# PIL only supports RGB, so convert to RGB and flip channels over below
conversion_format = format
if format == "BGR":
conversion_format = "RGB"
image = image.convert(conversion_format)
image = np.asarray(image)
if format == "BGR":
# flip channels if needed
image = image[:, :, ::-1]
# PIL squeezes out the channel dimension for "L", so make it HWC
if format == "L":
image = np.expand_dims(image, -1)
return image
```
Contributor guide
Assessment
This issue has not been assessed yet.