YAML area configuration does not allow to specify the dtype and dtype is ignored in equality comparisons
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 385
- Forks
- 102
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 9
Description
Today I learned that an areadefinition has a dtype attribute. This seems to be not very well documented and not very well supported, but it's relevant. It is used at least in get_lonlats(), which is used in resampling, so changing the dtype can have considerable memory implications and make the difference between having enough RAM or not.
Code Sample, a minimal, complete, and verifiable piece of code
import pyresample
ar = pyresample.AreaDefinition(
"area_id", "descr", "proj_id", 4326, 100, 100,
[-100, -100, 100, 100], dtype="float32")
s = ar.dump()
print(s)
ar2 = pyresample.area_config.load_area_from_string(s)
print("equals?", ar == ar2)
print("dtypes", ar.dtype, ar2.dtype)
s2 = """area_id:
description: descr
projection:
EPSG: 4326
shape:
height: 100
width: 100
area_extent:
lower_left_xy: [-100, -100]
upper_right_xy: [100, 100]
dtype: float32
"""
ar3 = pyresample.area_config.load_area_from_string(s2)
print(ar3.dtype)
Problem description
Executing the code reveals several problems:
ar.dump()does not output the dtypearandar2are considered equal despite having different dtypesar3, loaded from a string that does encode the dtype, getsfloat64despite the YAML definition statingfloat32.
It does not appear to be possible to specify the dtype in the YAML configuration (nor in pyresample.create_area_def()).
Expected Output
area_id:
description: descr
projection:
EPSG: 4326
shape:
height: 100
width: 100
area_extent:
lower_left_xy: [-100, -100]
upper_right_xy: [100, 100]
dtype: float32
equals? True
dtypes float32 <class 'numpy.float32'>
<class 'numpy.float32'>
Actual Result, Traceback if applicable
area_id:
description: descr
projection:
EPSG: 4326
shape:
height: 100
width: 100
area_extent:
lower_left_xy: [-100, -100]
upper_right_xy: [100, 100]
equals? True
dtypes float32 <class 'numpy.float64'>
/data/gholl/checkouts/pyresample/pyresample/area_config.py:91: UserWarning: Unused/unexpected area definition parameter(s) for area_id: params={'dtype': 'float32'}
area_list = parse_area_file(area_file_name, *regions)
<class 'numpy.float64'>
Versions of Python, package at hand and relevant dependencies
pyresample main (v1.28.2-2-g711f354)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with pyresample/area_config.py and the AreaDefinition dump/load and equality entry points; trace how dtype is passed by load_area_from_string and create_area_def. Done means YAML serialization preserves dtype, YAML loading and create_area_def accept it, and differing dtypes affect equality, with regression coverage for the sample cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python, yaml
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100