Geometry `__eq__` and `__hash__` violate Python's hash/equality contract
@pnuu is already working on this.
Since Sep 6, 2026.
- Dominant language
- Python
- Stars
- 385
- Forks
- 102
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 9
Description
Bug report written by Claude. I wanted to file this so we had a place to discuss this. I'm not sure who else should be assigned/CC'd to this and who has written resamplers. I generally disagree with Claude about how big of a deal this is for most of our normal use cases where Satpy is handling most of the "oh this dataset has the same area as this one, make sure we use the same area we created before". Claude also doesn't like some of the shortcuts that we do with dask arrays. Bottom line: CRSes aren't as equal as they should be (bug being filed) and floating point stuff is approximate and hashing is exact. I don't want to have to separate paths "exact hashing/equality" and "approximate hashing/equality".
Summary
Python's data model requires a == b to imply hash(a) == hash(b). Pyresample's geometry
classes break that in both directions:
BaseDefinition.__eq__(pyresample/geometry.py:142-174) is deliberately approximate
(np.allclose(..., atol=1e-6, rtol=5e-9, equal_nan=True)).AreaDefinition.__eq__(geometry.py:2115-2127) is approximate on the extent
(np.allclosewith default tolerances) and semantic on the CRS (self.crs == other.crs,
i.e.pyproj.CRS.__eq__, which is PROJ equivalence — not string equality).__hash__/update_hash(geometry.py:123-140,geometry.py:2133-2140) is exact —
SHA-1 over the raw array bytes and over thecrs_wktstring.
Consequences:
{area_a: x}[area_b]raisesKeyErroreven thougharea_a == area_b.len({area_a, area_b}) == 2for equal areas.- Anything keyed on the hash (
BaseResampler.get_hash,
future.resamplers.resampler.hash_resampler_geometries,_caching._hash_args) misses and
recomputes — correctness-preserving, but an invisible and potentially expensive surprise.
The interesting part is that the two geometry classes fail for completely different reasons,
and one of them fails without any numerical difference at all.
AreaDefinition vs SwathDefinition
AreaDefinition |
SwathDefinition (dask-backed) |
SwathDefinition (numpy-backed) |
|
|---|---|---|---|
__eq__ compares |
np.allclose(area_extent), crs == crs (semantic), shape == |
lons.name == other.name (dask task name) |
np.allclose(lons/lats, atol=1e-6, rtol=5e-9) |
update_hash hashes |
crs_wkt string bytes, shape, exact area_extent bytes |
arr.name.encode() — the same task name |
exact array bytes |
| Consistent? | No — on both the CRS and the extent | Yes | No |
| Realistic trigger | a CRS that arrives via a different route (GeoTIFF/GDAL vs YAML/proj-dict) | — | any float noise, e.g. recomputed lon/lats |
Two things follow that are easy to get backwards:
1. The AreaDefinition problem is the CRS, not the extent. The extent is a float
comparison and looks like the obvious suspect, but I could not trigger it through any ordinary
construction path — proj str vs proj dict, pickle, copy(), crs_wkt round-trip and
to_cartopy_crs() round-trip all produce byte-identical WKT and equal hashes. What does
trigger it is a CRS carrying different embedded names for identical parameters, which PROJ
correctly reports as equivalent while the WKT string differs. No numerical difference is
needed. (The extent half is still reachable — the default np.allclose tolerance is rtol=1e-5,
so ~20 m on a 2e6 m extent — but it needs two genuinely different numbers, whereas the CRS half
happens on identical inputs.)
2. The SwathDefinition problem does not exist on the dask path. The dask task-name
short-circuit in __eq__ (geometry.py:159-165) and get_array_hashable's
arr.name.encode('utf-8') branch (geometry.py:798-810) key off the same task name, so they
agree by construction. That short-circuit is intentional: a geometry used as a dict or cache key
is only meaningful within one process and one set of dask operations, and anything higher up
(e.g. Satpy) is expected to reuse the same geometry object and the same graph. Only numpy-backed
swaths — and mixed dask-vs-numpy comparisons, which fall through to np.allclose — violate the
contract.
Reproducers
All output below is from pyresample on main, pyproj 3.7.2, PROJ 9.8.1.
AreaDefinition — equal areas, different hashes, identical numbers
from pyresample import create_area_def
from pyresample.utils.rasterio import get_area_def_from_raster
geotiff_area = get_area_def_from_raster("some_file.tif") # CRS via GDAL/rasterio
proj4_area = create_area_def("a", geotiff_area.crs.to_proj4(),
shape=geotiff_area.shape,
area_extent=geotiff_area.area_extent)
geotiff_area == proj4_area # True
hash(geotiff_area) == hash(proj4_area) # False
geotiff_area.crs_wkt == proj4_area.crs_wkt # False
The WKT differs only in a name:
GeoTIFF crs_wkt : PROJCRS["unknown",BASEGEOGCRS["WGS 84",DATUM["World Geodetic System 1984",...
proj4-built : PROJCRS["unknown",BASEGEOGCRS["unknown",DATUM["World Geodetic System 1984",...
A self-contained version with no data file, using a WKT1 round-trip:
from pyproj import CRS
from pyresample import create_area_def
proj = "+proj=lcc +lat_0=25 +lon_0=-95 +lat_1=25 +lat_2=25 +a=6371229 +b=6371229 +units=m +no_defs"
ext = (-2e6, -2e6, 2e6, 2e6)
a = create_area_def("a", proj, shape=(100, 100), area_extent=ext)
b = create_area_def("b", CRS(a.crs.to_wkt("WKT1_GDAL")), shape=(100, 100), area_extent=ext)
a == b # True
hash(a) == hash(b) # False
The entire WKT difference:
-CONVERSION["unknown", +CONVERSION["unnamed",
-AXIS["(E)", +AXIS["easting",
-AXIS["(N)", +AXIS["northing",
SwathDefinition — dask is fine, numpy is not
import numpy as np
import dask.array as da
from pyresample.geometry import SwathDefinition
lons, lats = da.zeros((5, 5), chunks=5), da.ones((5, 5), chunks=5)
s1, s2 = SwathDefinition(lons, lats), SwathDefinition(lons, lats)
s1 == s2, hash(s1) == hash(s2) # (True, True) consistent
n1 = SwathDefinition(np.zeros((5, 5)), np.ones((5, 5)))
n2 = SwathDefinition(np.zeros((5, 5)) + 1e-9, np.ones((5, 5)))
n1 == n2, hash(n1) == hash(n2) # (True, False) violation
Upstream: pyproj has the same bug
from pyproj import CRS
a = CRS("+proj=lcc +lat_0=25 +lon_0=-95 +lat_1=25 +lat_2=25 +a=6371229 +b=6371229 +units=m +no_defs")
b = CRS(a.to_wkt("WKT1_GDAL"))
a == b # True
hash(a) == hash(b) # False
pyproj.CRS.__hash__ is literally:
def __hash__(self) -> int:
return hash(self.to_wkt())
So pyresample is inheriting the CRS half of this, not inventing it — see
pyproj#1548. A PROJ-level issue about WKT
export not being canonical for equivalent CRSes is being filed separately; note that any fix
upstream would necessarily make pyproj's hash coarser, which is the same trade-off one level up.
Why this is not a one-line fix
hash() on a geometry is currently doing two incompatible jobs:
- Bucketing — dict/set keys (
utils/cf.py:372-378, and Satpy's in-memory resampler
caches). Needs to be consistent with__eq__; collisions are cheap because the dict falls
back to__eq__. - Content addressing — used directly as a cache key or task name with no
__eq__
fallback, where a collision means silently wrong data:pyresample/_caching.py:93—hash(area)in the on-disk JSON key forget_area_slicespyresample/resampler.py:286—tokenize(func, hash(src_area), ..., hash(dst_area), ...),
the dask task name inresample_blockssatpy/modifiers/angles.py:248—hash(area)in the on-disk zarr key for the
sun/satellite angles cache
Making __hash__ consistent with __eq__ means making it coarser, so those three consumers
must first move to update_hash().hexdigest() (and Satpy's would need to land and release
first, or a pyresample release ships an angles cache that can return another area's angles).
pyresample/resampler.py:69-78 and future/resamplers/resampler.py:45-50 already use
update_hash() and are unaffected.
One non-obvious constraint: hash(area) is currently int(sha1_hexdigest, 16), and Python
does not salt int hashes with PYTHONHASHSEED — which is exactly why Satpy's on-disk cache
works across processes. Any replacement that mixes a str or bytes into hash(...) (a dask
task name, a WKT string) silently becomes per-process, and the failure mode is not an error but
a cache that never hits and grows one file per run.
Options
- Make the hash approximate too — quantize onto a grid coarser than the matching
allclose
tolerance, and drop the CRS from__hash__(no function ofcrs_wktcan be consistent with
a semanticcrs ==). This cannot fully restore the contract: for valuesδapart on a grid
of steph, the probability of landing in different buckets isδ/h, so a 10x margin turns
"always wrong for near-equal geometries" into "wrong ~5% of the time". Needs the consumer
migration above plus a_cachingcache_versionbump (see SPEC-003). - Make
__eq__exact and add a separateis_close(other, atol=...). Cleaner semantics, but
a breaking change for Satpy and for anyone relying on tolerant equality — including the
GeoTIFF-vs-YAML case above, which users legitimately want to compare equal. - Document the violation and leave it. Cheapest; the failure mode (a cache that silently
never hits) stays invisible.
Worth knowing while choosing: the only attribute compared exactly by every __eq__ in the
hierarchy is .shape. hash(self.shape) is therefore the only variant that is correct by
construction, and the only one that also covers the cross-type case — AreaDefinition.__eq__
falls back to super().__eq__ when the other object has no area_extent, so an area can compare
equal to a swath (pinned by test_area.py::TestAreaComparisons::test_swath_equal_area), and no
extent-based or lonlat-based hash can make those two agree. The cost is that same-shaped
geometries share a dict bucket.
Whichever is chosen, the decision is user-visible and belongs in
docs/source/concepts/geometries.rst.
Suggested test
a = create_test_area(...)
b = create_test_area(...) # same, with area_extent nudged by 1e-9
assert (a == b) == (hash(a) == hash(b))
Better as a property test over a corpus that includes: identical areas, extent-perturbed areas,
semantically-equal-CRS-different-WKT areas, a swath built from area.get_lonlats(), and
numpy/dask/xarray swaths — asserting a == b ⇒ hash(a) == hash(b) across the full cross product.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.