dssim-core 3.5.1 changes 216/216 DSSIM scores (worst 17.8%) — lock pinned to 3.4.0, requirement still admits it

What

dssim-core 3.5.1 does not agree with 3.4.0. Every score this crate reports through metrics::dssim::calculate_dssim moves.

Measured over a 12-size × 3-seed × 6-quantisation grid (216 reference/distorted pairs), hashing each f64 result bit-exactly rather than comparing rounded output:

• 216 of 216 cases differ (100%)
• worst relative delta 17.8% — at 8×8, seed 2, q=2: 2.48587047169124276e-05 → 2.92857097352605678e-05

The mean barely moves, which is exactly why this is dangerous: an aggregate check would call it noise. It is not noise — it is systematic, it touches every case, and on small/low-distortion images it is nearly a fifth of the value.

Why it matters here

codec-eval is a measurement tool. Its whole output is quality numbers meant to be compared across runs, codecs, and dates. A metric shift of this size silently makes every previously published number incomparable with anything produced afterwards — and when someone notices the delta, the natural reading is "this codec regressed", not "the metric changed underneath us".

What is in place now

Cargo.lock (which is committed here) pins dssim-core 3.4.0, so nothing moves for anyone building with the committed lock or --locked.

But the requirement does not hold it. Cargo.toml has:

dssim-core = "3.4.0"

which is a caret range and still admits 3.5.1 on any unlocked resolve — a fresh cargo update, a downstream consumer, or CI without --locked will take 3.5.1 again and silently shift every score.

Decision needed

1. Narrow the requirement to hold the metric — dssim-core = ">=3.4.0, <3.5.0" or =3.4.0. Safest for comparability; costs you upstream fixes in the 3.5 line.
2. Adopt 3.5.1 deliberately — take the new scores, and bump codec-eval's own version with a changelog note loud enough that nobody compares pre- and post- numbers. If 3.5's change is a correctness fix, this is the right call.
3. Investigate which of the two is more correct before choosing. Upstream's changelog for 3.5.0 would say whether this was intentional.

I did not narrow the requirement myself — that is a policy decision about the crate's public contract, not something a dependency refresh should make unilaterally.

Reproducing

Drive codec_eval::metrics::dssim::calculate_dssim over deterministic synthetic pairs (structured rings + gradients, quantised to simulate compression loss) at sizes 8×8 … 256×256 including non-multiple-of-8 shapes, 3 seeds, quantisation steps {2,4,8,16,32,64}; hash score.to_bits() for all 216. Flip dssim-core between 3.4.0 and 3.5.1 with cargo update -p dssim-core --precise.