arkavo-org / arkavo-org/VRMMetalKit

MToon rim lighting: high-mix rim-extent residual — VMK's rim doesn't wrap the unlit silhouette at rimLightingMix→1 (SSIM ~0.949 vs 0.95, only at max rim)

Open
#226 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
6
Forks
2
Avg merge
18h 51m
Merged PRs (30d)
26

Description

## Summary

New finding revealed by the vrm-conformance sRGB-encoding fix ([vrm-conformance@fd8221d](https://github.com/arkavo-org/vrm-conformance/commit/fd8221d) — closes [VMK#213](https://github.com/arkavo-org/VRMMetalKit/issues/213)'s root cause for shadingShift). VRMMetalKit's rim-fresnel rendering diverges from the three other MToon implementations on the conformance corpus's six `mtoon_rimLightingMix_*` tests. The divergence was masked pre-encoding-fix by a coincidence of linear-vs-sRGB byte alignment in the rim brightness range; with proper sRGB encoding now in place, the rim cluster is the largest remaining VMK ↔ UniVRM gap.

cc @arkavo-com (per offer to take this in parallel with #214)

## Versions

- VRMMetalKit: 0.13.5 + adapter sRGB fix landed on conformance `fd8221d`
- Reference renderers: UniVRM v0.131.0 (Unity 6 PlayMode), three-vrm 3.5.0, godot-vrm 4.6.2

## Observed: VMK is the only outlier; sweep-invariant divergence

Full pairwise SSIM matrix on `mtoon_rimLightingMix_0` (identical numbers on every test in the sweep — see below):

| Pair | SSIM |
|---|---|
| three-vrm ↔ UniVRM | **0.9902** |
| godot-vrm ↔ UniVRM | 0.9793 |
| three-vrm ↔ godot-vrm | 0.9902 |
| VMK ↔ three-vrm | 0.9144 |
| VMK ↔ godot-vrm | 0.9196 |
| **VMK ↔ UniVRM** | **0.9078** |

Three of the four pairs cluster at 0.98+. VMK's three pairs all sit in the 0.90–0.92 band. **VMK is the outlier**, separated from the consortium reference by ~0.08 SSIM that the other two non-reference implementations don't carry.

## The sweep-invariance is the strongest diagnostic signal

The corpus emits six rim tests with `parametricRimColorFactor = [1.0, 0.5, 0.0]` (orange), `parametricRimFresnelPowerFactor = 5.0`, and `rimLightingMixFactor` swept across `{0.0, 0.1, 0.25, 0.5, 0.75, 1.0}`. The expected behavior is that as `rimLightingMixFactor` increases, the orange rim becomes increasingly visible at the sphere's silhouette, with `0.0` showing no rim influence at all and `1.0` showing maximum rim.

Empirically, **every renderer pair produces the exact same SSIM across the entire `rimLightingMixFactor` sweep**:

```
mtoon_rimLightingMix_0 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p1 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p25 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p5 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p75 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_1 VMK↔UniVRM = 0.9078
```

Identical across the sweep, identical across pairs (each pair has its own constant). That's not noise — it's a **structural constant offset**. Either:

- (a) The parameter sweep isn't producing visibly different output in any renderer (the orange rim isn't visible at any mix factor), and the constant 0.9078 is the underlying material-baseline disagreement between VMK and UniVRM on the rim-color-zero render that all six tests collapse to. *or*
- (b) Every renderer correctly modulates rim contribution with `rimLightingMixFactor`, but VMK's rim contribution carries a fixed-magnitude error (e.g., a coordinate-space mismatch in the fresnel term) that produces the same SSIM gap regardless of the mix factor. The mix factor multiplies the whole rim term, which would produce a *varying* SSIM under most error models — but a coordinate-space bug that affects only the *direction* of the fresnel falloff (not its magnitude) would produce a constant offset because the visible rim band has the same total energy, just in slightly the wrong place.

(b) is the more interesting hypothesis. A fresnel coordinate-space bug — for instance, view-space vs world-space normal in the `dot(N, V)` term, or using `objectFromCamera` where the spec expects `cameraToObject` — would shift the rim band's *position* along the silhouette without changing its *intensity*. SSIM penalizes the position shift, the mix factor doesn't change the shift, the SSIM is constant across the sweep. That fits the data.

## Hypothesis to investigate first

The MToon-1.0 spec ([§3.7 Rim Lighting](https://github.com/vrm-c/vrm-specification/blob/master/specification/VRMC_materials_mtoon-1.0/README.md#rim-lighting)) defines the fresnel rim factor as `pow(1.0 - max(0.0, dot(viewDir, normalDir)), rimFresnelPower)` — where `viewDir` is from the surface point to the camera and `normalDir` is the shading normal, **both expressed in world space** (consistent with the rest of the MToon shader's vector conventions).

Suggested check: `MToonShader.metal` rim-term computation — verify `viewDir` and `normalDir` are in matching coordinate spaces. UniVRM's `Vrm10MToonShader.hlsl` Built-in RP path uses world-space normals; three-vrm's `mtoon_frag.glsl.js` uses world-space. If VMK's rim term reads `viewDir` from view-space camera position but `normalDir` from world-space (or vice versa), the dot product is meaningful but rotates with the model's orientation rather than with the camera — which would produce a constant offset relative to the spec-conformant renders.

A second-order check: verify the asset's rim color is actually reaching the shader. If `parametricRimColorFactor = [1, 0.5, 0]` is being read as `[0, 0, 0]` somewhere in the import path, all rim tests would render identically (no rim at all) and the constant SSIM offset would just be the rim-vs-no-rim baseline. The asset generator emits the rim color correctly into the MToon extension (verified in `goldens-cache/_assets/mtoon_rimLightingMix_1.vrm`); the chain to investigate is the importer-side.

## Why this only surfaced now

Pre-encoding-fix, VMK wrote linear bytes for rim renders (mean R ≈ 117 across the rim band). UniVRM wrote sRGB-encoded bytes (mean R ≈ 180 in the same region). The two byte ranges happened to fall within SSIM's local-luminance tolerance band for the rim-color-zero case (the rim parameter's contribution at `mix=0` is exactly zero, the rim color contribution is also exactly zero, the test plan camera produces a centered sphere where the rim band is small relative to the body — the rim divergence was *invisible* until the body brightness was lifted into the sRGB range and the rim band's position-error became the dominant signal).

With encoding fixed, VMK's rim band is now properly bright at byte ~180 like UniVRM's — but in a *different position* along the silhouette, producing the SSIM hit.

## Conformance impact

Closing this issue lifts 6 tests in the conformance corpus from the residual cluster. Current state (VMK ↔ UniVRM at declared per-test thresholds, post-sRGB-fix):

```
VRMMetalKit ≥ declared threshold vs UniVRM: 66/76 (87%)
```

Projected after this issue + the 4 residual shadingToony tests in [VMK#213](https://github.com/arkavo-org/VRMMetalKit/issues/213) close: **76/76 (100%)**.

## Recommendation on timing

#214 (pipeline pixel-format + sample-count cache key) is the upstream-side companion to our adapter sRGB fix — necessary so that `config.colorPixelFormat` writes get respected when the PSO cache is queried. This rim-fresnel issue is independent (different shader code path, different bug class). **Recommend taking these in parallel** — they don't overlap, the launch timeline is anchored on VMK 1.0 conformance, and `#214` lands the encoding-correctness story while this issue lands the last MToon-math story. Together they should close out the corpus residual.

If shipped as 0.13.6 with both fixes, projected VMK conformance against the consortium reference is the full 76/76 the corpus admits (4 outline tests permanently excluded for methodology reasons per [vrm-conformance#3](https://github.com/arkavo-org/vrm-conformance/issues/3)).

Contributor guide

Open the contributing guide

Research direction

Start with the rim-term computation in MToonShader.metal and verify that viewDir and normalDir use matching world-space coordinates, comparing the convention with the cited UniVRM and three-vrm shaders. Then check the importer-side path for parametricRimColorFactor using goldens-cache/_assets/mtoon_rimLightingMix_1.vrm. Done means the six rimLightingMix tests show the expected silhouette response and the conformance gap is resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
computer-graphics
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.