arkavo-org / arkavo-org/VRMMetalKit
MToon rim lighting: high-mix rim-extent residual — VMK's rim doesn't wrap the unlit silhouette at rimLightingMix→1 (SSIM ~0.949 vs 0.95, only at max rim)
- Dominant language
- Swift
- Stars
- 6
- Forks
- 2
- Avg merge
- 18h 51m
- Merged PRs (30d)
- 26
Description
## Summary
New finding revealed by the vrm-conformance sRGB-encoding fix ([vrm-conformance@fd8221d](https://github.com/arkavo-org/vrm-conformance/commit/fd8221d) — closes [VMK#213](https://github.com/arkavo-org/VRMMetalKit/issues/213)'s root cause for shadingShift). VRMMetalKit's rim-fresnel rendering diverges from the three other MToon implementations on the conformance corpus's six `mtoon_rimLightingMix_*` tests. The divergence was masked pre-encoding-fix by a coincidence of linear-vs-sRGB byte alignment in the rim brightness range; with proper sRGB encoding now in place, the rim cluster is the largest remaining VMK ↔ UniVRM gap.
cc @arkavo-com (per offer to take this in parallel with #214)
## Versions
- VRMMetalKit: 0.13.5 + adapter sRGB fix landed on conformance `fd8221d`
- Reference renderers: UniVRM v0.131.0 (Unity 6 PlayMode), three-vrm 3.5.0, godot-vrm 4.6.2
## Observed: VMK is the only outlier; sweep-invariant divergence
Full pairwise SSIM matrix on `mtoon_rimLightingMix_0` (identical numbers on every test in the sweep — see below):
| Pair | SSIM |
|---|---|
| three-vrm ↔ UniVRM | **0.9902** |
| godot-vrm ↔ UniVRM | 0.9793 |
| three-vrm ↔ godot-vrm | 0.9902 |
| VMK ↔ three-vrm | 0.9144 |
| VMK ↔ godot-vrm | 0.9196 |
| **VMK ↔ UniVRM** | **0.9078** |
Three of the four pairs cluster at 0.98+. VMK's three pairs all sit in the 0.90–0.92 band. **VMK is the outlier**, separated from the consortium reference by ~0.08 SSIM that the other two non-reference implementations don't carry.
## The sweep-invariance is the strongest diagnostic signal
The corpus emits six rim tests with `parametricRimColorFactor = [1.0, 0.5, 0.0]` (orange), `parametricRimFresnelPowerFactor = 5.0`, and `rimLightingMixFactor` swept across `{0.0, 0.1, 0.25, 0.5, 0.75, 1.0}`. The expected behavior is that as `rimLightingMixFactor` increases, the orange rim becomes increasingly visible at the sphere's silhouette, with `0.0` showing no rim influence at all and `1.0` showing maximum rim.
Empirically, **every renderer pair produces the exact same SSIM across the entire `rimLightingMixFactor` sweep**:
```
mtoon_rimLightingMix_0 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p1 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p25 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p5 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_0p75 VMK↔UniVRM = 0.9078
mtoon_rimLightingMix_1 VMK↔UniVRM = 0.9078
```
Identical across the sweep, identical across pairs (each pair has its own constant). That's not noise — it's a **structural constant offset**. Either:
- (a) The parameter sweep isn't producing visibly different output in any renderer (the orange rim isn't visible at any mix factor), and the constant 0.9078 is the underlying material-baseline disagreement between VMK and UniVRM on the rim-color-zero render that all six tests collapse to. *or*
- (b) Every renderer correctly modulates rim contribution with `rimLightingMixFactor`, but VMK's rim contribution carries a fixed-magnitude error (e.g., a coordinate-space mismatch in the fresnel term) that produces the same SSIM gap regardless of the mix factor. The mix factor multiplies the whole rim term, which would produce a *varying* SSIM under most error models — but a coordinate-space bug that affects only the *direction* of the fresnel falloff (not its magnitude) would produce a constant offset because the visible rim band has the same total energy, just in slightly the wrong place.
(b) is the more interesting hypothesis. A fresnel coordinate-space bug — for instance, view-space vs world-space normal in the `dot(N, V)` term, or using `objectFromCamera` where the spec expects `cameraToObject` — would shift the rim band's *position* along the silhouette without changing its *intensity*. SSIM penalizes the position shift, the mix factor doesn't change the shift, the SSIM is constant across the sweep. That fits the data.
## Hypothesis to investigate first
The MToon-1.0 spec ([§3.7 Rim Lighting](https://github.com/vrm-c/vrm-specification/blob/master/specification/VRMC_materials_mtoon-1.0/README.md#rim-lighting)) defines the fresnel rim factor as `pow(1.0 - max(0.0, dot(viewDir, normalDir)), rimFresnelPower)` — where `viewDir` is from the surface point to the camera and `normalDir` is the shading normal, **both expressed in world space** (consistent with the rest of the MToon shader's vector conventions).
Suggested check: `MToonShader.metal` rim-term computation — verify `viewDir` and `normalDir` are in matching coordinate spaces. UniVRM's `Vrm10MToonShader.hlsl` Built-in RP path uses world-space normals; three-vrm's `mtoon_frag.glsl.js` uses world-space. If VMK's rim term reads `viewDir` from view-space camera position but `normalDir` from world-space (or vice versa), the dot product is meaningful but rotates with the model's orientation rather than with the camera — which would produce a constant offset relative to the spec-conformant renders.
A second-order check: verify the asset's rim color is actually reaching the shader. If `parametricRimColorFactor = [1, 0.5, 0]` is being read as `[0, 0, 0]` somewhere in the import path, all rim tests would render identically (no rim at all) and the constant SSIM offset would just be the rim-vs-no-rim baseline. The asset generator emits the rim color correctly into the MToon extension (verified in `goldens-cache/_assets/mtoon_rimLightingMix_1.vrm`); the chain to investigate is the importer-side.
## Why this only surfaced now
Pre-encoding-fix, VMK wrote linear bytes for rim renders (mean R ≈ 117 across the rim band). UniVRM wrote sRGB-encoded bytes (mean R ≈ 180 in the same region). The two byte ranges happened to fall within SSIM's local-luminance tolerance band for the rim-color-zero case (the rim parameter's contribution at `mix=0` is exactly zero, the rim color contribution is also exactly zero, the test plan camera produces a centered sphere where the rim band is small relative to the body — the rim divergence was *invisible* until the body brightness was lifted into the sRGB range and the rim band's position-error became the dominant signal).
With encoding fixed, VMK's rim band is now properly bright at byte ~180 like UniVRM's — but in a *different position* along the silhouette, producing the SSIM hit.
## Conformance impact
Closing this issue lifts 6 tests in the conformance corpus from the residual cluster. Current state (VMK ↔ UniVRM at declared per-test thresholds, post-sRGB-fix):
```
VRMMetalKit ≥ declared threshold vs UniVRM: 66/76 (87%)
```
Projected after this issue + the 4 residual shadingToony tests in [VMK#213](https://github.com/arkavo-org/VRMMetalKit/issues/213) close: **76/76 (100%)**.
## Recommendation on timing
#214 (pipeline pixel-format + sample-count cache key) is the upstream-side companion to our adapter sRGB fix — necessary so that `config.colorPixelFormat` writes get respected when the PSO cache is queried. This rim-fresnel issue is independent (different shader code path, different bug class). **Recommend taking these in parallel** — they don't overlap, the launch timeline is anchored on VMK 1.0 conformance, and `#214` lands the encoding-correctness story while this issue lands the last MToon-math story. Together they should close out the corpus residual.
If shipped as 0.13.6 with both fixes, projected VMK conformance against the consortium reference is the full 76/76 the corpus admits (4 outline tests permanently excluded for methodology reasons per [vrm-conformance#3](https://github.com/arkavo-org/vrm-conformance/issues/3)).
Contributor guide
Research direction
Start with the rim-term computation in MToonShader.metal and verify that viewDir and normalDir use matching world-space coordinates, comparing the convention with the cited UniVRM and three-vrm shaders. Then check the importer-side path for parametricRimColorFactor using goldens-cache/_assets/mtoon_rimLightingMix_1.vrm. Done means the six rimLightingMix tests show the expected silhouette response and the conformance gap is resolved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- computer-graphics
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100