arkavo-org / arkavo-org/VRMMetalKit

MASK-aware depth prepass for VRM alpha-cutout materials

Open
#360 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Swift
Stars
6
Forks
2
Avg merge
18h 51m
Merged PRs (30d)
26

Description

## Finding
The existing depth prepass (`enableDepthPrepass`) is a **no-op on typical VRM avatars** because it only processes `effectiveAlphaMode == "opaque" && !isFaceMaterial`. AvatarSample_A has **0 opaque materials** — it's 100% MASK (face, skin, body, cloth, hair). The prepass issued zero draws in QA's 4K GPU A/B.

## Why standard depth prepass doesn't work for VRM
MToon avatars use MASK alpha-cutout extensively for hair strands, eyelashes, and cloth edges. Opaque geometry is rare-to-absent. A standard position-only depth prepass can't handle alpha-tested geometry because it doesn't know which fragments survive the `discard_fragment()` in the MASK shader.

## Approach
A MASK-aware prepass would need:
1. Run the fragment shader with `discard` (not the cheap position-only pipeline) — costs more up front
2. Write depth only for surviving fragments
3. Main pass uses `depthTest: .equal` to skip fully-occluded fragments

The up-front cost is significant (running the shader twice on visible fragments), so this only pays off when overdraw is high (crowd scenes, dense hair).

## Status: BLOCKED

This issue is blocked behind:
- **#362** (ICB) — must remove the CPU encode ceiling first
- **#199** (crowd benchmark) — must confirm MASK overdraw is the GPU bottleneck

The crowd is **CPU-encode-bound** (26ms encode vs 5.3ms GPU at 60 avatars). A GPU-side optimization like #360 cannot raise the crowd ceiling while CPU encode is the wall. Building it now would measure neutral, exactly as the opaque prepass did.

**Recommended order**: #362 ICB → re-measure crowd → #199 crowd benchmark → only if overdraw confirmed → #360 MASK prepass

In parallel, #359 (ASTC) is the lower-risk GPU win — attacks texture-read limiter directly, helps at all resolutions and distant/LOD avatars, and doesn't depend on ICB.

## Related
- #195 (original depth prepass — position-only, opaque-only)
- #199 (crowd rendering milestone)
- #362 (ICB — prerequisite for this to be measurable)
- QA GPU trace analysis confirmed prepass is a 0-draw no-op on VRM

Contributor guide

Open the contributing guide

Research direction

Start by reading the existing enableDepthPrepass path and the MASK shader's discard_fragment behavior. Check blockers #362 and #199 before changing it, since the issue says the optimization is only worthwhile if MASK overdraw is confirmed after CPU encoding is addressed. Done means surviving MASK fragments write depth and the main pass can use depthTest: .equal with a measured benefit.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
computer-graphics, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.