arkavo-org / arkavo-org/VRMMetalKit

GLTFMetalKit: GPU compute morph kernel for high-vertex-count assets

Open
#245 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
6
Forks
2
Avg merge
18h 51m
Merged PRs (30d)
26

Description

## Context

GLTFMetalKit currently applies morph targets via a CPU pre-pass (`GLTFPrimitiveMorphData.morphedVertices(weights:)` in `Sources/GLTFMetalKit/Renderer/GLTFRenderable.swift`), rebuilding a fresh interleaved vertex buffer each frame and uploading via `device.makeBuffer(bytes:length:)`.

This is fine for the canonical Khronos test asset (`AnimatedMorphCube`, 24 verts, 2 targets). It will NOT scale to blendshape-driven faces (typical VRoid avatar: ~30K verts × 30 expression blendshapes), where the CPU loop and the per-frame upload become the per-frame bottleneck.

The pattern already exists in `Sources/VRMMetalKit/Shaders/MorphAccumulate.metal` (SoA layout, `morph_accumulate_positions` + `morph_accumulate_normals` + `morph_accumulate_combined` kernels). VRMMetalKit's expression system uses these. The kernels are spec-correct glTF morph math — they're VRM-tagged by location, not by content.

Surfaced in PR #241. Deferred under the rule "extract on demonstrated need, not speculation" — no GLTFMetalKit consumer has hit the limit yet.

## What's needed

- Move `MorphAccumulate.metal` (or fork) into `Sources/GLTFCore/Shaders/`.
- Add a `GLTFMorphCompute` Swift driver that mirrors VRMMetalKit's dispatch path: SoA delta buffers, active-set filtering on non-zero weights, dispatch position + normal kernels per frame.
- `GLTFAsset.drawCalls(animationIndex:time:)` picks the GPU path when `morphData.targetCount > N` (threshold tunable, maybe 4 or by total vertex count).
- Keep the CPU pre-pass for small assets — sub-millisecond CPU work is faster than a compute dispatch's overhead.

## Acceptance

- `AnimatedMorphCube` continues to render identical to the current CPU path.
- A synthetic stress test (e.g. 30K verts × 16 targets at varying weights) shows the GPU path produces correct output and stays within budget for 60 fps real-time playback.
- VRMMetalKit's existing `MorphAccumulate.metal` usage is unaffected (no breakage in the 1441 existing tests).

## Estimate

Modest. The kernels are written. The work is in the Swift driver + threshold heuristic + a stress test. ~1 day.

Contributor guide

Open the contributing guide

Research direction

Start with GLTFPrimitiveMorphData.morphedVertices in Sources/GLTFMetalKit/Renderer/GLTFRenderable.swift and compare its path with Sources/VRMMetalKit/Shaders/MorphAccumulate.metal. Trace GLTFAsset.drawCalls(animationIndex:time:) and the existing VRMMetalKit dispatch usage before implementing the driver and path selection. Done means AnimatedMorphCube remains identical, the 30K-vertex stress test produces correct output within the 60 fps budget, and existing tests remain unaffected.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
computer-graphics, performance
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.