arkavo-org / arkavo-org/VRMMetalKit

Architecture: Crowd Rendering Scalability (10-100+ Avatars)

Open
#91 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Swift
Stars
6
Forks
2
Avg merge
18h 51m
Merged PRs (30d)
26

Description

We have a use case to utilize `VRMMetalKit` as the character rendering engine for a Swift/Metal game. The requirement is to render **10 to 100 concurrent VRM models** (PCs and NPCs) with stable framerates (60fps+ on Apple Silicon).

While the current architecture handles single-model high-fidelity rendering (Viewer Mode) efficiently, scaling to 100 models introduces specific CPU and GPU bottlenecks that require a Data-Oriented approach.

### The Bottlenecks
1. **Draw Call Overhead:** 100 Models $\times$ ~10 Primitives (Face, Hair segments, Clothes) = ~1,000 Draw Calls per frame. This will bottleneck the CPU.
2. **Compute Dispatch Overhead:** Dispatching the Physics Compute Kernel separately for every avatar ($O(N)$) incurs unnecessary driver overhead.
3. **Overdraw:** Overlapping transparent hair layers from 100 avatars will severely impact GPU fill rate.

### Proposed Roadmap & Tasks

#### 1. Global Physics Solver (Batching)
Instead of owning a private `SpringBoneSystem` per avatar, we need a global manager that aggregates all scene bones into single, monolithic buffers.
- [ ] **Feature:** Create `GlobalSpringBoneManager`.
- [ ] **Implementation:** Allocate a singular large `MTLBuffer` for all active SpringBones in the scene.
- [ ] **Compute:** Refactor the compute dispatch to execute **once per frame** for the entire crowd, rather than once per model.

#### 2. Indirect Command Buffers (ICB)
To solve the draw call bottleneck, we must move draw command encoding to the GPU.
- [ ] **Feature:** Implement `MTLIndirectCommandBuffer` support in `VRMRenderer`.
- [ ] **Logic:** Pre-encode draw commands. Update only the instance uniforms (Matrices) per frame, allowing the GPU to draw the whole crowd with minimal CPU intervention.
- [ ] **Culling:** (Optional) Use GPU-driven occlusion culling within the ICB pipeline.

#### 3. Distance-Based LOD System
We cannot afford full physics and morphing for NPCs 50 meters away.
- [ ] **Feature:** Implement a `LevelOfDetail` controller.
- [ ] **LOD 0 (Hero / < 5m):** Full Physics, Full MToon, Morph Targets enabled.
- [ ] **LOD 1 (Nearby / < 20m):** Full MToon, **Physics Disabled** (Reset to Rest Pose), Morphs Disabled.
- [ ] **LOD 2 (Crowd / > 20m):** Switch to simple `Unlit` or `Lambert` shader.
- [ ] **LOD 3 (Far):** Frustum/Distance Culling (Don't render).

#### 4. Fill Rate Optimization
- [ ] **Optimization:** Implement a **Depth Pre-Pass** (Z-Prepass). Render the scene to the Depth Buffer first with a cheap shader. This ensures the expensive MToon shader only executes for visible pixels, mitigating the impact of hair overdraw in crowds.

### Success Metrics / Benchmark
- **Target Hardware:** M1 / M2 Basic.
- **Scenario:** 100 "AliciaSolid" models in view.
- **Goal:** Stable 60 FPS.
- **Profile:** CPU usage should remain < 10ms per frame.

---
*Reference discussion regarding Game Scalability vs Viewer Architecture.*

Contributor guide

Open the contributing guide

Research direction

Start by reading the current Viewer Mode architecture, SpringBoneSystem, and VRMRenderer entry points to understand how per-avatar physics and draw work are organized. Use the proposed batching, ICB, LOD, and depth-pre-pass tasks as the scope, then benchmark 100 AliciaSolid models on M1/M2 hardware and verify stable 60 FPS with CPU usage below 10 ms per frame.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
computer-graphics, game-dev, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.