arkavo-org / arkavo-org/VRMMetalKit

visionOS sample host: turn the build-only CI action into a real render assertion

Open
#399 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Swift
Stars
6
Forks
2
Avg merge
18h 51m
Merged PRs (30d)
26

Description

Part of #87. Follows #389 (platform declaration) and #395 (xros/xrsimulator metallib slices + loader selection).

## Goal

Add a minimal native visionOS sample host to the repo, and use it to promote the Xcode Cloud `Build - visionOS` action from build-only to an actual render assertion.

## Why now

#395 closed the last known gap, but nothing in CI proves a VRM actually *renders* on visionOS — the action compiles the library and stops. As #87's progress update put it, step 5 "is what converts visionOS CI from a compile check into real coverage." The slices are in; the assertion is not.

I validated the path out-of-tree today and it works. A ~250-line SwiftUI app depending on nothing but `VRMMetalKit` 1.1.0-beta.2, run on the visionOS 26.5 Simulator:

```
✅ compiled for visionOS environment: simulator
✅ Metal device: Apple xrOS simulator GPU
✅ VRMRenderer init — shader library loaded, MToon pipelines built
✅ VRM loaded: 121 nodes, 7 meshes (AliciaSolid.vrm)
✅ model bound to renderer (springbone on)
✅ drawOffscreen committed, no GPU errors
✅ frame has content: 12.3% non-background pixels
```

The render is correct, not just non-crashing — MToon shading and textures are right in the readback.

## Scope

- A `VRMMetalKitVisionHost` sample target: load a VRM, set `viewMatrix`/`projectionMatrix`, drive `drawOffscreen` into an offscreen colour+depth pair, display the readback. Deliberately *not* a CompositorServices immersive host — this isolates "does the renderer work on this platform" from "is the stereo host correct", and the two should not fail as one signal.
- A coverage assertion. **This is the part that matters:** a green build that draws nothing is exactly the failure mode a compile check cannot see, and it is what the macOS-slice fallthrough looked like before #395. Asserting `> N%` non-clear pixels catches it; asserting "no crash" does not.
- Flip the Xcode Cloud action to run it.

## Notes from getting this working

Three things cost time and are worth encoding in the sample rather than rediscovering:

1. `waitUntilCompleted` is unavailable from async contexts, and `MTLCommandBuffer` is not `Sendable`. Submit-and-wait has to stay in a single synchronous, non-hopping scope.
2. A directory named `Resources` at the root of a visionOS app bundle makes CFBundle read a macOS-style layout — `simctl install` then fails with the very misleading `Missing bundle ID`. Bundle the `.vrm` as a flat resource.
3. `SUPPORTED_PLATFORMS = "xros xrsimulator"` on the *sample app* too, not just the framework. Without it the destination is satisfied by the `Designed for [iPad,iPhone]` variant, which emits an iOS binary and loads the **iOS** slice — so the very thing under test is skipped while CI goes green. The #389 trap applies to the host as much as the library; `vtool -show-build` reporting `VISIONOSSIMULATOR` and products landing in `Debug-xrsimulator` are the checks that confirm it.

## Acceptance criteria

- [ ] Sample host target in-repo, building for `xros` and `xrsimulator`
- [ ] Renders a bundled VRM via `drawOffscreen` and asserts non-trivial pixel coverage
- [ ] Asserts the built product is genuinely xros (`BUILT_PRODUCTS_DIR` ends in `-xrsimulator`), per #389
- [ ] Xcode Cloud `Build - visionOS` action runs it instead of only compiling
- [ ] #390 can drop "build-verified" for "supported"

## Not in scope

Stereo/CompositorServices per-eye rendering and foveation — separate issue. Device-slice (`xros`) validation needs Vision Pro hardware; the simulator run only exercises `xrsimulator`.

I have the harness working and can open the PR.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

https://claude.ai/code/session_01T96n9hxVYXnGvdY3TCHgL4

Contributor guide

Open the contributing guide

Research direction

Start with the Xcode Cloud `Build - visionOS` action and the proposed `VRMMetalKitVisionHost` sample target; trace `drawOffscreen` and the bundled `AliciaSolid.vrm` path. Verify the product with `vtool -show-build` and `BUILT_PRODUCTS_DIR`, then run the visionOS Simulator flow. Done means the host builds for `xros` and `xrsimulator`, renders the VRM with non-trivial pixel coverage, and the action runs it.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
ci-cd, computer-graphics
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.