Field validation: test iPhone LiDAR capabilities and RuView NLOS client

Open the interactive RuView NLOS explainer

Goal

Create a repeatable field test for the RuView native iOS client and the LiDAR capabilities that Apple exposes publicly.

This issue must answer four separate questions:

1. Does the RuView application install and run correctly on a physical iPhone?
2. Which ARKit depth, mesh, and pose capabilities are available on that device?
3. Can the iPhone receive and safely display RuView NLOS track hypotheses?
4. Which measurements remain unavailable without an external transient time of flight sensor?

Important evidence boundary

The current native application is a secure viewer and transport adapter. It performs static ARKit capability checks, but it does not start an ARSession and does not capture live depth frames.

Apple scene depth is a derived visible surface product. It is not a raw photon timing histogram. A successful iPhone LiDAR test must never be described as an around the corner reconstruction or as a reproduction of the MIT consumer NLOS result.

Expected evidence classes:

Test	Valid conclusion	Invalid conclusion
Safari Three.js model	WebGL and touch interface work	Physical LiDAR works
Native app installation	SwiftUI client runs on the iPhone	NLOS sensing works
ARKit depth diagnostics	Visible surface depth, pose, and mesh work	Raw transient data is available
Synthetic track stream	RuView transport and visualization work	A hidden object was measured
External VL53L8CH capture	Raw transient bytes were received	Full physical reproduction passed
Witnessed physical protocol	Hardware gate passed under the documented criteria	General production readiness

Relevant implementation

1. Pull request: https://github.com/ruvnet/RuView/pull/1687
2. Native application: ui/ios-nlos/RuViewNLOS.xcodeproj
3. Native documentation: ui/ios-nlos/README.md
4. Rust sensing service: v2/crates/ruview-nlos
5. Physical protocol: docs/research/consumer-nlos-acceptance-protocol.md
6. Interactive explainer: https://ruview-nlos.ruv.chatgpt.site
7. Apple device testing guidance: https://developer.apple.com/documentation/xcode/running-your-app-on-simulated-or-physical-devices
8. Apple personal development team limits: https://developer.apple.com/help/account/basics/about-your-developer-account/

Required equipment

For application and ARKit testing:

1. A Mac capable of running a current Xcode release.
2. A physical iPhone running iOS 16 or newer.
3. A USB cable for initial pairing.
4. An Apple Account added to Xcode.
5. A local Rust toolchain for the synthetic RuView server.
6. Internet access if using a temporary Cloudflare Tunnel.

For physical transient testing:

1. ST P NUCLEO 53L8A1 development board.
2. VL53L8CH histogram capable sensor.
3. USB connection to the Mac or a Raspberry Pi.
4. Stable relay wall or floor surface.
5. Measured sensor pose or a synchronized pose capture process.
6. A hidden target with a known reference position.

Part 1: record the test environment

Before changing anything, record:

Tester:
Date and timezone:
iPhone model:
iOS version:
Mac model:
macOS version:
Xcode version:
RuView commit:
Apple development team type: Personal or paid
Network type:
External sensor available: yes or no

Do not include Apple credentials, bearer tokens, signing certificates, device identifiers, or tunnel tokens.

Part 2: test the Safari graphics

On the iPhone:

1. Open https://ruview-nlos.ruv.chatgpt.site in Safari.
2. Rotate the phone between portrait and landscape.
3. Drag the Three.js scene with one finger.
4. Select LiDAR only.
5. Confirm the RF shells disappear.
6. Confirm confidence changes to 74 percent.
7. Select Synthetic fusion.
8. Confirm the RF shells return.
9. Confirm confidence changes to 92 percent.
10. Confirm the uncertainty volume contracts.
11. Tap Pause, wait five seconds, and confirm animation stops.
12. Tap Resume and confirm animation restarts.
13. Background Safari for ten seconds, return, and confirm the scene recovers.

Expected result:

1. No page crash.
2. Touch orbit remains responsive.
3. Text is readable in both orientations.
4. The scene retains the G0 synthetic evidence label.
5. No claim suggests that the visualization is live sensor output.

Record any visual defect with a screenshot and device orientation.

Part 3: install the native application

On the Mac:

git clone https://github.com/ruvnet/RuView.git
cd RuView
git fetch origin pull/1687/head:ruview-nlos
git checkout ruview-nlos
open ui/ios-nlos/RuViewNLOS.xcodeproj

In Xcode:

1. Select the RuViewNLOS project.
2. Open Signing & Capabilities.
3. Select an Apple development team.
4. Keep automatic signing enabled.
5. Change the bundle identifier to a unique value such as com.example.ruviewnlos.dev.
6. Connect the iPhone by cable.
7. Tap Trust on the iPhone if prompted.
8. Select the RuViewNLOS scheme.
9. Select the physical iPhone as the run destination.
10. Choose Product, then Run.
11. Enable Developer Mode on the iPhone if requested.
12. Reboot and confirm Developer Mode if iOS requires it.
13. Run the application again.

Expected result:

1. The project builds without source changes.
2. The application launches on the physical device.
3. The app displays the Apple capability summary.
4. No camera permission is requested by the current static probe.
5. No target appears until an authenticated track stream is connected.

A Personal Team is sufficient for initial testing, but the installed profile normally expires after seven days and must be rebuilt.

Part 4: capture the current static capability result

Record the values reported by the application for:

1. AR world tracking support.
2. Scene depth support.
3. Smoothed scene depth support.
4. Scene reconstruction mesh support.
5. Raw photon timing histogram availability.

Expected result on a LiDAR equipped iPhone Pro:

1. World tracking should normally be available.
2. Scene depth should normally be available.
3. Smoothed scene depth should normally be available.
4. Mesh reconstruction should normally be available.
5. Raw photon timing histograms should be reported unavailable.

The first four results are device and operating system capabilities. The fifth result is the critical NLOS limitation.

Part 5: add a live Apple LiDAR diagnostics mode

This is required before claiming that RuView has tested live Apple LiDAR frames. The current app does not yet start an ARSession.

Implement a clearly labeled Apple LiDAR Diagnostics research screen with these constraints:

1. Require an explicit user action before starting ARKit.
2. Request camera permission only when the diagnostics session starts.
3. Use ARWorldTrackingConfiguration.
4. Enable .sceneDepth when supported.
5. Enable .smoothedSceneDepth when supported.
6. Enable scene reconstruction mesh when supported.
7. Never classify ARKit depth as NLOS evidence.
8. Never promote ARKit output beyond a visible surface diagnostic.
9. Do not retain RGB camera frames.
10. Keep depth and pose data in memory unless the tester explicitly exports a diagnostic report.
11. Export summaries without faces, images, tokens, device identifiers, or precise location.
12. Stop the ARSession when the app backgrounds or the diagnostics screen closes.

Display these live metrics:

1. Depth map width and height.
2. Received depth frames per second.
3. Valid depth pixel percentage.
4. Minimum, median, and maximum visible depth.
5. Confidence map distribution when available.
6. Camera transform update rate.
7. Estimated sensor translation during the sample.
8. Mesh anchor count and approximate vertex count.
9. Dropped frame count.
10. Thermal state.
11. Session interruption and relocalization count.
12. Explicit label: Visible surface depth only. Not raw transient NLOS data.

Part 6: run the live visible surface test

Use a simple room with a wall corner and a box.

1. Measure the visible box position manually.
2. Start Apple LiDAR Diagnostics.
3. Hold the iPhone 1 to 2 metres from the visible box.
4. Move the phone slowly through approximately 10 to 20 centimetres of translation.
5. Record ten seconds of diagnostic metrics.
6. Repeat with the box partly occluded.
7. Move the box fully behind the corner.
8. Repeat the same phone motion.
9. Stop diagnostics.
10. Export the redacted metrics report.

Expected positive control:

1. The visible box contributes valid scene depth.
2. Depth and pose update while the phone moves.
3. Mesh anchors describe visible surfaces.

Expected negative control:

1. The fully hidden box does not appear in the ARKit depth map.
2. The application does not invent a hidden target.
3. The report explicitly records that derived Apple depth cannot establish NLOS.

This negative control is important. It proves the software respects the API boundary instead of silently turning missing data into a detection.

Part 7: run the synthetic RuView stream

On the Mac:

cd RuView/v2
export RUVIEW_NLOS_TOKEN="$(openssl rand -hex 32)"
cargo run -p ruview-nlos -- serve --synthetic

In a second terminal:

npx wrangler tunnel quick-start http://127.0.0.1:8787

The tunnel prints a temporary HTTPS hostname. Convert it into the native WebSocket endpoint:

wss://YOUR_HOST.trycloudflare.com/api/v1/nlos/ws

In the iPhone app:

1. Enter the wss endpoint.
2. Enter the generated pairing token.
3. Connect.
4. Confirm a moving target appears.
5. Confirm the frame is permanently labeled SYNTHETIC.
6. Background the app.
7. Confirm the app disconnects and clears the target.
8. Return to the foreground and reconnect explicitly.
9. Stop the Rust server.
10. Confirm the target clears no later than its five second expiry.
11. Enter an incorrect token and confirm authentication fails.
12. Restore the correct token and reconnect.

Cloudflare Quick Tunnels are for development only. Do not reuse the temporary hostname for production.

Part 8: optional external transient sensor test

Do this only when the ST hardware is available.

1. Connect the P NUCLEO 53L8A1 and VL53L8CH sensor.
2. Confirm the device appears as a serial port.
3. Prepare a synchronized pose JSONL file.
4. Keep the first 60 frames as an empty room calibration sequence.
5. Run:

cd RuView/v2
cargo run -p ruview-nlos --features hardware -- capture-st \
  --port /dev/ttyACM0 \
  --session iphone-lidar-lab-001 \
  --frames 300 \
  --sensor-id st-kit-001 \
  --sensor-model VL53L8CH \
  --firmware-version 15314de \
  --pose-jsonl synchronized-poses.jsonl

6. Preserve the raw capture locally.
7. Record the calibration digest.
8. Verify that raw histograms never enter the public track envelope.
9. Run the preregistered physical acceptance protocol.
10. Do not mark the hardware gate complete unless the witnessed criteria pass.

A fixed sensor is useful for transport diagnostics but cannot satisfy the motion induced aperture gate.

Troubleshooting

The iPhone is not listed in Xcode

1. Use a cable for first pairing.
2. Unlock the iPhone.
3. Tap Trust This Computer.
4. Open Xcode Device Hub and select the phone.
5. Confirm the device operating system is supported by the installed Xcode version.
6. Install the required iOS platform support when Xcode offers it.
7. Enable Developer Mode after the pairing process begins.

Signing fails

1. Confirm an Apple Account is added under Xcode Settings.
2. Select the correct development team.
3. Keep automatic signing enabled.
4. Use a unique bundle identifier.
5. Remove stale local copies of the app from the phone and rebuild.
6. With a Personal Team, remember that profiles expire after seven days.

Developer Mode is missing

1. Connect and pair the device in Xcode first.
2. Restart the iPhone.
3. Open Privacy & Security settings.
4. Enable Developer Mode if the option appears.
5. Confirm the mode after reboot.

Scene depth is unsupported

1. Confirm the device is a LiDAR equipped iPhone Pro model.
2. Run on the physical device, not the Simulator.
3. Confirm AR world tracking is supported.
4. Confirm the application checks feature support before enabling frame semantics.
5. Record the device model and operating system version.

Camera permission is denied

1. Open iOS Settings.
2. Find RuView NLOS.
3. Enable Camera access.
4. Restart the diagnostics session.
5. If permission remains unavailable, delete and reinstall the development app.

No live depth frames arrive

1. Confirm an ARSession is actually running.
2. Confirm .sceneDepth was added only when supported.
3. Move to a textured, well lit indoor area.
4. Avoid pointing only at reflective glass, mirrors, direct sunlight, or featureless surfaces.
5. Check ARSession interruption and failure callbacks.
6. Record the exact ARKit error without including private imagery.

Frame rate is low

1. Remove the protective case temporarily if the phone is hot.
2. Stop screen recording.
3. Close other graphics intensive applications.
4. Reduce visualization complexity before reducing measurement integrity.
5. Record thermal state and dropped frames.
6. Repeat after the device cools for ten minutes.

The Three.js scene is blank

1. Reload Safari.
2. Disable content blockers for the site.
3. Close other graphics heavy browser tabs.
4. Restart Safari.
5. Confirm WebGL works on another Three.js page.
6. Record the iOS version and whether the fallback message appeared.

The native app cannot connect

1. Confirm the endpoint starts with wss://.
2. Confirm the path is /api/v1/nlos/ws.
3. Confirm the bearer token is at least 32 visible characters.
4. Confirm the Rust server and tunnel are both running.
5. Open the tunnel HTTPS health endpoint at /health in Safari.
6. Regenerate the Quick Tunnel because its hostname changes after restart.
7. Confirm no token was placed in the URL.
8. Do not bypass TLS certificate validation or use a self signed certificate.

A target appears but immediately disappears

1. Check the frame expiry timestamp.
2. Confirm the iPhone clock is correct.
3. Confirm sequence numbers increase.
4. Confirm the stream does not switch session identifiers.
5. Confirm no frame is more than one second ahead of the phone clock.
6. Confirm the server is publishing continuously.
7. Preserve the rejection reason but remove tokens before sharing logs.

Privacy and security rules

1. Do not upload camera frames, raw room scans, faces, home interiors, tokens, signing keys, device identifiers, or precise location.
2. Redact tunnel hostnames after the test if they remain active.
3. Keep raw transient captures local until the dataset governance process is approved.
4. Treat all visualizations as advisory research output.
5. Do not connect target tracks to physical actuation, alarms, medical decisions, or safety decisions.
6. Report security defects privately rather than attaching exploit details to a public comment.

Acceptance criteria

Software and device gate:

1. Native project builds on a Mac without source changes.
2. App installs and launches on a physical LiDAR equipped iPhone.
3. Static capability results are recorded.
4. Native synthetic stream authenticates and displays at approximately 30 updates per second.
5. Synthetic evidence remains visibly watermarked.
6. Incorrect authentication fails closed.
7. Backgrounding clears the current track.
8. Stream loss clears the current track within five seconds.
9. Safari Three.js interaction works in portrait and landscape.

Apple LiDAR diagnostics gate:

1. Live ARKit session starts only after explicit consent.
2. Visible scene depth, pose, and mesh metrics are recorded.
3. No RGB imagery is retained.
4. Fully occluded objects are not presented as ARKit detections.
5. Exported metrics remain clearly labeled as visible surface diagnostics.
6. Raw transient availability remains unavailable.

Physical NLOS gate:

1. External histogram capable hardware is used.
2. Calibration and pose lineage are recorded.
3. Sustained tracking reaches at least 27 frames per second.
4. Measured CSI fusion reduces position error or lost track rate by at least 25 percent.
5. The physical protocol is independently witnessed.
6. Hardware evidence is never inferred from the synthetic benchmark.

Feedback requested

Please add one comment per device using this template:

Device model:
iOS version:
Xcode version:
RuView commit:
Installation result: pass or fail
World tracking:
Scene depth:
Smoothed scene depth:
Mesh reconstruction:
Raw transient histograms: expected unavailable
Safari Three.js result:
Native synthetic stream result:
Average observed update rate:
Stale clearing result:
Thermal state:
External sensor used:
Unexpected behaviour:
Troubleshooting attempted:
Suggested improvement:

Attach only redacted screenshots and logs. Include exact reproduction steps for failures. Do not include pairing tokens or private room imagery.

Requested contributor feedback

We specifically need feedback on:

1. Which LiDAR equipped iPhone models report different capability combinations.
2. Whether depth frame rate changes materially across iOS releases.
3. Safari Three.js performance and memory pressure on older devices.
4. Signing and Developer Mode friction for first time testers.
5. Accessibility and readability of the native visualization.
6. Whether stale state and authentication failures are understandable.
7. Whether the evidence boundary is clear to a non researcher.
8. Which external transient sensor setup is easiest to reproduce.
9. Any result that appears to overstate what Apple LiDAR actually measured.

Estimated effort

1. Safari graphics test: 5 minutes.
2. Native installation with Xcode already configured: 15 to 30 minutes.
3. Synthetic end to end stream test: 15 to 30 minutes.
4. Live ARKit diagnostics implementation: 1 to 2 engineering days.
5. Controlled external sensor capture: 1 day after hardware and firmware are ready.
6. Full physical and CSI witness protocol: 2 to 5 days depending on synchronization and environment control.

Review PR #1687 and the native project at ui/ios-nlos/RuViewNLOS.xcodeproj, then read ui/ios-nlos/README.md and v2/crates/ruview-nlos. Run the existing static capability and synthetic-stream checks before assessing the live diagnostics work. Done means the documented iPhone, ARKit, synthetic-stream, and optional sensor tests produce redacted evidence without presenting derived depth as NLOS data.