wcandillon / wcandillon/react-native-webgpu
Onscreen<->offscreen transition is broken
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 1.2k
- Forks
- 70
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 23
Description
I've been debugging onscreen/offscreen transitions on Android for the last couple of days, and I believe the current way they're handled is broken fundamentally.
The root reason: it's possible (and easy!) to crash Dawn by racing the JavaScript rendering with the UI thread. In my app, we have several wgpu contexts in a virtualized list and inside tabs, so we quite frequently exercise the onscreen/offscreen transitions, so our app crashes within seconds on Android. And to add even more fun, we often resize our views dynamically.
I was able to keep the app stable by spamming the rendering code with makeshift rendering barriers, but I'm thinking about a more permanent fix. My plan right now:
- Use "epoch counter" in
GPUTextureand other derived objects. They'll have a reference to an "epoch object" that is invalidated the moment a surface gets destroyed (when Android calls onSurfaceTextureDestroyed/surfaceDestroyed). Each method of theGPUTexturegets wrapped in an "epoch guard" that will immediately throw a JS exception once the surface goes away. - The onscreen/offscreen transition will also invalidate the texture, causing any concurrent JS rendering to fail.
- Surface (haha) the paint event to the React world, so that rn-wgpu can request JS to make sure the current rendering is up-to-date. This is a departure from the way wgpu is handled on the web, unfortunately, but I think it's unavoidable.
- Resize will be handled just as an offscreen/onscreen transition: change the epoch to force any rendering code to fail fast, then request a repaint.
I think this will work reasonably well, but I'm worried about getting bad frames displayed during transitions, resulting in subpar experience. Alternatively, we can always render to an offscreen canvas?
I'd love to discuss this to get some feedback/ideas, but I'm not sure this is the best forum for this.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by examining the Android lifecycle callbacks onSurfaceTextureDestroyed and surfaceDestroyed, then trace how GPUTexture handles rendering during onscreen/offscreen transitions and resize. Review the proposed epoch invalidation, paint event, and offscreen canvas alternatives. Done would require an agreed design and reliable handling without crashes or bad frames during transitions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, react-native, typescript
- Domain
- mobile
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100