Facilitate custom layer usage by separately passing projection+view matrices
- Dominant language
- TypeScript
- Stars
- 12.4k
- Forks
- 2.4k
- PR merge metrics
- No merged PRs in 30d
Description
## Motivation
I have tried a few times to use external libraries with custom layers (regl, three.js), and nearly every time have encountered difficulty using the combined projection-view matrix.
Some examples:
- Three.js fog uses `-mvPosition.z` (model * view * position).z as the input to fog. If `view` is the identity since it's instead already multiplied into projection, then fog is fixed to the global axes and doesn't move with the camera. See: [three.js/src/renderers/shaders/ShaderChunk/fog_vertex.glsl.js](https://github.com/mrdoob/three.js/blob/d4aa9e00ea29808534a3e082f602c544e5f2419c/src/renderers/shaders/ShaderChunk/fog_vertex.glsl.js#L4). I think you could instead apply the custom layer matrix to the entire scene graph and then use a trivial camera whose matrix is the identity, but this seems exceptionally convoluted and would probably make everything downstream awful to work with.
- Third party [THREE.MeshLine](https://github.com/spite/THREE.MeshLine) also [uses modelView and projection separately](https://github.com/spite/THREE.MeshLine/blob/168f9b28a9cf1f123df874d2d253872cadfa209c/src/THREE.MeshLine.js#L443) from modelViewProjection. I've not yet succeeded in successfully using the plugin to render 3D lines. https://observablehq.com/d/c598a929f986d9e1
- @astojilj solved ray-picking in https://github.com/mapbox/mapbox-gl-js/issues/10595#issuecomment-823906783. It works great, but I suspect the extra matrix gymnastics could be avoided by configuring the camera according to THREE.js's preferred conventions.
- I also encountered this with THREE.js + GLTF lighting. The MeshStandardMaterial appeared to be lighting based on the world coordinate rather than the relative orientation with the camera, in a way that led to incorrect lighting.
I suspect but am not certain that this leads to more difficulty than success with custom layers. (I'd love to know if people have a sense of this.)
## Design Alternatives
These issues can often be worked around by changing your reference frame. I'm still not 100% certain that the combined matrix is not sufficient. But if I have it correct in my head that some of the above issues could be solved by applying a projective transform to the scene graph and the identity matrix to the camera, it suggests that things are getting convoluted enough that it would be much easier to simply pass two separate matrices.
## Design
The call signature of custom layer `render` is:
https://github.com/mapbox/mapbox-gl-js/blob/85ce4d4bc781d184b85b87f97d1a9cb2c9be22d3/src/render/draw_custom.js#L43
Without a breaking change, we could pass `projectionMatrix` and `viewMatrix` as third and fourth arguments.
### Concepts
Model/View/Projection is a fairly standard convention, so I expect people looking to do custom rendering would know how to interpret the result. Additionally, this would be conveyed through docs and a modification of the THREE.js example to work more closely within the confines of THREE.js's preferred camera model.
### Implementation
This could be accomplished by simply passing matrices `transform` is already computing to the custom layer.
Contributor guide
Assessment
This issue has not been assessed yet.