armory3d / armory3d/armory

Ability of rendering from multiple cameras at once and composit/layer it into a single final image - A simple solution to 3D UI

Open
#2,257 4 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
3.3k
Forks
353
Avg merge
3d 16h
Merged PRs (30d)
1

Description

An example of how this kind of solution is used in another game engine:
https://www.unrealengine.com/marketplace/en-US/product/lgui-lex-gui-3d-ui-system-for-ue4?sessionInvalidated=true

The idea is that you have 2 cameras, not necessarily in the same scene rendering at the same time to composite into the final image that will be renderer on the screen. This way you could have a fixed FOV for the camera shooting the scene holding the UI elements and an adjustable FOV for the camera shooting the game level.

If one could create something similar to UE4s node based post processing/compositing: https://www.youtube.com/watch?v=71FjgIJ4vck by adding a "camera input" node, a "camera UV input" and allowing materials to act as a layer between that the camera renders the scene & the finished frames being outputted to the screen the possible use cases for a feature like this could also expand from just overlaying something simple like a UI to:
* Split screen solutions (of course there would be a lot of overdraw caused here unless one come up with a solution to avoid just that) where the game scripter or artist creates a material used to composite multiple cameras so that each camera overs a part of the screen. For instance 2 camers could be combined with a "mixRGB" node where half the screen gets a factor of 0 and the other half a factor of 1. One potential solution that I could see to avoid overdraw in this case would be if the "camera input" node had a UV input (and output for all the channels that are normally rendered as a pass in the deffered renderer), so that you could perform vector math on where the image would be outputted to the composited "material" or image that later gets displayed on the screen. I suppose one would also need to have separate control over the output screen resolution and camera resolution *separately* here, as otherwise if you have say 4 cameras each rendering 1080p and having that get mixed into a single 1080p final image, 75% of the pixels rnedered by the camera would be completely wasted (or act as some really expensive anti aliasing heh).
* Well, post process effects with materials! Say you look at your finished level and you decide that you went too agressive with your roughness on your materials, but you dont wanna go and adjust each and every material. A solution here would be to set a specific post process material for that level and do some math in the shader graph to bring down that roughness to somewhere reasonable.

*To summerize:*
* A simple list of cameras to render and a way of setting the order that they should be layered would be sufficient to solve the 3D UI usecase. You would also need a method for the artist/game scripter to set the "alpha" for each camera (for instance a new rendering pass for alpha as well as a new "alpha output" node in the material editor to assign to surfaces of any mesh or the world background).
* For more advanced and customizable usecases like post process effects or highly customizable splitscreen solutions. Some sort of compositing node graph would be cool, preferably reusing the preexisting material node graph

Contributor guide

No contributing guide indexed for this repository

Research direction

No files, tests, or entry points are named. Start by locating the camera-rendering and material-node systems, then determine whether the requested camera ordering, alpha control, and compositing graph can be scoped; done would require a documented implementation plan or a complete compositing feature.

Written by the indexing model from the issue text.

Assessment

Domain
computer-graphics, game-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.