invoke-ai / invoke-ai/InvokeAI
[enhancement]: Canvas 3D: Interactive Scene Editor for ControlNet Guidance
- Dominant language
- Python
- Stars
- 28.2k
- Forks
- 3k
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 19
Description
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Contact Details
_No response_
### What should this feature add?
# I’d like to propose a Canvas 3D module — a lightweight, interactive 3D scene editor to assist with ControlNet-based workflows.
### The core idea is to give users the ability to compose simple 3D scenes (using primitives or imported models), position a camera, and take snapshots with lighting and perspective cues that can serve as structured input for ControlNet.
The editor should support basic camera movement (orbit, free/fly, locked views, lens type editing to control prespective distortion, DOF etc), basic object manipulation, Humanoid Auto Rigging, Custom Rigging and exporting snapshots directly into the Invoke pipeline to be used to output for controlnets, raster images, etc into Canvas 2D (Current Canvas).
This would greatly enhance control over image generation—especially for structural, pose-driven, or perspective-aware generations. The idea behind this, is mostly for people who have prepared assets for image gen or placing basic assets in such a way to be able to get the best control results they can get, especially with the advent of 3D model generators slowly coming on to the scene. it can also be something that can one day be the foundation of invoke's own 3D model generators when the tech gets good opensource wise.
### Alternatives
While users can import reference images manually or use ControlNet with sketch/canny/etc., there’s no integrated tool in InvokeAI that allows for precise 3D spatial planning.
Third-party tools (e.g. Blender, Online Openpose Editor) are overly complex for this task and require significant setup time. A built-in editor would streamline this for creators similar to the creation of the current canvas which in this addition would become "Canvas 2D".
### Additional Content

I am currently away, so i generated with 4o on this one lol
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.