Feature Request: World Lab API
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Feature Idea
https://www.worldlabs.ai/blog/announcing-the-world-api
Somr initial ideas dumping:
**1. "Image-to-World" Custom Nodes**
The most direct integration would be a set of nodes that wrap the World API endpoints.
The Workflow: An artist generates a high-quality 2D environment using a standard Stable Diffusion/Flux workflow in ComfyUI. The output image is then passed into a "WorldLabs: Generate World" node.
The Benefit: This allows creators to iterate on the aesthetic of a scene in 2D (lighting, style, mood) and then "instantiate" it into a navigable 3D space with a single click.
**2. Multi-View & Panorama Stitching**
The World API supports multi-view inputs and 360° panoramas.
The Workflow: ComfyUI is excellent at batch-processing. You could use a workflow to generate 4–6 consistent views of a room (using ControlNet to maintain layout) and pipe them into a "WorldLabs: Multi-Image World" node.
The Benefit: This would produce much higher-fidelity 3D reconstructions than a single image, as ComfyUI can precisely manage the spatial consistency of the input images before they hit the World API.
**3. "World-to-Video" Motion Pipelines**
One of the most exciting potential workflows is using the generated world as a foundation for video.
The Workflow: 1. Generate a 3D world via World API.
2. Use a "WorldLabs: Virtual Camera" node to define a path (spline) through that 3D world.
3. Export those frames back into ComfyUI for AnimateDiff or SVD (Stable Video Diffusion) to add "micro-movements" like flowing water or moving clouds.
The Benefit: It solves the "temporal consistency" problem in AI video by grounding the movement in a persistent 3D geometry.
**4. Interactive Pre-Visualization for Filmmakers**
Targeting the professional VFX and indie film community (a key demographic for Comfy.org):
The Idea: A "Live Link" node. As you adjust your text prompt or LoRA strengths in ComfyUI, a low-resolution preview of the 3D world is streamed back into a side panel.
Use Case: A director can "block" a scene by generating a 3D world from a sketch, then use the World API's exported Gaussian Splat or Mesh files directly in Unreal Engine or Blender via the ComfyUI-to-3D pipeline.
### Existing Solutions
_No response_
### Other
_No response_
Contributor guide
Research direction
The issue links the World API announcement and describes possible ComfyUI nodes for image-to-world, multi-view, video, and virtual-camera workflows, but names no repository files or tests. Start by reading the linked API announcement and reviewing ComfyUI's existing node and external-service integration entry points. Done criteria need an agreed first workflow, API/authentication details, and acceptance tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- blender, python, unreal-engine
- Domain
- ai, api, computer-graphics
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100