openai / openai/codex

Feature request: Bridge ChatGPT visual/image capabilities with Codex for game-asset workflows

Open
#45,007 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement imagen
Dominant language
Rust
Stars
125k
Forks
19.4k
PR merge metrics
PR metrics pending

Description

Problem

For game-development workflows, ChatGPT is often much better at understanding visual references and generating images, while Codex is much better at editing the actual project and writing code. Today these feel too disconnected.

A concrete example is a Unity game project that needs a consistent character/sprite pipeline. The user can show ChatGPT a visual target and discuss art direction, sprite layering, 8-direction animation, SPR/ACT-style data, etc. ChatGPT can reason about the image and generate/iterate on visuals, but Codex working inside the project does not automatically inherit that visual understanding or the generated design decisions.

This creates a frustrating gap:

  • ChatGPT can analyze the reference image well, but cannot directly continue the implementation inside the Codex project workspace.
  • Codex can modify the Unity project, but its visual/image generation and visual analysis workflow is weaker or disconnected from what ChatGPT already understood.
  • The user has to manually copy prompts, screenshots, conclusions, and requirements between ChatGPT and Codex.
  • Context and design intent can be lost between the two, which leads to rework.

Requested improvement

Please make Codex either stronger at image generation/visual analysis, or provide a first-class bridge between ChatGPT and Codex.

Ideal behavior:

  1. ChatGPT and Codex can share the same task/project context when the user explicitly links them.
  2. Codex can ask ChatGPT's image/vision system to analyze a reference image or generate an asset, then receive the result back in a structured way.
  3. ChatGPT can hand an implementation task to Codex with the full design context, image references, constraints, and accepted decisions preserved.
  4. Codex can return code changes, screenshots/renders, test results, and project state back to ChatGPT for review.
  5. Both sides should understand which local project/folder/repository the task belongs to so users with multiple projects/accounts do not accidentally modify the wrong workspace.
  6. Generated assets should be attachable to the active project workflow without manual downloading/re-uploading between products.

Why this matters

For game development, image/design and implementation are tightly coupled. A typical loop is:

visual reference -> art/design discussion -> generated concept -> implementation -> in-engine preview -> visual review -> code/art revision

ChatGPT is already strong at the visual/design side, while Codex is strong at project/code execution. A reliable bridge between them would be substantially more useful than treating them as isolated assistants.

Suggested workflow

A user should be able to say in ChatGPT:

Use this visual target for the MYTHWEAVE character system. Hand the implementation to Codex in my Unity project, then bring the result back here for review.

Then Codex could work on the project, generate an in-engine preview or screenshot, and return that result to ChatGPT, where the user can continue the visual review without manually rebuilding the context.

This would be especially valuable for Unity/Unreal/Blender/game-asset pipelines, UI implementation, shaders, sprite systems, and other workflows where code and visual output must be iterated together.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or implementation entry points are identified. Start by reviewing the Codex project architecture and the requested ChatGPT handoff workflow, including shared context, visual inputs and outputs, project selection, and asset transfer. Done would require a defined, reliable bridge that preserves design context and returns implementation results for review.

Written by the indexing model from the issue text.

Assessment

Tech stack
blender, unity, unreal-engine
Domain
ai, game-dev
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.