dotCMS / dotCMS/core

fix(dotAI): Support new OpenAI image models (gpt-image-1)

Open
#34,762 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dotCMS : AI Multiple Vendors stale
Dominant language
Java
Stars
970
Forks
486
Avg merge
3d 33m
Merged PRs (30d)
170

Description

Description

Following the dotAI refactor that introduced support for multiple AI vendors/models (#34058), the image generation feature was broken when using gpt-image-1 (and related models: gpt-image-1-mini, gpt-image-1.5) as the configured image model. Two distinct bugs were identified and fixed.


Bug 1 — Backend: b64_json response not handled

Root cause:
OpenAIImageAPIImpl.createTempFile() expected a url field in the OpenAI image response, but the gpt-image-1 family always returns b64_json (base64-encoded PNG). These newer models do not support the url response format or the response_format parameter at all, unlike DALL-E 2/3.

Model Response field
DALL-E 2 / DALL-E 3 data[0].url (default)
gpt-image-1 / gpt-image-1.5 / gpt-image-1-mini data[0].b64_json (always)

Error thrown: "Image Response does not include URL" even though the request succeeded.

Fix:
Updated OpenAIImageAPIImpl.createTempFile() to detect the response format and branch accordingly:

  • b64_json present → decode with Base64.getDecoder() → create temp file from ByteArrayInputStream via TempFileAPI.createTempFile()
  • url present → existing behavior via TempFileAPI.createTempFileFromUrl()

Added AiKeys.B64_JSON = "b64_json" constant.


Bug 2 — Frontend: Wrong pixel dimensions sent for non-dall-e-3 models

Root cause:
The AI image prompt form used semantic orientation values (landscape, square, portrait) and a client-side mapper (DotAIImageSizeMapperService) to translate these to model-specific pixel dimensions. However:

  1. The form component defaulted to dall-e-3 before loading the actual model from the config endpoint.
  2. If the user selected an orientation before the config API response arrived, the mapper computed dall-e-3 pixel dimensions (e.g., 1792x1024 for landscape).
  3. When the config then resolved the correct model (e.g., gpt-image-1 → landscape = 1536x1024), the form's size control was updated silently ({ emitEvent: false }) — so the NgRx store was never notified.
  4. The store still held the stale dall-e-3 size and sent it to /api/v1/ai/image/generate.

Additionally, the DotAICompletionsConfig TypeScript interface used an incorrect field name imageModel (which does not exist in the backend response). The backend returns:

  • imageModelNames — comma-separated list of configured image model names
  • availableModels — array of { name, type, current } objects (mirrors com.dotcms.ai.model.SimpleModel)

Fixes:

  • DotAICompletionsConfig: replaced imageModel: string with imageModelNames: string; typed availableModels as DotAISimpleModel[] (new interface matching SimpleModel.java)
  • Model resolution now mirrors the text model approach in CompletionsResource: find availableModels.find(m => m.type === 'IMAGE' && m.current), falling back to the first entry in imageModelNames
  • Removed { emitEvent: false } from sizeControl.setValue() inside loadImageModelConfig() so the NgRx store receives the correct model-specific size after async config load

Files Changed

Backend:

  • dotCMS/src/main/java/com/dotcms/ai/AiKeys.java — added B64_JSON constant
  • dotCMS/src/main/java/com/dotcms/ai/api/OpenAIImageAPIImpl.java — dual-path createTempFile()

Frontend:

  • core-web/libs/dotcms-models/src/lib/dot-ai.model.tsDotAISimpleModel interface; fixed DotAICompletionsConfig
  • core-web/libs/ui/src/lib/components/dot-ai-image-prompt/components/ai-image-prompt-form/ai-image-prompt-form.component.ts — correct model resolution; emits size update to store
  • core-web/libs/ui/src/lib/components/dot-ai-image-prompt/components/ai-image-prompt-form/ai-image-prompt-form.component.spec.ts — regression test for the race condition

Acceptance Criteria
  • Image generation works end-to-end with gpt-image-1 (landscape, square, portrait orientations)
  • Image generation continues to work with DALL-E 3 (no regression)
  • The correct model-specific pixel dimensions are sent to /api/v1/ai/image/generate regardless of when the user selects orientation relative to config load
  • No "Image Response does not include URL" error when using gpt-image-1
Related
  • Epic: #34058
  • Regression Test Task: #34075

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with OpenAIImageAPIImpl.java and AiKeys.java to trace image response handling, then read dot-ai.model.ts and the image prompt form component for model resolution and size updates. Run ai-image-prompt-form.component.spec.ts and verify the listed acceptance cases, including base64 responses, DALL-E 3 compatibility, and model-specific dimensions.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, typescript
Domain
api, backend, frontend, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.