griptape-ai / griptape-ai/griptape-nodes-library-diffusers

Decode Media's image output can't connect to conditioning inputs without a Display Image node in between

Open
#62 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
0
Avg merge
1d 15h
Merged PRs (30d)
11

Description

Problem

Decode Media's output_image cannot be connected directly to the media-gen conditioning inputs on Generate Media Latents (e.g. reference_images). The editor rejects the connection as a type mismatch. Inserting a Display Image node between the two makes the same connection succeed, so the only way to feed a decoded image back in as a reference is to route it through an unrelated passthrough node.

Decode Media declares output_image with output_type="ImageArtifact", but at runtime it publishes an ImageUrlArtifact (_handle_image_output assigns the result of pil_to_image_artifact, which returns ImageUrlArtifact). The declared type and the produced type disagree; the connection is validated against the declared one.

The conditioning inputs accept ["media_gen_conditioning", "ImageUrlArtifact"]. Type compatibility in the engine is exact-match — there is no subtype relationship between ImageArtifact and ImageUrlArtifact — so the declared ImageArtifact matches neither entry and the connection is refused. Display Image "fixes" it only because it re-emits the value as ImageUrlArtifact, which does match.

Two things worth noting about the scope:

  • This is not specific to Flux dev. Every runtime-parameters class that exposes a media-gen conditioning input is affected, since they all build the input through the same shared component with the same input_types. That's flux2_dev (reference_images), flux2_klein (reference_images), wan_vace (source_media, mask, reference_images), qwen_edit (image_references), wan_i2v / hunyuan_video1_5_i2v / minimax_h3 (conditioning_images), and ltx / ltx2 (media_conditions, reference_conditions).
  • It only shows up on conditioning inputs. Other image consumers in the library (ControlNet, the VAE mask encoder, latent composite mask) list both ImageArtifact and ImageUrlArtifact in their input_types, so the mislabeled output happens to be accepted there. That's why the mismatch has gone unnoticed until now.

output_video on the same node does not have this problem — it is declared VideoUrlArtifact, which is what it actually publishes.

Decode HDR Latents inherits _handle_image_output from the same base class, so its image output carries the same mismatch.

Origin

Reported by Diogo Teixeira, hit while wiring a decoded image back into Generate Media Latents as a reference image on Flux dev. Reported against Flux dev specifically, with a suspicion it affected other providers too — confirmed above that it does.

Deterministic: it's a static type check on the parameter declarations, not a runtime or data-dependent failure.

Impact

Any workflow that feeds a decoded image back in as conditioning — iterative refinement, multi-pass generation, using one generation as a reference for the next — needs a Display Image node inserted purely to launder the artifact type. Affects all conditioning-capable providers in the library. Not blocking, since the workaround exists, but it's non-obvious: the failure gives no hint that an unrelated display node is the fix, and the extra node is confusing to anyone reading the workflow later.

Repro / evidence

  1. Build any workflow with Decode Media feeding a Generate Media Latents node configured for a conditioning-capable provider (Flux dev reproduces it).
  2. Drag Decode Media.output_image onto reference_images.
  3. The connection is refused with a type-mismatch error.
  4. Insert a Display Image node between them — Decode Media.output_imageDisplay Image.image, Display Image.imagereference_images. Both connections are accepted.

Checked directly against the engine's connection validation:

Decode Media output_image .output_type = 'ImageArtifact'
reference_images .input_types = ['media_gen_conditioning', 'ImageUrlArtifact']

CONNECT Decode Media.output_image -> reference_images ALLOWED?  False
CONNECT (via Display Image, ImageUrlArtifact) ALLOWED?          True

are_types_compatible ImageArtifact -> ImageUrlArtifact:         False

The rejection is the is_incoming_type_allowed(source_param.output_type) check in the engine's flow manager, reached before either node gets a say in the connection.

Possible direction

Not investigated as to which end should change. Correcting output_image's declared type on Decode Media to match what it publishes looks like the narrower fix, but it changes the advertised type of an output that existing saved workflows already connect, so the compatibility impact of that wants checking before it's chosen. Widening the conditioning inputs to accept ImageArtifact as well is the other option — though if nothing in the library actually emits a bare ImageArtifact, that papers over the mismatch rather than fixing it. Worth a look at whether ImageArtifact is the right declared type anywhere it currently appears.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect _handle_image_output, pil_to_image_artifact, and the flow manager's is_incoming_type_allowed check first. Compare the declared and produced types on Decode Media and Decode HDR Latents with the conditioning input types across the listed providers, including saved-workflow compatibility. Done means direct decoded-image connections work without Display Image and existing valid connections remain compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.