Comfy-Org / Comfy-Org/ComfyUI

Autogrow input via /prompt API: nested-dict link format silently ignored (no error, empty data reaches the model)

Open
#15,305 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
133k
Forks
15.7k
Avg merge
1d 7h
Merged PRs (30d)
158

Description

### Custom Node Testing

- [x] I have tried disabling custom nodes and the issue persists — this reproduces with `TextEncodeMageFlowEdit`, a core node shipped in `comfy_extras/nodes_mage.py`, no third-party custom nodes involved.

### Expected Behavior

Wiring a link into a sub-slot of an `io.Autogrow` input via the flat `/prompt` API JSON (not the graph editor UI) should either resolve it into real data, or fail validation/execution with a clear error if the format is wrong.

### Actual Behavior

**Two different "obvious" JSON shapes for wiring a link into an Autogrow sub-slot both pass `/prompt` validation *and* execute to `status_str: success` with zero errors — while silently feeding the node an empty/missing value.** There is no error, warning, or any other signal that the reference never connected; the render just quietly produces output as if the input were never given at all.

Tested against `TextEncodeMageFlowEdit` (`images` is `io.Autogrow.Input` with `io.Autogrow.TemplateNames(..., names=[f"image_{i}" for i in range(1, 17)])`), but the underlying cause is generic to any Autogrow input wired by hand through the API rather than the graph editor.

Shape 1 — flat key matching the sub-slot name:
```json
"5": {"class_type": "TextEncodeMageFlowEdit", "inputs": {
"clip": ["3", 0], "vae": ["4", 0], "prompt": "...", "negative_prompt": "",
"width": 1024, "height": 1024, "batch_size": 1,
"image_1": ["14", 0]
}}
```
→ `/prompt` returns `node_errors: {}`, queues and runs to completion. At execution, the node function itself errors ONLY because I explicitly added an `image_1` kwarg that doesn't exist on `execute()`'s signature (`TypeError: execute() got an unexpected keyword argument 'image_1'. Did you mean 'images'?`) — this particular shape does fail, just very late and unhelpfully.

Shape 2 — nested dict matching the Python kwarg shape:
```json
"5": {"class_type": "TextEncodeMageFlowEdit", "inputs": {
...,
"images": {"image_1": ["14", 0]}
}}
```
→ `/prompt` returns `node_errors: {}`, queues and runs to completion with `status_str: success`, produces a real, valid, error-free image — **but the reference image was never actually used**. I confirmed this by adding temporary debug prints through the whole call chain: `TextEncodeMageFlowEdit.execute()` received `images == {}` (so `images_in=0`, `ref_latents_built=0`), all the way down to `MageFlowTransformer2DModel._forward()` receiving `ref_latents=None`. The render this produces is a plausible, coherent, *completely unrelated* image with no connection to the reference at all — which is what makes this bug so easy to miss: everything downstream behaves exactly as if the feature is working correctly.

**Root cause**: `execution.py`'s `is_link()` (`comfy_execution/graph_utils.py`) only recognizes a bare `[node_id: str, output_index: int]` 2-element list as a resolvable link when it is a **top-level value** in a node's `inputs` dict (`get_input_data`, `for x in inputs: input_data = inputs[x]; if is_link(input_data): ...`). It never recurses into a nested dict/list value looking for links to resolve, so Shape 2's inner `["14", 0]` is never seen as a link at all — it's handed to the node as a literal, unresolved value (or dropped, depending on what happens next in the V3 dynamic-input dict-wrapping).

**The actual correct shape** (found by reading `comfy_api/latest/_io.py`'s `build_nested_inputs`) is a **flat, dotted key per sub-slot**:
```json
"5": {"class_type": "TextEncodeMageFlowEdit", "inputs": {
...,
"images.image_1": ["14", 0]
}}
```
This works correctly end-to-end — `build_nested_inputs` reassembles dotted-prefix keys (via the `dynamic_paths` map) back into the `images` dict the node's `execute()` signature expects, but only *after* normal top-level link resolution has already run on the dotted key itself, since at that point it's a plain top-level `is_link()`-recognizable value.

### Steps to Reproduce

1. Load a workflow using `TextEncodeMageFlowEdit` (or any `io.Autogrow`-input node) with the `image_mage_flow_edit_turbo_int8` template, or build the graph directly.
2. POST it to `/prompt` with the reference image wired as `"images": {"image_1": [node_id, 0]}` (nested dict matching the Python kwarg shape) instead of `"images.image_1": [node_id, 0]` (flat dotted key).
3. Observe: `node_errors` is empty, the prompt queues and completes with `status_str: success`, and a real image is produced — but it has no relationship to the reference image at all.
4. Re-run with `"images.image_1": [node_id, 0]` instead — the same prompt now produces an image that correctly preserves the reference's composition.

### Debug Logs

```
[DEBUG] node execute: images_in=0 vae=True ref_latents_built=0 # nested-dict shape (Shape 2)
[DEBUG] QwenImage.extra_conds kwargs keys=[..., no 'reference_latents' key at all ...]
[DEBUG] MageFlowTransformer2DModel._forward: ref_latents=None

[DEBUG] node execute: images_in=1 vae=True ref_latents_built=1 # dotted-key shape (fixed)
[DEBUG] QwenImage.extra_conds kwargs keys=[..., 'reference_latents' ...] reference_latents=1
[DEBUG] MageFlowTransformer2DModel._forward: ref_latents=[torch.Size([1, 128, 64, 64])]
```

### Other

Possibly related to #13500 (Autogrow + Primitive connection "behaving as disconnected" in the graph editor) - that one is a hard validation failure through the UI, this one is a silent, undetected pass-through of empty data through the raw API, which seems like the more dangerous variant since there's no error at any point to signal something is wrong. Might share a root cause in how Autogrow sub-slot paths get resolved, or might be two separate issues in the same subsystem - filing separately since the actual failure mode (loud vs. silent) is different enough to need its own repro.

Given how easy this is to hit silently (both "wrong" JSON shapes I tried looked completely reasonable and neither errored), it might be worth having `/prompt` validation actively reject an Autogrow input that received a raw dict/nested value instead of the expected dotted-key link, rather than silently treating it as either missing or opaque data.

Contributor guide

Open the contributing guide

Research direction

Start with comfy_execution/graph_utils.py, especially is_link() and get_input_data(), then read comfy_api/latest/_io.py and build_nested_inputs(). Reproduce the nested-dict and dotted-key /prompt payloads against an Autogrow input. Done means the nested form either resolves the reference correctly or returns a clear validation/execution error instead of silently succeeding with empty data.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.