Standardized Workflow Capability Layer for OpenAI-Compatible Image Generation API

### Feature Idea

# Issue: Proposal: Standardized Workflow Capability Layer for OpenAI-Compatible Image Generation API

## Background

Currently, ComfyUI provides a powerful node-based workflow system, but workflows are highly flexible and do not have a standardized external interface.

Meanwhile, many AI applications are adopting OpenAI-compatible APIs such as:

* `POST /images/generations`
* `POST /images/edits`

However, mapping these APIs to ComfyUI workflows currently requires each integration project to implement its own conversion logic.

Different implementations need to solve the same problems repeatedly:

* How to map API parameters to workflow nodes?
* How to describe supported models and capabilities?
* How to handle dynamic inputs such as multiple images?
* How to validate workflow limitations?
* How to dynamically generate workflows based on request parameters?

A standardized capability layer could make ComfyUI workflows consumable as AI services.

---

# Proposed Architecture

Introduce a **Workflow Capability Description Layer** between API requests and ComfyUI workflows.

Architecture:

```
Client
|
| OpenAI Compatible API
|
v
ComfyUI API Gateway
|
| Capability Resolver
|
v
Workflow Template Engine
|
| Dynamic Workflow Generation
|
v
ComfyUI Execution Engine
|
v
Result
```

---

# 1. Workflow Capability Manifest

Each workflow should optionally provide a capability description.

Example:

```json
{
"id": "flux-image-edit",
"type": "image.edit",
"version": "1.0",

"inputs": {
"prompt": {
"type": "string",
"required": true
},

"images": {
"type": "array",
"itemType": "image",
"maxItems": 4
},

"strength": {
"type": "number",
"min": 0,
"max": 1,
"default": 0.8
}
},

"outputs": {
"images": {
"type": "array"
}
},

"constraints": {
"maxResolution": "2048x2048",
"batchLimit": 8
}
}
```

The manifest describes:

* supported API type
* parameters
* validation rules
* model limitations
* input/output capabilities

---

# 2. Model Capability List API

Provide a model discovery endpoint:

```
GET /v1/models
```

Example:

```json
{
"data": [
{
"id": "flux-edit",
"type": "image",
"capabilities": {
"generation": true,
"editing": true,
"multiImage": true,
"maxImages": 4,
"maxResolution": "2048x2048"
}
}
]
}
```

Clients can dynamically determine:

* supported operations
* image count limits
* resolution limits
* supported parameters

---

# 3. Dynamic Workflow Generation

The workflow should not be stored as a single fixed JSON.

Instead:

```
Template Workflow

+

Request Parameters

|

Workflow Generator

|

Temporary Workflow JSON

|

ComfyUI Execution
```

Example:

Request:

```json
{
"model":"flux-edit",
"image":[
"a.png",
"b.png",
"c.png"
],
"prompt":"combine these images"
}
```

The generator creates:

```
LoadImage
|
+----+
| |
LoadImage
|
+----+
|
LoadImage

|
Flux Edit

|
Output 4 Images
```

The number of nodes is generated dynamically.

---

# 4. Input Binding System

Workflow nodes should support parameter binding.

Example:

```json
{
"node":"KSampler",
"bindings":{
"steps":"$.steps",
"cfg":"$.cfg"
}
}
```

Request:

```json
{
"steps":30,
"cfg":7
}
```

Automatically maps:

```
request.steps
|
v
KSampler.steps
```

---

# 5. Multi Image Handling

Special handling is required for image arrays.

Example:

API:

```json
{
"images":[
"1.png",
"2.png",
"3.png"
]
}
```

Workflow generator:

```
foreach image:

Create LoadImage Node

Connect dynamically

Create Batch Node

Connect downstream
```

The workflow itself should not need to know the number of images.

---

# 6. Desktop / Server Responsibility

Dynamic workflow generation should belong to the service layer, not individual nodes.

Reason:

The workflow's responsibility:

```
execute computation
```

The API layer's responsibility:

```
validate request
generate execution graph
manage sessions
manage queue
manage synchronization
```

Therefore:

```
ComfyUI Node
|
|
Workflow Template
|
|
API Service Layer
```

should be separated.

---

# 7. Compatibility Goal

After this layer exists:

Any client supporting:

```
OpenAI Image API
```

can connect to ComfyUI without knowing:

* node structure
* workflow JSON format
* model details

Example:

```python
client.images.generate(
model="flux-edit",
prompt="change background",
image=[
img1,
img2
]
)
```

Internally:

```
OpenAI Request

|

Capability Resolver

|

Dynamic Workflow

|

ComfyUI
```

---

# Expected Benefits

* Unified AI image API ecosystem
* Easier third-party integration
* Workflow sharing without manual configuration
* Dynamic multi-image support
* Model capability discovery
* Better ComfyUI Desktop backend support
* Avoid every project implementing its own adapter

---

# Open Questions

1. Should capability manifests become part of workflow JSON?
2. Should there be a new workflow metadata file format?
3. Should dynamic workflow generation happen inside ComfyUI core or external service?
4. How should custom nodes expose their supported parameters?
5. How should model limitations be standardized?

---

## Summary

This proposal does not add another API wrapper.

The goal is to introduce a **standard capability description and workflow generation layer**, allowing ComfyUI workflows to become discoverable, configurable, and compatible with modern AI service APIs.

### Existing Solutions

_No response_

### Other

_No response_

The proposal names an API gateway, capability resolver, workflow template engine, and execution engine, but no existing files or tests. Start by locating ComfyUI's current API and workflow JSON handling, then determine where manifests, model discovery, validation, and dynamic graph generation would belong. Done would require an agreed architecture and implementation scope for the open questions.

Written by the indexing model from the issue text.