ag-ui-protocol / ag-ui-protocol/ag-ui

[Bug]: ADK converter drops valid binary data when `id` or `url` is also present

Aperta
#2,241 8 commenti 0 reazioni 1 assegnatario Rivendicata da @kinKingen Vedi su GitHub
bug
Lingua principale
Python
Stelle
15.9k
Fork
1.4k
Merge medio
1g 17h
PR unite (30g)
163

Descrizione

### Pre-flight Checklist

- [x] I have searched [existing issues](https://github.com/ag-ui-protocol/ag-ui/issues) and this hasn't been reported yet.
- [x] I am using the **latest** version AG-UI.

### Describe the Bug

The ADK middleware drops a legacy `BinaryInputContent` item containing valid inline base64 `data` whenever the same item also contains an `id` or `url`.

`BinaryInputContent` allows `id`, `url`, and `data` to coexist. The ADK converter currently supports only inline `data`, so unsupported reference fields should be ignored when usable data is available.

Instead, `_to_binary_part` currently returns `None` as soon as it sees an `id` or `url`:

```python
if url or binary_id:
logger.warning(
"BinaryInputContent: only data is supported; ignoring url/id fields."
)
return None
```

The warning says the `url`/`id` fields are being ignored, but the implementation actually ignores the entire binary item, including its valid inline data. The attachment is silently removed from the converted user message.

Affected file:

`integrations/adk-middleware/python/src/ag_ui_adk/utils/converters.py`

Affected functions:

- `_to_binary_part`
- `convert_message_content_to_parts`

### Steps to Reproduce

1. Run the following from `integrations/adk-middleware/python`:

```python
import base64

from ag_ui.core import BinaryInputContent
from ag_ui_adk.utils.converters import convert_message_content_to_parts

raw = b"hello"
encoded = base64.b64encode(raw).decode("ascii")

item = BinaryInputContent(
mime_type="application/pdf",
data=encoded,
id="upload-123",
filename="report.pdf",
)

parts = convert_message_content_to_parts([item])

print(parts)
```

2. Observe the warning:

```text
BinaryInputContent: only data is supported; ignoring url/id fields.
```

3. Observe that the valid inline attachment was dropped:

```text
[]
```

4. The same behavior occurs when `url` is present alongside `data`:

```python
item = BinaryInputContent(
mime_type="application/pdf",
data=encoded,
url="https://example.com/report.pdf",
filename="report.pdf",
)
```

### Expected Behavior

When valid inline `data` is present, the converter should use it to construct an inline `types.Blob` and ignore only the unsupported `id` or `url` fields:

```python
len(parts) == 1
parts[0].inline_data.mime_type == "application/pdf"
parts[0].inline_data.data == b"hello"
```

If filename propagation is available, the display name should also remain intact:

```python
parts[0].inline_data.display_name == "report.pdf"
```

Items containing only `id` or `url`, without inline `data`, can continue to be ignored because the ADK converter does not currently support those source types.

Or Even allow items with url to exist, only if data is not set, for example files from a gcs bucket.

### Environment

```text

Repository: ag-ui-protocol/ag-ui
Component: integrations/adk-middleware/python
ag-ui-adk: 0.7.0
ag-ui-protocol: 0.1.19
google-adk: 1.35.0
google-genai: 1.75.0
Python: 3.13.7
OS: Windows
```

### Screenshots

_No response_

### Logs & Errors

```shell
BinaryInputContent: only data is supported; ignoring url/id fields.

No exception is raised. The attachment is silently omitted from the converted parts.
```

### Additional Context

`BinaryInputContent` is deprecated, but the ADK converter explicitly retains support for it for backward compatibility. Neither the Python nor TypeScript legacy binary schema makes `id`, `url`, and `data` mutually exclusive; validation requires at least one source but permits multiple fields.

Other AG-UI integrations choose a supported source when multiple legacy binary source fields are present rather than dropping the complete item.

A minimal fix is to retain the warning but remove the early return when inline data is available:

```python
if url or binary_id:
logger.warning(
"BinaryInputContent: using data and ignoring url/id fields."
)

decoded = base64.b64decode(data, validate=True)
return types.Part(
inline_data=types.Blob(
mime_type=mime_type,
data=decoded,
)
)
```

Suggested regression tests:

- `BinaryInputContent` with `data` plus `id` produces one inline part.
- `BinaryInputContent` with `data` plus `url` produces one inline part.
- A dictionary payload with `data` plus `id` produces one inline part.
- Existing `id`-only and `url`-only cases remain ignored.
- Invalid base64 remains ignored even if `id` or `url` is also present.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.