ag-ui-protocol / ag-ui-protocol/ag-ui

[Bug]: ADK converter drops valid binary data when `id` or `url` is also present

未關閉
#2,241 8 則留言 0 個 reaction 已指派 1 人 已被 @kinKingen 認領 在 GitHub 檢視
bug
主要語言
Python
星號
15.9k
分支
1.4k
平均合併
1 天 17 小時
30 天內合併 PR
163

描述

### Pre-flight Checklist

- [x] I have searched [existing issues](https://github.com/ag-ui-protocol/ag-ui/issues) and this hasn't been reported yet.
- [x] I am using the **latest** version AG-UI.

### Describe the Bug

The ADK middleware drops a legacy `BinaryInputContent` item containing valid inline base64 `data` whenever the same item also contains an `id` or `url`.

`BinaryInputContent` allows `id`, `url`, and `data` to coexist. The ADK converter currently supports only inline `data`, so unsupported reference fields should be ignored when usable data is available.

Instead, `_to_binary_part` currently returns `None` as soon as it sees an `id` or `url`:

```python
if url or binary_id:
logger.warning(
"BinaryInputContent: only data is supported; ignoring url/id fields."
)
return None
```

The warning says the `url`/`id` fields are being ignored, but the implementation actually ignores the entire binary item, including its valid inline data. The attachment is silently removed from the converted user message.

Affected file:

`integrations/adk-middleware/python/src/ag_ui_adk/utils/converters.py`

Affected functions:

- `_to_binary_part`
- `convert_message_content_to_parts`

### Steps to Reproduce

1. Run the following from `integrations/adk-middleware/python`:

```python
import base64

from ag_ui.core import BinaryInputContent
from ag_ui_adk.utils.converters import convert_message_content_to_parts

raw = b"hello"
encoded = base64.b64encode(raw).decode("ascii")

item = BinaryInputContent(
mime_type="application/pdf",
data=encoded,
id="upload-123",
filename="report.pdf",
)

parts = convert_message_content_to_parts([item])

print(parts)
```

2. Observe the warning:

```text
BinaryInputContent: only data is supported; ignoring url/id fields.
```

3. Observe that the valid inline attachment was dropped:

```text
[]
```

4. The same behavior occurs when `url` is present alongside `data`:

```python
item = BinaryInputContent(
mime_type="application/pdf",
data=encoded,
url="https://example.com/report.pdf",
filename="report.pdf",
)
```

### Expected Behavior

When valid inline `data` is present, the converter should use it to construct an inline `types.Blob` and ignore only the unsupported `id` or `url` fields:

```python
len(parts) == 1
parts[0].inline_data.mime_type == "application/pdf"
parts[0].inline_data.data == b"hello"
```

If filename propagation is available, the display name should also remain intact:

```python
parts[0].inline_data.display_name == "report.pdf"
```

Items containing only `id` or `url`, without inline `data`, can continue to be ignored because the ADK converter does not currently support those source types.

Or Even allow items with url to exist, only if data is not set, for example files from a gcs bucket.

### Environment

```text

Repository: ag-ui-protocol/ag-ui
Component: integrations/adk-middleware/python
ag-ui-adk: 0.7.0
ag-ui-protocol: 0.1.19
google-adk: 1.35.0
google-genai: 1.75.0
Python: 3.13.7
OS: Windows
```

### Screenshots

_No response_

### Logs & Errors

```shell
BinaryInputContent: only data is supported; ignoring url/id fields.

No exception is raised. The attachment is silently omitted from the converted parts.
```

### Additional Context

`BinaryInputContent` is deprecated, but the ADK converter explicitly retains support for it for backward compatibility. Neither the Python nor TypeScript legacy binary schema makes `id`, `url`, and `data` mutually exclusive; validation requires at least one source but permits multiple fields.

Other AG-UI integrations choose a supported source when multiple legacy binary source fields are present rather than dropping the complete item.

A minimal fix is to retain the warning but remove the early return when inline data is available:

```python
if url or binary_id:
logger.warning(
"BinaryInputContent: using data and ignoring url/id fields."
)

decoded = base64.b64decode(data, validate=True)
return types.Part(
inline_data=types.Blob(
mime_type=mime_type,
data=decoded,
)
)
```

Suggested regression tests:

- `BinaryInputContent` with `data` plus `id` produces one inline part.
- `BinaryInputContent` with `data` plus `url` produces one inline part.
- A dictionary payload with `data` plus `id` produces one inline part.
- Existing `id`-only and `url`-only cases remain ignored.
- Invalid base64 remains ignored even if `id` or `url` is also present.

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。