MemberJunction / MemberJunction/MJ
Issue: Image/Media Attachments in Conversation Need Placeholder System
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 323
Description
## Problem Statement
There are two related issues with how images/media attachments are handled in agent conversations:
### Issue 1: LLMs Truncate Base64 Data in Outputs
When Sage (or any agent) needs to pass an image from the conversation to an action like `Generate Image` with `SourceImage` parameter, the LLM cannot reliably copy the full base64 data. Base64 images are typically 500KB-2MB+ of text, and LLMs naturally truncate such massive strings in their outputs.
**Example of truncated output from Sage:**
```json
{
"actions": [
{
"name": "Generate Image",
"params": {
"Prompt": "Transform the man to a cowboy...",
"SourceImage": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAYAAAByNR6YAAAgAElEQVR4Xu3dB3Mc1R3H8e8f//784iIiIiIiIogqiHv9+/jzP3vnPf5...",
"Model": "Nano Banana Pro"
}
}
]
}
```
The `...` is literally in the LLM output - it truncated the base64 string, making it unusable.
### Issue 2: Token Inefficiency in Conversation Messages
The conversation messages sent to the LLM contain full base64 data, which is extremely token-inefficient:
```json
{
"conversationMessages": [
{
"role": "user",
"content": [
{
"type": "text",
"content": "Sage can you read about this image and generate me something similar?"
},
{
"type": "image_url",
"content": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAm0AAAJyCAIAAAAZ6WCnAAAAIGNIUk0AAHomAACAhAAA+gAAAIDoAAB1MAAA6mAAADqYAAAXcJy6UTwAAAAGYktHRAD/AP8A/6C9p5MAAAABb3JOVAHPoneaAACAAElEQVR42uz995Nl6XnfCT6vPf7atOW72sCQcCRIihqMxNidCEVshEzExowUCsVKG9of9Fco9Iv+jNVsSCNDShqtZikORQokBIIE0A7t0VVdXSarKt3Na48/r3n2h/fmrWwDotFddAPE+4mKjKzMc93Jm/k9j/s+BBHB4/F4PB7Px4J+1k/A4/F4PJ6fY7yOejwej8fz8fE66..."
}
]
}
]
}
```
This wastes tokens and increases costs significantly, especially with multiple images in a conversation.
Contributor guide
Assessment
This issue has not been assessed yet.