traceloop / traceloop/openllmetry
๐ Bug Report: Whole message history is stringified into a single 'user prompt'
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.4k
- Forks
- 1.1k
- Avg merge
- 8d 14h
- Merged PRs (30d)
- 2
Description
Which component is this bug for?
VertexAI Instrumentation
๐ Description
Passing a list of messages to generate_content combines them all into a single string and stores/presents them as a single user prompt instead of keeping them separate.
๐ Reproduction steps
import vertexai
from traceloop.sdk import Traceloop
from vertexai.generative_models import Content, GenerativeModel, Part
Traceloop.init()
vertexai.init()
model = GenerativeModel('gemini-1.5-flash-002')
response = model.generate_content(
[
Content(
role='user',
parts=[
Part.from_text("What's 2+2?"),
],
),
Content(
role='assistant',
parts=[
Part.from_text('5'),
],
),
Content(
role='user',
parts=[
Part.from_text('really?'),
],
),
]
)
๐ Expected behavior
Attributes look something like:
{
"gen_ai.prompt.0.role": "user",
"gen_ai.prompt.0.content": "What's 2+2?\n",
"gen_ai.prompt.1.role": "assistant",
"gen_ai.prompt.1.content": "5\n",
"gen_ai.prompt.2.role": "user",
"gen_ai.prompt.2.content": "really?\n",
"gen_ai.completion.3.role": "assistant",
"gen_ai.completion.3.content": "Oops! My apologies. 2 + 2 = 4. I'm still under development and learning to perform these calculations correctly.\n"
}
๐ Actual Behavior with Screenshots
Attributes look like:
{
"gen_ai.prompt.0.user": "role: \"user\"\nparts {\n text: \"What\\'s 2+2?\"\n}\n\nrole: \"assistant\"\nparts {\n text: \"5\"\n}\n\nrole: \"user\"\nparts {\n text: \"really?\"\n}\n\n",
"gen_ai.completion.0.role": "assistant",
"gen_ai.completion.0.content": "Oops! My apologies. 2 + 2 = 4. I'm still under development and learning to perform these calculations correctly.\n"
}
๐ค Python Version
No response
๐ Provide any additional context for the Bug.
๐ Have you spent some time to check if this bug has been raised before?
- I checked and didn't find similar issue
Are you willing to submit PR?
None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up โ it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in opentelemetry/instrumentation/vertexai/init.py at lines 124-138, then run the provided VertexAI generate_content reproduction. Trace how the list of Content messages becomes prompt attributes and preserve each message's role and content separately. Done means the attributes match the expected prompt.0, prompt.1, and prompt.2 structure rather than one stringified user prompt.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100