open-telemetry / open-telemetry/opentelemetry-python-contrib

opentelemetry-instrumentation-vertexai: support StreamGenerateContent

Open
#3,297 2 comments 1 reaction 1 assignee View on GitHub

@aabmass is already working on this.

Since Feb 25, 2025.

feature-request gen-ai instrumentation
Dominant language
Python
Stars
1.1k
Forks
1.1k
Avg merge
4d 15h
Merged PRs (30d)
16

Description

What problem do you want to solve?

Right now, if you run use langchain with VertexAI in a chatbot, the initialization will appear like this:

    return ChatVertexAI(
        model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
    )

When a generation occurs, if you have bootstrapped your deps, you'll see a normal platform span like this:

google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent

You won't yet see a genai span as while non-streaming has, streaming hasn't yet been implemented.
https://github.com/open-telemetry/opentelemetry-python-contrib/blob/opentelemetry-instrumentation-vertexai%3D%3D2.0b0/instrumentation-genai/opentelemetry-instrumentation-vertexai/src/opentelemetry/instrumentation/vertexai/__init__.py#L81-L82

Describe the solution you'd like

I'd like the next release of opentelemetry-instrumentation-vertexai to include google.cloud.aiplatform.v1beta1.PredictionService/StreamGenerateContent

Describe alternatives you've considered

Currently, we use langtrace as the data is most similar to the semantic conventions.

    from langtrace_python_sdk.instrumentation import VertexAIInstrumentation

    VertexAIInstrumentation().instrument()
    return ChatVertexAI(
        model_name=os.getenv("CHAT_MODEL"), streaming=True, temperature=temperature
    )
Additional Context

cc @aabmass and FYI this is the specific code I would like to remove https://github.com/elastic/elasticsearch-labs/blob/main/example-apps/chatbot-rag-app/api/llm_integrations.py#L23-L26

Would you like to implement a fix?

None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.