getsentry / getsentry/sentry-javascript

Instrument GenAI image generation (OpenAI, Google GenAI, Vercel AI)

Aperta
#22,046 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
TypeScript
Stelle
8.7k
Fork
1.8k
Merge medio
1g 17h
PR unite (30g)
515

Descrizione

Image generation is not instrumented by any Sentry JS AI integration. This is a platform-wide gap (not OpenAI- or Vercel-specific) and ultimately stems from the OpenTelemetry GenAI semantic conventions not defining an image-generation operation yet. Text, agent, and embedding calls are instrumented fine.

*Investigated against* `@sentry/node` */* `@sentry/core` *10.62.0,* `ai` *7.0.4,* `@ai-sdk/openai` *4.0.2.*

## Current state

| Integration | Instrumented methods | Image gen? |
| -- | -- | -- |
| OpenAI | `responses.create`, `chat.completions.create`, `embeddings.create`, `conversations.create` | ❌ |
| Anthropic | `messages.create`, `messages.stream`, `completions.create`, `models.*`, `beta.messages.create` | ❌ (no API) |
| Google GenAI | `models.generateContent(Stream)`, `models.embedContent`, `chats.create`, `chat.sendMessage(Stream)` | ❌ |
| LangChain | chat / LLM / embeddings callbacks | ❌ |
| LangGraph | agent / graph invocation | ❌ |
| Vercel AI | `generateText`, `streamText`, `generateObject`, `streamObject`, `embed`, `embedMany`, `rerank` | ❌ |

### Why it's missing (two layers)

1. **Sentry side:** No integration lists an image method. An exhaustive grep for `generateImage` / `images.generate` / `imagen` / `dall` / `gen_ai.image` across every AI tracing module returns nothing.
2. **Provider side:** Even where Sentry relies on the SDK's own telemetry (Vercel AI), the SDK emits none for images. `ai`'s `generateImage` opens no OTel span and does not accept `experimental_telemetry` (confirmed: its compiled body has zero telemetry/tracer references, unlike `generateText`).

## What would need to happen

### Provider methods to instrument

* **OpenAI:** add `images.generate`, `images.edit`, `images.createVariation` to `OPENAI_METHOD_REGISTRY`
* **Google GenAI:** add `models.generateImages` (Imagen)
* **Vercel AI:** add `generateImage` to `INSTRUMENTED_METHODS`
* **Anthropic / LangChain / LangGraph:** N/A (no first-class image-gen primitive)

### Required SDK work beyond the method list

1. **New span op + semantics:** define `gen_ai.generate_image` and map image-specific fields (`size`, `quality`, image count `n`, model) onto `gen_ai.request.*`; handle image-based usage/billing (per-image/resolution, not just tokens) for `gen_ai.usage.*`.
2. **Vercel AI needs a self-built span:** unlike text methods, `generateImage` emits no telemetry, so Sentry's Proxy wrapper must create the span itself (as the OpenAI integration already does manually) rather than toggling `experimental_telemetry`.
3. **Dashboard support:** confirm Insights → AI Agents renders a `generate_image` op (or treat it as a trace-only span initially).
4. **Upstream alignment:** ideally land an image-generation operation in the OTel GenAI semconv so attribute names are standard rather than Sentry-specific.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by locating OPENAI_METHOD_REGISTRY and INSTRUMENTED_METHODS, then compare their existing AI tracing entry points with Vercel AI's generateImage telemetry gap. Review the proposed gen_ai.generate_image fields and image-based usage requirements. Done means the listed provider methods are instrumented, dashboard handling is confirmed, and upstream semantic-convention alignment is addressed.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
typescript
Ambito
ai, observability-sre
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.