googleapis / googleapis/nodejs-agentplatform

Incorrect type: `content.parts` is undefined when request finishes due to safety reason (`generativeModel.generateContent`)

Aperta
#480 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
api: aiplatform priority: p2 type: bug
Lingua principale
TypeScript
Stelle
182
Fork
69
Merge medio
1g 8h
PR unite (30g)
10

Descrizione

When making a call to `vertexAI.getGenerativeModel(...).generateContent(...)` that finishes due to a safety reason, the `candidates` list looks like this:
```json
[
{
"content": {
"role": "model"
},
"finishReason": "SAFETY",
"index": 0,
"safetyRatings": [
{
"category": "HARM_CATEGORY_HATE_SPEECH",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.3984375,
"severity": "HARM_SEVERITY_LOW",
"severityScore": 0.33789063
},
{
"category": "HARM_CATEGORY_DANGEROUS_CONTENT",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.14453125,
"severity": "HARM_SEVERITY_LOW",
"severityScore": 0.24316406
},
{
"blocked": true,
"category": "HARM_CATEGORY_HARASSMENT",
"probability": "LOW",
"probabilityScore": 0.49804688,
"severity": "HARM_SEVERITY_MEDIUM",
"severityScore": 0.47265625
},
{
"blocked": true,
"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
"probability": "MEDIUM",
"probabilityScore": 0.7265625,
"severity": "HARM_SEVERITY_HIGH",
"severityScore": 0.77734375
}
]
}
]
```
This is not assignable to the type given as the return value of `generateContent`, as `[0].content.parts` is undefined.

#### Environment details

- Programming language: Typescript
- OS: Linux `6.6.63-1-lts`
- Language runtime version: node `v22.9.0` / `@swc-node/register/esm-register@1.10.9`
- Package version: `1.9.2`

#### Steps to reproduce

1. Make a `generateContent` request that fails due to a safety reason. I set the following safety settings and then prompted the model to translate a swear word into several languages. (Note that I am asking for json here, so the prompt should instruct the model to output json)
```typescript
const vertexAI = new VertexAI({ project: GOOGLE_CLOUD_PROJECT });
const generativeModel = vertexAI.getGenerativeModel({
model: "gemini-1.5-pro",
systemInstruction: {
role: "system",
parts: [{ text: prompt }],
},
safetySettings: [
{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE },
{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE },
{ category: HarmCategory.HARM_CATEGORY_UNSPECIFIED, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE },
{
category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
},
{
category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
},
],
generationConfig: {
temperature: 0,
},
});

const result = await generativeModel.generateContent({
contents: [
{
role: "user",
parts: [
{
text,
},
],
},
],
generationConfig: {
responseMimeType: "application/json",
},
});
```
2. See that `result.response.candidates[0].content.parts` is undefined, even though typescript says it should be `Part[]`

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia dalle dichiarazioni TypeScript per vertexAI.getGenerativeModel(...).generateContent e confronta il tipo Candidate/Content con la risposta bloccata da SAFETY mostrata qui. Riproduci il caso SAFETY, quindi aggiungi o aggiorna la coverage in modo che il risultato dichiarato consenta content senza parts, mentre le risposte normali rimangano tipizzate correttamente.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
google-cloud, typescript
Ambito
api
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.