googleapis / googleapis/nodejs-agentplatform

Incorrect type: `content.parts` is undefined when request finishes due to safety reason (`generativeModel.generateContent`)

Open
#480 2 comments 0 reactions 0 assignees View on GitHub
api: aiplatform priority: p2 type: bug
Dominant language
TypeScript
Stars
182
Forks
69
Avg merge
1d 8h
Merged PRs (30d)
10

Description

When making a call to `vertexAI.getGenerativeModel(...).generateContent(...)` that finishes due to a safety reason, the `candidates` list looks like this:
```json
[
{
"content": {
"role": "model"
},
"finishReason": "SAFETY",
"index": 0,
"safetyRatings": [
{
"category": "HARM_CATEGORY_HATE_SPEECH",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.3984375,
"severity": "HARM_SEVERITY_LOW",
"severityScore": 0.33789063
},
{
"category": "HARM_CATEGORY_DANGEROUS_CONTENT",
"probability": "NEGLIGIBLE",
"probabilityScore": 0.14453125,
"severity": "HARM_SEVERITY_LOW",
"severityScore": 0.24316406
},
{
"blocked": true,
"category": "HARM_CATEGORY_HARASSMENT",
"probability": "LOW",
"probabilityScore": 0.49804688,
"severity": "HARM_SEVERITY_MEDIUM",
"severityScore": 0.47265625
},
{
"blocked": true,
"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
"probability": "MEDIUM",
"probabilityScore": 0.7265625,
"severity": "HARM_SEVERITY_HIGH",
"severityScore": 0.77734375
}
]
}
]
```
This is not assignable to the type given as the return value of `generateContent`, as `[0].content.parts` is undefined.

#### Environment details

- Programming language: Typescript
- OS: Linux `6.6.63-1-lts`
- Language runtime version: node `v22.9.0` / `@swc-node/register/esm-register@1.10.9`
- Package version: `1.9.2`

#### Steps to reproduce

1. Make a `generateContent` request that fails due to a safety reason. I set the following safety settings and then prompted the model to translate a swear word into several languages. (Note that I am asking for json here, so the prompt should instruct the model to output json)
```typescript
const vertexAI = new VertexAI({ project: GOOGLE_CLOUD_PROJECT });
const generativeModel = vertexAI.getGenerativeModel({
model: "gemini-1.5-pro",
systemInstruction: {
role: "system",
parts: [{ text: prompt }],
},
safetySettings: [
{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE },
{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE },
{ category: HarmCategory.HARM_CATEGORY_UNSPECIFIED, threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE },
{
category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
},
{
category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
threshold: HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
},
],
generationConfig: {
temperature: 0,
},
});

const result = await generativeModel.generateContent({
contents: [
{
role: "user",
parts: [
{
text,
},
],
},
],
generationConfig: {
responseMimeType: "application/json",
},
});
```
2. See that `result.response.candidates[0].content.parts` is undefined, even though typescript says it should be `Part[]`

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.