cloudflare / cloudflare/workerd

Workers AI Function Calling types

Open
#2,418 1 comment 1 reaction 1 assignee Claimed by @workers-devprod View on GitHub
types
Dominant language
C++
Stars
8.7k
Forks
739
Avg merge
2d 20h
Merged PRs (30d)
174

Description

I copied the "traditional" example at https://developers.cloudflare.com/workers-ai/function-calling/ into a worker like this:

```typescript
export default {
async fetch(request, env, ctx): Promise {
const response = await env.AI.run(
"@hf/nousresearch/hermes-2-pro-mistral-7b", {
messages: [{
role: "user",
content: "Who is Cloudflare on github?"
}],
tools: [{
name: "getGithubUser",
description: "Provides publicly available information about someone with a GitHub account.",
parameters: {
type: "object",
properties: {
username: {
type: "string",
description: "The handle for the GitHub user account.",
},
},
required: ["username"],
},
}, ],
},
);

const selected_tool = response.tool_calls[0];
let res;

if (selected_tool.name == "getGithubUser") {
try {
const username = selected_tool.arguments.username;
const url = `https://api.github.com/users/${username}`;
res = await fetch(url, {
headers: {
// Github API requires a User-Agent header
"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
},
}).then((res) => res.json());
} catch (error) {
return error;
}
}

const finalResponse = await env.AI.run(
"@hf/nousresearch/hermes-2-pro-mistral-7b", {
messages: [{
role: "user",
content: "Who is Cloudflare on github?",
},
{
role: "assistant",
content: "",
tool_call: selected_tool.name,
},
{
role: "tool",
name: selected_tool.name,
content: JSON.stringify(res),
},
],
tools: [{
name: "getGithubUser",
description: "Provides publicly available information about someone with a GitHub account.",
parameters: {
type: "object",
properties: {
username: {
type: "string",
description: "The handle for the GitHub user account.",
},
},
required: ["username"],
},
}, ],
},
);
return new Response(JSON.stringify(finalResponse));
},
} satisfies ExportedHandler;
```

where `Env` is:

```typescript
interface Env {
AI: Ai;
}
```

This resulted in the following TypeScript errors in VSCode and when I ran `npx tsc --noEmit`:
```
src/index.ts:4:4 - error TS2769: No overload matches this call.
The last overload gave the following error.
Argument of type '"@hf/nousresearch/hermes-2-pro-mistral-7b"' is not assignable to parameter of type 'BaseAiImageToTextModels'.

4 "@hf/nousresearch/hermes-2-pro-mistral-7b", {
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/@cloudflare/workers-types/experimental/index.d.ts:3718:3
3718 run(
~~~
The last overload is declared here.

src/index.ts:26:34 - error TS2339: Property 'tool_calls' does not exist on type '(((AiTextClassificationOutput & Uint8Array & (AiTextEmbeddingsOutput & AiSpeechRecognitionOutput & AiTextGenerationOutput)) & AiTranslationOutput) & AiSummarizationOutput) & AiImageToTextOutput'.
Property 'tool_calls' does not exist on type 'AiTextClassificationOutput & Uint8Array & AiTextEmbeddingsOutput & AiSpeechRecognitionOutput & ReadableStream<...> & AiTranslationOutput & AiSummarizationOutput & AiImageToTextOutput'.

26 const selected_tool = response.tool_calls[0];
~~~~~~~~~~

src/index.ts:40:5 - error TS2322: Type 'unknown' is not assignable to type 'Response'.

40 return error;
~~~~~~

src/index.ts:45:4 - error TS2769: No overload matches this call.
The last overload gave the following error.
Argument of type '"@hf/nousresearch/hermes-2-pro-mistral-7b"' is not assignable to parameter of type 'BaseAiImageToTextModels'.

45 "@hf/nousresearch/hermes-2-pro-mistral-7b", {
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

node_modules/@cloudflare/workers-types/experimental/index.d.ts:3718:3
3718 run(
~~~
The last overload is declared here.

Found 4 errors in the same file, starting at: src/index.ts:4
```

I suspect this is somewhat related to the function overloading DX issues as reported at https://github.com/cloudflare/workerd/issues/2181.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.