cloudflare / cloudflare/workerd
Improve Workers AI types
- Dominant language
- C++
- Stars
- 8.7k
- Forks
- 739
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 174
Description
Today, the Workers AI types rely heavily on function overloads to specify arguments for different models. This unfortunately results in very difficult to debug types, and poor DX.
As an example with a more simplified non-AI class: https://tsplay.dev/Nan9ym
Throughout this during development, you get very little assistance with auto-complete, and the errors you get back, are extremely complex and hard to parse:
```
No overload matches this call.
Overload 1 of 3, '(fruit: "apple" | "orange", color: "apple" | "orange", options: { foo: boolean; bar: boolean; }): Promise', gave the following error.
Argument of type '"lemon"' is not assignable to parameter of type '"apple" | "orange"'.
Overload 2 of 3, '(fruit: "tomato" | "pear", color: "tomato" | "pear", options: { foo: string; bar?: boolean | undefined; }): Promise', gave the following error.
Argument of type '"lemon"' is not assignable to parameter of type '"tomato" | "pear"'.
Overload 3 of 3, '(fruit: "lemon" | "lime", color: "lemon" | "lime", options: { foo?: string | undefined; bar: number; somethingElse: boolean; }): Promise', gave the following error.
Type 'null' is not assignable to type 'number'.
```
Let's use some real AI examples. These can all be found in https://tsplay.dev/wQB41N - scroll down to line 255 for examples.
## Invalid model names
```ts
AI.run('@hf/thebloke/neural-chat-7b-v3-awq', {
messages: [
{
role: "system",
content: `[very large system prompt]`
},
{
role: "user",
content: `How many pounds of food does a person eat in a year?`
}
],
stream: false
});
```
This errors with:
```
No overload matches this call.
The last overload gave the following error.
Argument of type '"@hf/thebloke/neural-chat-7b-v3-awq"' is not assignable to parameter of type '"@cf/unum/uform-gen2-qwen-500m" | "@cf/llava-hf/llava-1.5-7b-hf"'.
```
This is because I typo'd the model name and it should be `@hf/thebloke/neural-chat-7b-v3-1-awq`.
## Bad inputs
```ts
AI.run('@hf/thebloke/neural-chat-7b-v3-1-awq', {
message: [
{
role: "system",
content: `[very large system prompt]`
},
{
role: "user",
content: `How many pounds of food does a person eat in a year?`
}
],
stream: false
});
```
Here I've typo'd `messages` to `message`, but the error is still just:
```
No overload matches this call.
The last overload gave the following error.
Argument of type '"@hf/thebloke/neural-chat-7b-v3-1-awq"' is not assignable to parameter of type '"@cf/unum/uform-gen2-qwen-500m" | "@cf/llava-hf/llava-1.5-7b-hf"'.
```
## Bad options
```ts
AI.run('@hf/thebloke/neural-chat-7b-v3-1-awq', {
messages: [
{
role: "system",
content: `[very large system prompt]`
},
{
role: "user",
content: `How many pounds of food does a person eat in a year?`
}
],
stream: false
}, {
extraHeaders: true
});
```
Here I mis-typed `extraHeaders` to be a boolean instead of an object, and my error is still:
```
No overload matches this call.
The last overload gave the following error.
Argument of type '"@hf/thebloke/neural-chat-7b-v3-1-awq"' is not assignable to parameter of type '"@cf/unum/uform-gen2-qwen-500m" | "@cf/llava-hf/llava-1.5-7b-hf"'.
```
---
Essentially, due to the function overload complexity here, it seems that the errors surfaced to users from these types are really not that helpful. People are confused about this regularly in the Discord.
Contributor guide
Assessment
This issue has not been assessed yet.