denoland / denoland/deploy_feedback

[Bug]: Transformers.js and onnx runtimes do not work in Deno Deploy

Open
#762 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
79
Forks
5
PR merge metrics
No merged PRs in 30d

Description

### Problem description

I am trying to get an AI model based on transformers.js (which uses the onnx runtime) working in deno. Transformers.js [added support for Deno in v3](https://github.com/huggingface/transformers.js/pull/545). This works great for deno running locally, but does not appear to work in Deno Deploy. When deployed, it gives the following error:

```
{"type":"error","code":"deploymentFailed","ctx":"The deployment failed: UNCAUGHT_EXCEPTION\n\nError: This API is not supported in this environment\n at Object.Module._extensions..node (node:module:790:21)\n at Module.load (node:module:655:32)\n at Function.Module._load (node:module:523:13)\n at Module.require (node:module:674:19)\n at require (node:module:801:16)\n at Object. (file:///node_modules/.deno/onnxruntime-node@1.20.1/node_modules/onnxruntime-node/dist/binding.js:9:1)\n at Object. (file:///node_modules/.deno/onnxruntime-node@1.20.1/node_modules/onnxruntime-node/dist/binding.js:11:4)\n at Module._compile (node:module:736:34)\n at Object.Module._extensions..js (node:module:757:11)\n at Module.load (node:module:655:32)"}
```

### Steps to reproduce

Sample code to reproduce:

```
const quantized = false; // change to `true` for a much smaller model (e.g. 87mb vs 345mb for image model), but lower accuracy
import {
AutoProcessor,
CLIPVisionModelWithProjection,
RawImage,
AutoTokenizer,
CLIPTextModelWithProjection,
} from "npm:@huggingface/transformers";

const imageProcessor = await AutoProcessor.from_pretrained(
"Xenova/clip-vit-base-patch16"
);
const visionModel = await CLIPVisionModelWithProjection.from_pretrained(
"Xenova/clip-vit-base-patch16",
{ quantized }
);
const tokenizer = await AutoTokenizer.from_pretrained(
"Xenova/clip-vit-base-patch16"
);
const textModel = await CLIPTextModelWithProjection.from_pretrained(
"Xenova/clip-vit-base-patch16",
{ quantized }
);

function cosineSimilarity(A: number[], B: number[]) {
if (A.length !== B.length) throw new Error("A.length !== B.length");
let dotProduct = 0,
mA = 0,
mB = 0;
for (let i = 0; i < A.length; i++) {
dotProduct += A[i] * B[i];
mA += A[i] * A[i];
mB += B[i] * B[i];
}
mA = Math.sqrt(mA);
mB = Math.sqrt(mB);
const similarity = dotProduct / (mA * mB);
return similarity;
}

export async function getImageEmbedding(imageLocation: string) {
const image = await RawImage.read(imageLocation);
const imageInputs = await imageProcessor(image);
const { image_embeds } = await visionModel(imageInputs);
// console.log(image_embeds.data);
return image_embeds.data;
}

Deno.serve(async () => {
const embed = await getImageEmbedding("./image.png");
console.log(embed);
console.log(embed.length);
return new Response(JSON.stringify(embed));
});
```
Note: this code does have some typescript and lint errors, but it runs fine on my local machine. I usually focus on getting the code functional before messing with type/lint errors that don't appear to cause issues.

1. Create a deno project with this code
2. Deploy using deployctl
3. Observe error in deployctl command or in Deploy dashboard

### Expected behavior

Since this code runs fine without error in local Deno, I would expect it to run in Deno Deploy without error.

### Environment

_No response_

### Possible solution

_No response_

### Additional context

A similar error can be reproduced using onnx JS libraries directly and bypassing transformers.js. In that test, I was loading the model from a local file.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.