huggingface / huggingface/transformers.js

dims undefined when converting own model to ONNX

Open
#584 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
16.3k
Forks
1.2k
Avg merge
6d 2h
Merged PRs (30d)
6

Description

### System Info

```
└─┬ llamaindex@0.1.10
└── @xenova/transformers@2.15.0
```

### Environment/Platform

- [ ] Website/web-app
- [ ] Browser extension
- [X] Server-side (e.g., Node.js, Deno, Bun)
- [ ] Desktop app (e.g., Electron)
- [ ] Other (e.g., VSCode extension)

### Description

After trying to use my own quantized model, I'm getting this error:

```ts
TypeError: Cannot read properties of undefined (reading 'dims')
at mean_pooling (file:///Users/x/Documents/GitHub/llm-newsflow/node_modules/@xenova/transformers/src/utils/tensor.js:729:36)
at Function._call (file:///Users/x/Documents/GitHub/llm-newsflow/node_modules/@xenova/transformers/src/pipelines.js:1194:22)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
```

### Reproduction

First I'm converting an existing model to ONNX

```bash
python -m convert --model_id sentence-transformers/paraphrase-multilingual-mpnet-base-v2
```

Then I'm using the HuggingFace embedding from Llamaindex (although the pipeline is set up by transformers directly):

```ts
import {
HuggingFaceEmbedding
} from "llamaindex";

const MODEL = "sentence-transformers/paraphrase-multilingual-mpnet-base-v2";

//...
const embedModel = new HuggingFaceEmbedding({
modelType: MODEL,
getExtractor: async () => {
const { pipeline } = await import("@xenova/transformers");
const p = await pipeline("feature-extraction", MODEL, {
quantized: false,
local_files_only: true,
cache_dir: './models',
});

return p;
}
});
```

The HuggingFaceEmbedding class is just [a thin wrapper around transformers](https://github.com/run-llama/LlamaIndexTS/blob/main/packages/core/src/embeddings/HuggingFaceEmbedding.ts).

The above code works fine with existing models on HF.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.