huggingface / huggingface/transformers.js
dims undefined when converting own model to ONNX
- Dominant language
- JavaScript
- Stars
- 16.3k
- Forks
- 1.2k
- Avg merge
- 6d 2h
- Merged PRs (30d)
- 6
Description
### System Info
```
└─┬ llamaindex@0.1.10
└── @xenova/transformers@2.15.0
```
### Environment/Platform
- [ ] Website/web-app
- [ ] Browser extension
- [X] Server-side (e.g., Node.js, Deno, Bun)
- [ ] Desktop app (e.g., Electron)
- [ ] Other (e.g., VSCode extension)
### Description
After trying to use my own quantized model, I'm getting this error:
```ts
TypeError: Cannot read properties of undefined (reading 'dims')
at mean_pooling (file:///Users/x/Documents/GitHub/llm-newsflow/node_modules/@xenova/transformers/src/utils/tensor.js:729:36)
at Function._call (file:///Users/x/Documents/GitHub/llm-newsflow/node_modules/@xenova/transformers/src/pipelines.js:1194:22)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
```
### Reproduction
First I'm converting an existing model to ONNX
```bash
python -m convert --model_id sentence-transformers/paraphrase-multilingual-mpnet-base-v2
```
Then I'm using the HuggingFace embedding from Llamaindex (although the pipeline is set up by transformers directly):
```ts
import {
HuggingFaceEmbedding
} from "llamaindex";
const MODEL = "sentence-transformers/paraphrase-multilingual-mpnet-base-v2";
//...
const embedModel = new HuggingFaceEmbedding({
modelType: MODEL,
getExtractor: async () => {
const { pipeline } = await import("@xenova/transformers");
const p = await pipeline("feature-extraction", MODEL, {
quantized: false,
local_files_only: true,
cache_dir: './models',
});
return p;
}
});
```
The HuggingFaceEmbedding class is just [a thin wrapper around transformers](https://github.com/run-llama/LlamaIndexTS/blob/main/packages/core/src/embeddings/HuggingFaceEmbedding.ts).
The above code works fine with existing models on HF.
Contributor guide
Assessment
This issue has not been assessed yet.