huggingface / huggingface/transformers.js

[Question] How to convert to ONNX a fine-tuned model

Open
#274 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
JavaScript
Stars
16.3k
Forks
1.2k
Avg merge
6d 2h
Merged PRs (30d)
6

Description

Hi, we're playing with this library to see if it can be useful for our project. I find it very easy and well done (congratulations).

The idea is not to use it directly as a frontend library but via node.js.
We've tried scripting a model directly from HF (google/flan-t5-small) and it worked but we're having trouble using a fine-tuned model.

Here what we tried. We fine-tuned a model (again google/flan-t5-small) and then converted it using the onnx script (in README.md).

The script generated the following files:

```
onnx/decoder_model_quantized.onnx
onnx/decoder_model.onnx
onnx/encoder_model_quantized.onnx
onnx/encoder_model.onnx
config.json
generation_config.json
quantize_config.json
special_tokens_map.json
spice.model
tokenizer_config.json
tokenizer.json
```

But when we tried to use it it gave us this error:

`local_files_only=true` or `env.allowRemoteModels=false` and file was not found locally at ./models/google/flan-t5-small-2/onnx/decoder_model_merged_quantized.onnx

Some advice or useful doc/link?
Thanks

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.