huggingface / huggingface/setfit

exception:enum PyPreTokenizerTypeWrapper, while loading the fine-tuned model for evaluation

Open
#520 7 comments 3 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

Hi - currently we are fine-tuning the model: "paraphrase-multilingual-MiniLM-L12-v2" for our use case. In our pipeline, we have a model validation part where we are loading the trained model with:

model = SetFitModel.from_pretrained(model_dir)

but unfortunately, we are getting the below exception: -

Exception: data did not match any variant of untagged enum PyPreTokenizerTypeWrapper at line 83 column 3.

Note: I am using amazon Sagemaker platform for finetuning with below configuration:

for traning:
instance_type: "ml.g5.2xlarge"
instance_count: 1
transformers_version: "4.28.1"
pytorch_version: "2.0.0"
setfit_version: "0.7.0"
py_version: "py310"

for validation:
instance_type: "ml.t3.xlarge"
instance_count: 1

it was working fine with the above configuration but since last couple of days we are getting the above-mentioned exception. So, it would be great if anyone can help us out to fix the issue.

Do let me know if any other information is required from our side.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.