explosion / explosion/spaCy

Unable to fine-tune previously trained transformer based spaCy NER.

Open
#13,709 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
33.9k
Forks
4.7k
Avg merge
3m
Merged PRs (30d)
1

Description

## How to reproduce the behaviour
Use spacy to fine-tune a base model with a transformer from hugging face:
python -m spacy train config.cfg --output ./output --paths.train ./train.spacy --paths.dev ./dev.spacy

Collect new tagged entries under new sets and set your model location to the output/model-last in a new config:
python -m spacy train fine_tune_config.cfg --output ./fine_tune_output --paths.train ./newtrain.spacy --paths.dev ./newdev.spacy

You will get an error about a missing config.json. Even replacing this will then lead to an error of a missing tokenizer.

## Your Environment

* Operating System: Windows 11
- **spaCy version:** 3.7.2
- **Platform:** Linux-5.15.167.4-microsoft-standard-WSL2-x86_64-with-glibc2.35
- **Python version:** 3.10.13

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.