HumanSignal / HumanSignal/label-studio-ml-backend
NER ml backend example breaks during learning process
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 490
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 3
Description
For starters apologies for throwing two issues in just one day. I've much appreciated working with transformer branch of your project last year without considerable problems. Move to core ml-backend branch in which all transformer backends were already incorporated since that time appears to be challenging.
Currently i'm unable to finish any learning NER process with following error thrown each time:
File "/Users/user/Projects/label-studio-ml-backend/label_studio_ml/exceptions.py", line 39, in exception_f
return f(*args, **kwargs)
```
File "/Users/user/Projects/label-studio-ml-backend/label_studio_ml/api.py", line 70, in _train
job = _manager.train(annotations, project, label_config, **params)
File "/Users/user/Projects/label-studio-ml-backend/label_studio_ml/model.py", line 667, in train
cls.get_or_create(project, label_config, force_reload=True, train_output=train_output)
File "/Users/user/Projects/label-studio-ml-backend/label_studio_ml/model.py", line 471, in get_or_create
if not cls.has_active_model(project) or force_reload or (cls.get(project).model_version != version and version is not None): # noqa
File "/Users/user/Projects/label-studio-ml-backend/label_studio_ml/model.py", line 447, in has_active_model
return cls._key(project) in cls._current_model
TypeError: argument of type 'ModelWrapper' is not iterable
```
Steps to recreate:
0. Set up NER backend with Label Studio following guidelines from the repo (ner folder requirements.txt on top of it)
1. Load Dataset with any text attribute inside and set it up as NER ml template.
2. Annotate some examples and start the learning process (initiated from Label Studio frontend in my case)
It starts to learn but throws above error after some time.
Found this resolution for similar error here ([link](https://githubhot.com/repo/heartexlabs/label-studio-ml-backend/issues/77)) but unfortunately doesn't work in my case.
I'm currently using Label Studio 1.4 and the most recent version of ML backend.
Update: Also checked with 1.4.1post1 and there it doesn't even start learning process. It just throws:
`ValueError: num_samples should be a positive integer value, but got num_samples=0` which seems like indicating that there is no annotations provided.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the NER training flow using the setup described in the issue and inspect the traceback through label_studio_ml/api.py, model.py, and exceptions.py, especially the reported training and model-loading entry points. Compare behavior on Label Studio 1.4 and 1.4.1post1; done means the annotated dataset begins and completes training without the reported TypeError or zero-sample failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100