HumanSignal / HumanSignal/label-studio-ml-backend
"Use for interactive pre annotations" does not work properly
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 490
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 3
Description
# Problem
After linking a **Question Answering model** and activating the **"Use for interactive preannotations"** button, I expect that when I click on a task, it will run the predictions of the questions that I specified in my label_studio_backend python file. This process happens but in the following way:
1. click on the task where to make the annotations
2. from the label studio backend log I see the predictions of the model
3. the task is loaded without the predictions
4. exit from task
5. enter on the same task and the predictions appear correctly (the query to the model is not re-executed from the log)
I believe that exiting and reentering the task is not the correct procedure to view the predictions of the model.
The same problem occurs performing the prediction of a single Question (I exclude problems relative to multiple predictions of many Questions).
# Screenshots
# Environment
I ran the test in local environment (no docker) with the following versions:
- Windows 11
- Python 3.9.13
- label-studio 1.7.3
- label-studio-ml 1.0.8rc3
- label-studio-converter 0.0.51
- label-studio-tools 0.0.2
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the interactive preannotation flow with the listed Label Studio versions, using the label_studio_backend Python file and its backend log. Compare the first task load with reopening the same task, and trace where the Question Answering predictions are returned or displayed. Done means predictions appear immediately when the task is first opened without rerunning the model query.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100