HumanSignal / HumanSignal/label-studio-ml-backend

"Use for interactive pre annotations" does not work properly

Open
#270 2 comments 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.1k
Forks
490
Avg merge
1d 14h
Merged PRs (30d)
3

Description

# Problem

After linking a **Question Answering model** and activating the **"Use for interactive preannotations"** button, I expect that when I click on a task, it will run the predictions of the questions that I specified in my label_studio_backend python file. This process happens but in the following way:
1. click on the task where to make the annotations
2. from the label studio backend log I see the predictions of the model
3. the task is loaded without the predictions
4. exit from task
5. enter on the same task and the predictions appear correctly (the query to the model is not re-executed from the log)

I believe that exiting and reentering the task is not the correct procedure to view the predictions of the model.

The same problem occurs performing the prediction of a single Question (I exclude problems relative to multiple predictions of many Questions).

# Screenshots
image

# Environment
I ran the test in local environment (no docker) with the following versions:
- Windows 11
- Python 3.9.13
- label-studio 1.7.3
- label-studio-ml 1.0.8rc3
- label-studio-converter 0.0.51
- label-studio-tools 0.0.2

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the interactive preannotation flow with the listed Label Studio versions, using the label_studio_backend Python file and its backend log. Compare the first task load with reopening the same task, and trace where the Question Answering predictions are returned or displayed. Done means predictions appear immediately when the task is first opened without rerunning the model query.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.