HumanSignal / HumanSignal/label-studio

JSON export is missing the predictions data

Open
#5,654 1 comment 5 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
28.3k
Forks
3.7k
Avg merge
14h
Merged PRs (30d)
15

Description

**Describe the bug**
For each task, the JSON export "predictions" key has an array containing only the id of the predictions, but not the data from the remaining predictions keys. I'm having the same issue with a classification and an NER project. I'm not sure if this is a bug, or whether I don't understand the intended functionality.

**To Reproduce**
Steps to reproduce the behavior:
1. Create pre-annotations in a notebook and export to JSON in this format:

{
"data": {
"email": "some email text",
"MNE": "an MNE",
"meta_info": null
},
"predictions": [
{
"model_version": "gpt-4",
"result": [
{
"from_name": "class",
"to_name": "email",
"type": "choices",
"readonly": false,
"hidden": false,
"value": {
"choices": [
"class 1"
]
}
}
],
"score": null
}
]
}

My labeling interface is:







...

The JSON loads in the UI and then I annotate the data. The task source for a single task now looks like this:

{
"id": 7,
"data": {
"email": "some email text",
"MNE": "an MNE",
"meta_info": null
},
"annotations": [
{
"id": 18,
"created_username": " my email, 1",
"created_ago": "28 minutes",
"completed_by": {
"id": 1,
"first_name": "",
"last_name": "",
"avatar": null,
"email": "my email",
"initials": "ra"
},
"result": [
{
"value": {
"choices": [
"class 1"
]
},
"id": "kgEj1EWWhq",
"from_name": "class",
"to_name": "email",
"type": "choices",
"origin": "prediction"
}
],
"was_cancelled": false,
"ground_truth": false,
"created_at": "2024-04-01T03:28:19.703650Z",
"updated_at": "2024-04-01T03:45:19.865863Z",
"draft_created_at": null,
"lead_time": 6.824,
"import_id": null,
"last_action": null,
"task": 7,
"project": 1,
"updated_by": 1,
"parent_prediction": 10,
"parent_annotation": null,
"last_created_by": null
}
],
"predictions": [
{
"id": 10,
"model_version": "gpt-4",
"created_ago": "20 hours, 26 minutes",
"result": [
{
"from_name": "class",
"to_name": "email",
"type": "choices",
"readonly": false,
"hidden": false,
"value": {
"choices": [
"class 1"
]
}
}
],
"score": null,
"cluster": null,
"neighbors": null,
"mislabeling": 0,
"created_at": "2024-03-31T07:30:24.871982Z",
"updated_at": "2024-03-31T07:30:24.872022Z",
"task": 7,
"project": 1
}
]
}

2. Click Export with the JSON option

The exported JSON looks like this:

[
{
"id": 7,
"annotations": [
{
"id": 18,
"completed_by": 1,
"result": [
{
"value": {
"choices": [
"class 1"
]
},
"id": "kgEj1EWWhq",
"from_name": "class",
"to_name": "email",
"type": "choices",
"origin": "prediction"
}
],
"was_cancelled": false,
"ground_truth": false,
"created_at": "2024-04-01T03:28:19.703650Z",
"updated_at": "2024-04-01T03:45:19.865863Z",
"draft_created_at": null,
"lead_time": 6.824,
"prediction": {},
"result_count": 0,
"unique_id": "13f68934-4905-462e-9486-5c744affdfc1",
"import_id": null,
"last_action": null,
"task": 7,
"project": 1,
"updated_by": 1,
"parent_prediction": 10,
"parent_annotation": null,
"last_created_by": null
}
],
"file_upload": "ec6ee996-test3.json",
"drafts": [],
"predictions": [
10
],
"data": {
"email": "some email text",
"MNE": "an MNE",
"meta_info": null
},
"meta": {},
"created_at": "2024-03-31T07:30:24.868032Z",
"updated_at": "2024-04-01T03:45:19.912165Z",
"inner_id": 1,
"total_annotations": 1,
"cancelled_annotations": 0,
"total_predictions": 1,
"comment_count": 0,
"unresolved_comment_count": 0,
"last_comment_updated_at": null,
"project": 1,
"updated_by": 1,
"comment_authors": []
},
...
]

**Expected behavior**
I expected the export JSON to include the pre-annotations under the "predictions" keys.

**Environment (please complete the following information):**
- OS: iOS
- Label Studio Version: 1.11.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.