facebookresearch / facebookresearch/fairo

[Dashboard in Turk] Next Steps in end-to-end Dashboard Pipeline

Open
#446 0 comments 0 reactions 3 assignees Claimed by @snyxan View on GitHub
dashboard tooling
Dominant language
Jupyter Notebook
Stars
929
Forks
123
PR merge metrics
No merged PRs in 30d

Description

## Type of Issue

Select the type of issue:
- [ ] Bug report (to report a bug)
- [ ] Feature request (to request an additional feature)
- [x] Tracker (I am just using this as a tracker)
- [ ] Refactor request
- [ ] Documentation Ask

## Description

Tracking the remaining components that need to be built to achieve the end-to-end Turk interaction to model retraining pipeline necessary for our vision for automated self improvement.

Annotation Tools: #447
- Streamline annotation tools spinup
- Update the surveys
- Connect task spinup to the main pipeline

Dashboard in Turk #448
- Set up qualifications tasks
- A/B test job completion to ensure errors are marked

Retrain models #449
- Collect annotations and arrange training data
- create visualization showing log linear improvement

## Checklist

Use this checklist if you are using this issue as a tracker:

- [ ] Get Turkers to mark errors
- [ ] Annotations pipeline
- [ ] Retrain models offline

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.