facebookresearch / facebookresearch/fairo
[Dashboard in Turk] Next Steps in end-to-end Dashboard Pipeline
- Dominant language
- Jupyter Notebook
- Stars
- 929
- Forks
- 123
- PR merge metrics
- No merged PRs in 30d
Description
## Type of Issue
Select the type of issue:
- [ ] Bug report (to report a bug)
- [ ] Feature request (to request an additional feature)
- [x] Tracker (I am just using this as a tracker)
- [ ] Refactor request
- [ ] Documentation Ask
## Description
Tracking the remaining components that need to be built to achieve the end-to-end Turk interaction to model retraining pipeline necessary for our vision for automated self improvement.
Annotation Tools: #447
- Streamline annotation tools spinup
- Update the surveys
- Connect task spinup to the main pipeline
Dashboard in Turk #448
- Set up qualifications tasks
- A/B test job completion to ensure errors are marked
Retrain models #449
- Collect annotations and arrange training data
- create visualization showing log linear improvement
## Checklist
Use this checklist if you are using this issue as a tracker:
- [ ] Get Turkers to mark errors
- [ ] Annotations pipeline
- [ ] Retrain models offline
Contributor guide
Assessment
This issue has not been assessed yet.