Umbrella issue -- parallelism improvements
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 195
- Avg merge
- 6d 10h
- Merged PRs (30d)
- 17
Description
All the improvements to parallelism that we want to make post the first release:
- [ ] UI improvements
- [x] Expand parallel apps inline (#421)
- [x] Show success/failure
- [ ] Ensure insights tab shows aggregation of all parallel sub-applications
- [ ] Show the sub-applications graphs in the graph
- [x] Ensure live mode works (#421)
- [x] Ensure the hierarchy of traces is clear (#421)
- [ ] Task creation/management
- [ ] Add API for task IDs
- [ ] Add API
- [ ] Wire through to UI
- [ ] Add documentation
- [ ] Guard-rails around async problems
- [ ] Ensure that generators that are both async and not async work as expected
- [x] Wiring through persisters
- [x] Allow wiring through of persisters to sub-trackers
- [x] Implement this spec: https://burr.dagworks.io/concepts/parallelism/#persistence-tracking -- not currently doable
- [ ] Execution improvements
- [ ] Reevaluate execution API
- [ ] Add guard-rails for executors (E.G. with async, etc...)
- [ ] Allow customization of execution capabilities on a per-task basis
- [x] Ensure recursion works -- multiple levels deep
Contributor guide
Assessment
This issue has not been assessed yet.