Lightning-AI / Lightning-AI/litData
Integration with DAG framework Prefect
Open
enhancement
help wanted
- Dominant language
- Python
- Stars
- 614
- Forks
- 106
- Avg merge
- 15h 8m
- Merged PRs (30d)
- 22
Description
## 🚀 Feature
### Motivation
The library Prefect https://github.com/PrefectHQ/prefect is pretty great at building DAGs and orchestrating then.
In the future, LitData will enables to build a DAG of map, optimize, etc.. operators to perform large distributed processing.
The ideal goal would be the reverse engineer Prefect Orion Engine and monitoring and have an adapted version for the Lightning.AI platform.
### Pitch
### Alternatives
### Additional context
Contributor guide
Assessment
This issue has not been assessed yet.