mars-project / mars-project/mars
Integrate TensorFlow more seamlessly
- Dominant language
- Python
- Stars
- 2.7k
- Forks
- 325
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
I think basically, we have two things to do to make integrating TensorFlow more seamlessly.
1. Support start TensorFlow cluster in Mars cluster. API like:
```
from mars.learn.contrib.tensorflow import start_tf_cluster
start_tf_cluster(n_worker=3, n_ps=2)
```
The training of TensorFlow can be deployed to the cluster created by Mars.
2. Support dataset from Mars. We could implement sort of `MarsDataSet` by leveraging TensorFlow Dataset API so that the data from Mars tensor or DataFrame could be feed into TensorFlow for training.
Contributor guide
Research direction
No files or tests are named. Start by tracing how Mars creates and manages its cluster, then review the TensorFlow Dataset API and Mars tensor/DataFrame interfaces. Done means defining and implementing both the start_tf_cluster API and a MarsDataSet path for TensorFlow training, with tests for each integration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100