tensorflow / tensorflow/probability
Distributing TFP
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 4.4k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
Is it possible to implement data parallelism in an example such as
https://github.com/tensorflow/probability/blob/master/tensorflow_probability/examples/bayesian_neural_network.py
?
Recently I simply edited this with Horovod and it runs although I am not convinced it is averaging gradients.
Does the max ELBO calculation need to be serial?
Is there anything in the use of the distributions which prevents adata parallel run?
Thanks in advance for your help!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The request points to tensorflow_probability/examples/bayesian_neural_network.py and asks about data parallelism with Horovod. Start by reading that example and examining the proposed gradient averaging, max ELBO calculation, and distribution usage. Done would require a clear implementation or documented determination of whether the example supports data-parallel execution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100