ps-lite

Hi all,

I am not sure if I understand the communication part in ps. The dataflow approach is to pull out the model from ps and training it on local data, then finally push the update to the ps.

My doubt about when the worker push the update back to ps. Is it after each iteration ? What is Iteration ? My understanding of iteration is feedForward then propagate backward. This is one epoch.

If I am write, we will have communication overhead. also something the communication cost more than computation cost.

Sincerely