Cross platform issues with Remote Workers / SSH Cluster Manager / Native Dependecies

Hi -- Using a head node (i.e. procid == 1) that is a mac on v0.3.6, I am trying to use linux based workers using SSHClusterManager.

I experience problems with e.g. using HDF5 --- the basic cause seems to be that

• workers delegate include to node 1 (include == include_from_node1)
• HDF5 (and many others) use BinDeps
• BinDeps creates a custom deps.jl which is platform (and presumably even box) specific

so my linux boxes complain when the cannot locate the mac dylib

I've thought a bit about how to resolve this, but nothing obvious and elegant pops to mind. (For now I've just hacked my deps.jl on my mac to support both OS X and linux)

Have others seen this kind of issue? Is there some simple way to have the workers not pull code from node1 but simply rely on the locally installed packages?

I was thinking of hacking include_from_node1 in the .juliarc.jl on the linux boxes to simply not pull code from node1, but that seems a bit drastic -- any thoughts about whether this would work?

As an aside, while I can understand the motivation for include, using etc to work by delegating to node1 (e.g, simplify the need for code distribution), it does seems a bit difficult to do robustly or in a way that will scale nicely to dozens or hundreds of workers....

thanks

Start by reading the SSHClusterManager behavior and the include_from_node1 path referenced in the issue, along with the .juliarc.jl configuration. Reproduce the setup with a macOS head node and Linux workers using HDF5, then determine how platform-specific dependencies are distributed. Done means establishing a supported behavior or a clearly defined fix for local worker dependencies.