[WIP] SiloAddress enhancements to support metaclusters and more networking scenarios
@ReubenBond is already working on this.
Since Jan 28, 2022.
- Dominant language
- C#
- Stars
- 10.9k
- Forks
- 2.1k
- Avg merge
- 13h 56m
- Merged PRs (30d)
- 351
Description
I am listing some thoughts here which are half-baked and need more careful consideration - I'll try to come back to them soon.
This is a part of the metaclusters epic, #7485, where we laid out some plans/ideas for adding support for "clusters of clusters" to Orleans. Specifically, this point:
- SiloAddress should have some way of identifying the cluster which it belongs to. That might include embedding a cluster id or it might involve migrating to a string instead of an IPEndPoint, where the string can map to an IPEndPoint for local communication (either via parsing or a mapping included in the membership table (eg, "ClusterX/SiloY" has endpoints: { IPv4: 10.0.0.2:11111 IPv6: ::::2:11111, FQDN: "silox.clustery.internal.contoso.com"}, or something), or an external
We discussed the following as a team:
SiloAddresswill contain a stringly address instead of anIPEndPoint- We will add public abstractions for mapping from
SiloAddresstoEndPoint(the abstract base classs), and fromEndPointtoIConnectionFactory(Bedrock's abstraction for creating outbound connections) - We will change membership providers to include a customizable property bag (string to string map) so that a wider variety of endpoint mappings and placement providers can be implemented.
- The default implementation of the abovementioned abstractions will query membership to get those properties and produce an endpoint and then connection context.
Here is a sketch of those abstractions:
public interface IConnectionEndPointResolver
{
bool TryResolveEndPoint(SiloAddress address, out EndPoint endPoint);
}
public interface IConnectionFactoryResolver
{
bool TryGetConnectionFactory(EndPoint endPoint, out IConnectionFactory connectionFactory);
}
This approach allows us to support different address formats, for example:
- WebSockets and HTTP2
- Unix Domain Sockets and Named Pipes
- In-memory transports for test scenarios
I think having the SiloAddress be an opaque string which is globally unique across all clusters in the service would be ideal as long as it's persistent for a logical silo (like IP:Port is today), but it requires more consideration. Eg, how would such strings be created.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.