dotnet / dotnet/orleans

[WIP] SiloAddress enhancements to support metaclusters and more networking scenarios

Open
#7,517 0 comments 1 reaction 1 assignee View on GitHub

@ReubenBond is already working on this.

Since Jan 28, 2022.

Needs: triage :mag:
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
13h 56m
Merged PRs (30d)
351

Description

I am listing some thoughts here which are half-baked and need more careful consideration - I'll try to come back to them soon.


This is a part of the metaclusters epic, #7485, where we laid out some plans/ideas for adding support for "clusters of clusters" to Orleans. Specifically, this point:

  • SiloAddress should have some way of identifying the cluster which it belongs to. That might include embedding a cluster id or it might involve migrating to a string instead of an IPEndPoint, where the string can map to an IPEndPoint for local communication (either via parsing or a mapping included in the membership table (eg, "ClusterX/SiloY" has endpoints: { IPv4: 10.0.0.2:11111 IPv6: ::::2:11111, FQDN: "silox.clustery.internal.contoso.com"}, or something), or an external

We discussed the following as a team:

  • SiloAddress will contain a stringly address instead of an IPEndPoint
  • We will add public abstractions for mapping from SiloAddress to EndPoint (the abstract base classs), and from EndPoint to IConnectionFactory (Bedrock's abstraction for creating outbound connections)
  • We will change membership providers to include a customizable property bag (string to string map) so that a wider variety of endpoint mappings and placement providers can be implemented.
  • The default implementation of the abovementioned abstractions will query membership to get those properties and produce an endpoint and then connection context.

Here is a sketch of those abstractions:

public interface IConnectionEndPointResolver
{
    bool TryResolveEndPoint(SiloAddress address, out EndPoint endPoint);
}

public interface IConnectionFactoryResolver
{
    bool TryGetConnectionFactory(EndPoint endPoint, out IConnectionFactory connectionFactory);
}

This approach allows us to support different address formats, for example:

  • WebSockets and HTTP2
  • Unix Domain Sockets and Named Pipes
  • In-memory transports for test scenarios

I think having the SiloAddress be an opaque string which is globally unique across all clusters in the service would be ideal as long as it's persistent for a logical silo (like IP:Port is today), but it requires more consideration. Eg, how would such strings be created.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.