dotnet / dotnet/orleans

With UseKubernetesHosting, check pods for the existence of the orleans/serviceId and orleans/clusterId labels

Open
#8,147 0 comments 0 reactions 0 assignees View on GitHub
Needs: triage :mag:
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
14h 42m
Merged PRs (30d)
354

Description

The Kubernetes agent is not able to match pods to silos correctly if the orleans/serviceId and orleans/clusterId labels are missing in the Kubernetes manifest. Therefore the agent will kills pods that it shouldn't be killing.

Specifically The [_podLabelSelector](https://github.com/dotnet/orleans/blob/86e270717bd86f40ec2d42911253ee99be93b9ed/src/Orleans.Hosting.Kubernetes/KubernetesClusterAgent.cs#L50) will not be able to match any Pods:
```
_podLabelSelector = $"{KubernetesHostingOptions.ServiceIdLabel}={_clusterOptions.ServiceId},{KubernetesHostingOptions.ClusterIdLabel}={_clusterOptions.ClusterId}";
```

It could additionally make sense to make sure the label values match the values of the environment variables ORLEANS_SERVICE_ID and ORLEANS_CLUSTER_ID.

Contributor guide

Open the contributing guide

Research direction

Start with src/Orleans.Hosting.Kubernetes/KubernetesClusterAgent.cs, especially the _podLabelSelector construction at the referenced line. Trace how pod labels and the ORLEANS_SERVICE_ID and ORLEANS_CLUSTER_ID environment variables are obtained, then verify that pods missing or disagreeing on the orleans/serviceId and orleans/clusterId labels are not treated as matching silos. Done means the agent no longer kills unrelated pods.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, kubernetes
Domain
devops, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.