Azure / Azure/AKS

[Question] Is session affinity available for gRPC services publicly reachable via Application Gateway for Containers?

Open
#5,432 1 comment 0 reactions 2 assignees Claimed by @therealmitchconnors View on GitHub
action-required app-gateway-for-containers question
Dominant language
TypeScript
Stars
2.1k
Forks
395
Avg merge
2d 22h
Merged PRs (30d)
13

Description

**Describe scenario**
Hi.

Our C# .NET Framework-based (4.8) Windows service is using Grpc.Core v2.46.6 (latest) and is deployed on AKS v1.30.12.
Our gRPC Windows service is publicly exposed by using Application Gateway for Containers. I deployed v1.6.7 of the Helm ALB controller chart to AKS.
The AGC configuration is simple and consists of a HTTPS listener, protected with our TLS certificate hosted on AKS, that redirects the traffic to our pod via port 9090. Gateway and GRPCRoute are working fine. TLS mode is set to "Terminate". The backend is a k8s service acting as entrypoint for our service.

```
-- GATEWAY

{{- if .Values.gateway.enabled }}
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: {{ include "service.fullname" . }}-gateway
namespace: {{ .Values.namespace }}
annotations:
alb.networking.azure.io/alb-id: {{ .Values.azure.albId | default "agfc-main" }}
{{- with .Values.gateway.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "service.labels" . | nindent 4 }}
{{- with .Values.gateway.labels }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
gatewayClassName: {{ .Values.gateway.className | default "azure-alb-external" }}
listeners:
{{- if .Values.gateway.http.enabled }}
- name: {{ .Values.gateway.http.name | default "http-listener" }}
port: {{ .Values.gateway.http.port | default 80 }}
protocol: HTTP
allowedRoutes:
namespaces:
from: {{ .Values.gateway.http.allowedRoutes.from | default "Same" }}
{{- end }}
{{- if .Values.gateway.https.enabled }}
- name: {{ .Values.gateway.https.name | default "https-listener" }}
port: {{ .Values.gateway.https.port | default 443 }}
protocol: HTTPS
allowedRoutes:
namespaces:
from: {{ .Values.gateway.https.allowedRoutes.from | default "Same" }}
tls:
mode: {{ .Values.gateway.https.tls.mode | default "Terminate" }}
certificateRefs:
- kind: {{ .Values.gateway.https.tls.certificateRef.kind | default "Secret" }}
group: {{ .Values.gateway.https.tls.certificateRef.group | default "" }}
name: {{ .Values.gateway.https.tls.certificateRef.name | required "gateway.https.tls.certificateRef.name is required when HTTPS is enabled" }}
{{- end }}
addresses:
- type: {{ .type | default "alb.networking.azure.io/alb-frontend" }}
value: {{ .Values.azure.albFrontendName | default "my-frontend-fqdn" }}
{{- range .Values.gateway.addresses }}
- type: {{ .type | default "Hostname" }}
value: {{ .value | default "my-hostname" }}
{{- end }}
{{- end }}

-- GRPCRoute

{{- if .Values.alb.enabled }}
apiVersion: gateway.networking.k8s.io/v1
kind: GRPCRoute
metadata:
name: {{ include "service.fullname" . }}-grpc-route
namespace: {{ .Values.namespace }}
labels:
{{- include "service.labels" . | nindent 4 }}
spec:
parentRefs:
- name: {{ include "service.fullname" . }}-gateway
namespace: {{ .Values.namespace }}
kind: Gateway
{{- if .Values.alb.hostnames }}
hostnames:
{{- range .Values.alb.hostnames }}
- {{ . | quote }}
{{- end }}
{{- end }}
rules:
- matches:
backendRefs:
- name: {{ include "service.fullname" . }}
port: {{ .Values.service.port }}
namespace: {{ .Values.namespace }}
weight: {{ .weight | default 100 }}
{{- end }}

-- RoutePolicy

{{- if .Values.alb.enabled }}
apiVersion: alb.networking.azure.io/v1
kind: RoutePolicy
metadata:
name: {{ include "service.fullname" . }}-grpc-route-policy
namespace: {{ .Values.namespace }}
spec:
targetRef:
kind: GRPCRoute
name: {{ include "service.fullname" . }}-grpc-route
namespace: {{ .Values.namespace }}
group: gateway.networking.k8s.io
default:
timeouts:
routeTimeout: {{ .Values.alb.routePolicy.timeout | default "0s" }}
{{- end }}
```

Our gRPC service consists of some unary calls and 2 streams. One is a client stream, where the client sends a stream of messages and the server keeps reading the stream until the client completes it, while the other is a server stream, where the client sends a request and the server responds by sending a stream of messages. Here's a sample of the .proto file:

```
syntax = "proto3";

package Test.Grpc.Clients;

service MyService {
rpc GetEventStream (GetEventStreamRequest) returns (stream GetEventStreamResponse); --> server stream
...
rpc RecordingStreamWrite (stream RecordingStreamWriteRequest) returns (RecordingStreamWriteResponse); --> client stream
}

message GetEventStreamRequest {
}
message GetEventStreamResponse {
string eventJson = 1;
}

message RecordingStreamWriteRequest {
bytes buffer = 1;
}
message RecordingStreamWriteResponse {
}

```

**Question**
Our gRPC service isn't stateless and, like I said above, requests are redirected to the k8s service in front of our service instances.

When we have more than 1 replica of our gRPC service available, the requests are randomly picked up by any replica causing problems to our clients. We tried setting up session affinity in the GRPCRoute, like described here (https://learn.microsoft.com/en-us/azure/application-gateway/for-containers/session-affinity?tabs=session-affinity-gateway-api), but k8s does not accept the route policy, returning us this error, with either a managed cookie or application cookie:

_sessionAffinity not support on RoutePolicy with a GRPCRoute TargetRef_

The documentation shows an example with HTTPRoute, but it does not say whether session affinity is also supported for GRPCRoute.

My question is: Is session affinity available for gRPC services publicly reachable via Application Gateway for Containers?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.