aws / aws/aws-app-mesh-examples

Grpc Load Balancing is not working

Open
#569 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Shell
Stars
868
Forks
384
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
GRPC Load Balancing is not working for several pods in one service. VirtualRouter is sending all requests to only one pod.

**Platform**
EKS Fargate

**To Reproduce**
Create two services (let's call them A and B), with one service (A) receiving traffic from the outside world, and invoking GRPC method on the other service (B). Create multiple pods for service B.

**Expected behavior**
Service (A) make GRPC requests to different instances of service (B). Round-robin strategy...

**AppMesh YAML confurigations**

```html
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualNode
metadata:
name: service-B-vn
namespace: app
spec:
podSelector:
matchLabels:
app: service-B
listeners:
- portMapping:
port: 6565
protocol: grpc
healthCheck:
port: 6565
protocol: grpc
healthyThreshold: 2
unhealthyThreshold: 3
timeoutMillis: 2000
intervalMillis: 5000
timeout:
grpc:
idle:
unit: s
value: 0
perRequest:
unit: s
value: 0
serviceDiscovery:
awsCloudMap:
namespaceName: app
serviceName: service-B-vs

---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualService
metadata:
name: service-B-vs
namespace: app
spec:
awsName: service-B.app.svc.cluster.local
provider:
virtualRouter:
virtualRouterRef:
name: service-B-vr

---
apiVersion: appmesh.k8s.aws/v1beta2
kind: VirtualRouter
metadata:
namespace: app
name: service-B-vr
spec:
listeners:
- portMapping:
port: 6565
protocol: grpc
routes:
- name: service-B-vr
grpcRoute:
match:
prefix: /
action:
weightedTargets:
- virtualNodeRef:
name: service-B-vn
weight: 1`
```

**Environment**

- App Mesh controller version:
840364872350.dkr.ecr.us-west-2.amazonaws.com/amazon/appmesh-controller:v1.6.0
- Sidecar Envoy version:
840364872350.dkr.ecr.us-west-2.amazonaws.com/aws-appmesh-envoy:v1.22.2.1-prod
- Kubernetes version: 1.23
- Using EKS Fargate

Contributor guide

Open the contributing guide

Research direction

No repository files or tests are identified; start by reproducing the two-service setup on EKS Fargate using the provided VirtualNode, VirtualService, and VirtualRouter YAML. Check whether requests from service A are distributed across service B's pods, and consider the issue resolved when the documented round-robin behavior works consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, grpc, kubernetes
Domain
cloud, distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.