kubeslice / kubeslice/worker-operator

Bug: The External LB is getting stuck into "pending" state on AWS & few other clouds when protocol is TCP

Open
#330 12 comments 0 reactions 1 assignee Assigned to @Rajan-226 View on GitHub
bug
Dominant language
Go
Stars
62
Forks
33
Avg merge
3d 10h
Merged PRs (30d)
3

Description

### 📜 Description

while testing the SGE on private clusters, we found some issues with AWS specific annotations that worker-operator is adding.

```
// Note: Special treatment for AWS EKS clusters. The LB is not provisioned unless we add AWS specific annotations
// to the service. This is needed only for EKS.
if clusterProvider, _ := getClusterProviderID(ctx, r.Client); clusterProvider == "aws" {
if svc.ObjectMeta.Annotations == nil {
svc.ObjectMeta.Annotations = make(map[string]string)
}
svc.ObjectMeta.Annotations["service.beta.kubernetes.io/aws-load-balancer-type"] = "external"
svc.ObjectMeta.Annotations["service.beta.kubernetes.io/aws-load-balancer-nlb-target-type"] = "ip"
svc.ObjectMeta.Annotations["service.beta.kubernetes.io/aws-load-balancer-scheme"] = "internet-facing"
}
```

https://github.com/kubeslice/worker-operator/blob/master/controllers/slice/slice_gw_edge.go#L128C1-L137C3
the LB is getting stuck into "pending" state on AWS when protocol is TCP. The issues were resolved simply by removing the annotations.

### 👟 Reproduction steps

while testing the SGE on private clusters, we found some issues with AWS specific annotations that worker-operator is adding.

### 👍 Expected behavior

external lb start running without any issue.

### 👎 Actual Behavior

The External LB is getting stuck into "pending" state.

### 🐚 Relevant log output

_No response_

### Version

1.2.0

### 🖥️ What operating system are you seeing the problem on?

Windows

### ✅ Proposed Solution

We can introduce a new field in the Cluster Custom Resource Definitions (CRDs) for a set of annotations for the External Load Balancer (Ext LB). If any cloud provider needs specific annotations for their Network Load Balancer (Network LB), these can be passed as input during the creation of the Cluster CRD. The Slice Gateway Edge (SGE) will then take these annotations and apply them to the External Load Balancer during its creation.

### 👀 Have you spent some time to check if this issue has been raised before?

- [X] I checked and didn't find any similar issue

### Code of Conduct

- [X] I agree to follow this project's Code of Conduct

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.