kubernetes / kubernetes/kops

Request to allow specifying extra routes to add to kops subnets to point to pre-existing VPN gateway

Open
#2,214 38 comments 24 reactions 0 assignees View on GitHub
area/networking Feature Request lifecycle/frozen
Dominant language
Go
Stars
16.7k
Forks
4.7k
Avg merge
17h 21m
Merged PRs (30d)
68

Description

We have a pre-existing VPC in AWS with a VPN connection between our company's network and the VPC. I have been able to successfully create a completely private Kubernetes cluster with kops by ensuring that I:

- set the api load balancer type to `Internal` in the kops cluster configuration
- set the dns, masters, and nodes topology to `private` in the kops cluster configuration (via flags to `kops create cluster`)
- do not pass the `--bastion` flag to `kops create cluster` so a bastion is not created

I found in the [run_in_existing_vpc document](https://github.com/kubernetes/kops/blob/master/docs/run_in_existing_vpc.md) that I can use pre-existing subnets in the cluster configuration. I gave this a try. The route tables for the subnets I made have routes that point to our VPN gateway. The kops utility subnets share a route table that points to an Internet gateway for the default route, and the private subnets each have a route table that points to a NAT gateway for the default route. This all appears to have worked well. I have been able to create `LoadBalancer` type Kubernetes services configured so that they create private load balancers, and I can access the services via the VPN. (Note I did make sure to add the subnet tags mentioned by [this issue](https://github.com/kubernetes/kops/issues/2011) to ensure the load balancer ends up in one of the private subnets).

Rather than creating the Kubernetes resources in our pre-existing subnets, we would prefer to keep the kops cluster separated in its own dedicated subnets that are ideally managed by kops. It would be nice if kops could create the subnets and their corresponding route tables with the VPN gateway routes added. The [run_in_existing_vpc document](https://github.com/kubernetes/kops/blob/master/docs/run_in_existing_vpc.md) shows how to have kops create the subnets for you but use pre-existing NAT gateways by specifying an `egress` setting on the private subnets. I gave this a try, and kops created all the subnets for me in our existing VPC and pointed the route tables to the NAT gateways I specified. I then manually added the VPN gateway routes to both the utility and private subnet route tables that kops created (and the tags mentioned in [this issue](https://github.com/kubernetes/kops/issues/2011)) and found that all seems to work fine. Without the VPN gateway routes, I can't communicate with the masters or nodes over the VPN because the networking on the AWS side doesn't know how to route the response back to me.

So, in short, this is a request to add the ability to specify one or more extra routes to add to the subnets created by kops to point them to a pre-existing VPN gateway (while simultaneously allowing for specifying a pre-existing NAT gateway as you can today).

Contributor guide

Open the contributing guide

Research direction

Read docs/run_in_existing_vpc.md and the existing cluster configuration for private subnets and egress settings. Trace how kops creates subnet route tables and applies pre-existing NAT gateway routes. Done means users can specify additional routes targeting a pre-existing VPN gateway while creating dedicated kops-managed subnets, with the resulting utility and private route tables containing those routes.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
cloud, infrastructure, networking
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.