Request to allow specifying extra routes to add to kops subnets to point to pre-existing VPN gateway
- Dominant language
- Go
- Stars
- 16.7k
- Forks
- 4.7k
- Avg merge
- 17h 21m
- Merged PRs (30d)
- 68
Description
We have a pre-existing VPC in AWS with a VPN connection between our company's network and the VPC. I have been able to successfully create a completely private Kubernetes cluster with kops by ensuring that I:
- set the api load balancer type to `Internal` in the kops cluster configuration
- set the dns, masters, and nodes topology to `private` in the kops cluster configuration (via flags to `kops create cluster`)
- do not pass the `--bastion` flag to `kops create cluster` so a bastion is not created
I found in the [run_in_existing_vpc document](https://github.com/kubernetes/kops/blob/master/docs/run_in_existing_vpc.md) that I can use pre-existing subnets in the cluster configuration. I gave this a try. The route tables for the subnets I made have routes that point to our VPN gateway. The kops utility subnets share a route table that points to an Internet gateway for the default route, and the private subnets each have a route table that points to a NAT gateway for the default route. This all appears to have worked well. I have been able to create `LoadBalancer` type Kubernetes services configured so that they create private load balancers, and I can access the services via the VPN. (Note I did make sure to add the subnet tags mentioned by [this issue](https://github.com/kubernetes/kops/issues/2011) to ensure the load balancer ends up in one of the private subnets).
Rather than creating the Kubernetes resources in our pre-existing subnets, we would prefer to keep the kops cluster separated in its own dedicated subnets that are ideally managed by kops. It would be nice if kops could create the subnets and their corresponding route tables with the VPN gateway routes added. The [run_in_existing_vpc document](https://github.com/kubernetes/kops/blob/master/docs/run_in_existing_vpc.md) shows how to have kops create the subnets for you but use pre-existing NAT gateways by specifying an `egress` setting on the private subnets. I gave this a try, and kops created all the subnets for me in our existing VPC and pointed the route tables to the NAT gateways I specified. I then manually added the VPN gateway routes to both the utility and private subnet route tables that kops created (and the tags mentioned in [this issue](https://github.com/kubernetes/kops/issues/2011)) and found that all seems to work fine. Without the VPN gateway routes, I can't communicate with the masters or nodes over the VPN because the networking on the AWS side doesn't know how to route the response back to me.
So, in short, this is a request to add the ability to specify one or more extra routes to add to the subnets created by kops to point them to a pre-existing VPN gateway (while simultaneously allowing for specifying a pre-existing NAT gateway as you can today).
Contributor guide
Research direction
Read docs/run_in_existing_vpc.md and the existing cluster configuration for private subnets and egress settings. Trace how kops creates subnet route tables and applies pre-existing NAT gateway routes. Done means users can specify additional routes targeting a pre-existing VPN gateway while creating dedicated kops-managed subnets, with the resulting utility and private route tables containing those routes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go
- Domain
- cloud, infrastructure, networking
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100