wso2 / wso2/testgrid

Address the EC2 limitation on maximum number of VPCs

Open
#528 3 comments 0 reactions 1 assignee View on GitHub

@harshanL is already working on this.

Since Feb 27, 2018.

Priority/High Severity/Blocker Type/New Feature
Dominant language
Java
Stars
60
Forks
78
PR merge metrics
No merged PRs in 30d

Description

Description:
We need to address the EC2 limitations such as the number of VPCs possible. We can immediately start addressing the issue with # of VPCs and internet gateways. See the image below:

image

We need to come up with a strategy on how to address this limitation. Immediate options:

  1. Request AWS support to increase the limitation of 5 (to may be 20?). We get this done for multiple regions, not just for us-east-1.
  2. Leverage AWS regions to workaround this limitation. We can create five VPCs in each of the regions. There are total of 10+ regions. If we leverage this, we can go for a maximum of 50 (5VPCs * 10 regions) parallel deployments.
    If the per-region limitation was increased from 5 to 20 for each region, then the total parallel deployments possible become 200 (20 * 10).
  3. Leverage subnets within existing VPCs. Each VPC can have 255 different subnets. Testgrid should be able to leverage this.
    To do this, we have to update the existing cfn scripts to use an existing VPC (by taking it as a input parameter) instead of creating a new VPC for every new deployment. We can pre-process the cfn templates from testgrid side in order to do this.

Out of the above three, we should immediately start doing #1.
At the same time, we should start doing #2 in parallel. This implementation will allow testgrid to be scalable. So, this is not just about addressing VPC count limitation.

Option #3 is also important. But, I think we can live with AWS regions for now.

Affected Product Version:
0.9.0-m14

OS, DB, other environment details and versions:
N/A

Steps to reproduce:
N/A

Related Issues:
#649

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.