AlexsLemonade / AlexsLemonade/refinebio

Stop using AWS's elasticsearch service

Open
#2,106 1 comment 0 reactions 0 assignees View on GitHub
cost cutting exploration
Dominant language
Python
Stars
135
Forks
21
PR merge metrics
No merged PRs in 30d

Description

### Context

AWS Elasticsearch somehow takes forever to spin up. I had a fresh deploy fail because ES didn't come up within an HOUR:

```
aws_elasticsearch_domain.es: Still creating... (1h0m10s elapsed)

Error: Error applying plan:

1 error(s) occurred:

* aws_elasticsearch_domain.es: 1 error(s) occurred:

* aws_elasticsearch_domain.es: "arn:aws:es:us-east-1:589864003899:domain/es-kurt-dev": Timeout while waiting for the domain to be created
```

### Problem or idea

I'm pretty sure ES is just a server with some software installed on it. I cannot imagine what's taking so incredibly long, but I bet we can do it faster if we just install it ourselves. Also according to https://code972.com/blog/2017/12/111-why-you-shouldnt-use-aws-elasticsearch-service there's a lot of other issues with the service and it's twice as expensive.

### Solution or next step

Use an EC2 instance to host Elasticsearch for us rather than AWS's provided service.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the infrastructure configuration that defines aws_elasticsearch_domain.es and the deployment path that waits for it; the issue provides no file or test path. Done means managed AWS Elasticsearch is replaced by an EC2-hosted Elasticsearch setup and a fresh deploy completes without the managed-domain timeout.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, elasticsearch
Domain
cloud, infrastructure, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.