minimize the cost when deploying services onto AWS

## Overview

To start with and to be clear, the Texera project involves three key phases to achieve broad general public access and scalability:

1. **Phase 1: Local Deployment**
- Initially, all services are deployed on a large, powerful local cluster.
- **Challenges**:
- Handling large file issues.

2. **Phase 2: AWS Deployment**
- We run all the services on remote AWS VMs to support a limited number of users.
- **Challenges**:
- Determining the minimum costs when user activity is low or non-existent.
- High latency when new users initiate computing units.

3. **Phase 3: User-Credentialed Local Deployment**
- The final goal is to run the brain layer locally, where users provide their own credentials of AWS for computation. We will create VMs on their behalf and use their resources to compute for them.

This issue focuses on **Phase 2**, specifically addressing the cost issues related to deploying on AWS.

---

## Framework

To better understand the structure in **Phase 2**, here's an overview of the planned framework (The Texera project is already divided into **8 microservices**):

When deploying on AWS, we do not want all the services to be running all the time (otherwise that will be too costly). Instead, we want them to **start and stop on request**. This means that when a new request comes in, one or multiple services will "wake up" and respond. If there are no requests, no services should wake up.

In this model, we **only pay AWS when computation requests are made**, and we do not incur costs when the system is idle. To achieve this, we plan to use **AWS ECS (Elastic Container Service)**. Here's how it works:

- **Resource Threshold**:
- We set a threshold on resource usage (e.g., only 70% of CPU should be used).
- If total resource usage exceeds the threshold (e.g., CPU usage reaches 85%), AWS ECS will automatically create a new task to handle the additional load (a **task** in ECS is a collection of resources automatically allocated by AWS ECS that can handle multiple requests).

- **Scaling Conditions**:
- The threshold for scaling can be based on CPU usage, RAM usage, or the number of incoming requests.

### Challenges
This approach, while cost-efficient, introduces the latency problem: If all microservices are idle and need to start when a user makes a request, it takes time to initialize. This delays the response and harms user experience.

### Observations and Guesses
The **8 microservices** do not serve user requests simultaneously but instead operate in a **sequential order**. For example: Some key microservices, such as the **"Config Service"** and **"Texera WebApplication"**, are the first to handle user requests. If these micro services are running, then other micro services can be deployed while they are responding the users. This method should reduce the user waiting time by large.

### Current Solution
To conclude what we observe:
- **Keep a minimum number of microservices running at all times** to improve responsiveness.
- While these critical services handle the initial user request, other microservices can start in the background.

---

## Approach

To address the challenges in **Phase 2**, we plan to conduct experiments and analyze the trade-offs between cost and performance.

No files or tests are identified; start by reviewing the AWS ECS deployment model and the request flow through the Config Service and Texera WebApplication. Define cost-versus-latency experiments for minimum running services and scaling thresholds, with a documented recommendation as the completion criterion.

Written by the indexing model from the issue text.