azavea / azavea/simple-raster-processing

Investigate AWS lambda execution

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
brainstorming
Dominant language
Python
Stars
6
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Brainstorming.

If #3 works well for windowed reads, create a lambda function for `Counts` and deploy to a variety of memory configurations (ie, 0.5GB, 1GB, 1.5GB). Route requests through a load balancer which can take the input geometry and the target raster metadata (cell size + data type) and evaluate the amount of memory needed to read in the bounding box window - then route to that configured function for processing.

Pros:
- Scalable for concurrent requests, no server to overwhelm
- Minimizes cost per request
- Could scale to many generic functions, ie `Counts`, `MapAlgebra_Add`, etc

Cons:
- S3 networking latency
- Can GDAL and all the requirements actually fit?
- Cold start up time adds to request time
- Lambda memory seems to max out at 1.5 GB, not sure how limiting that would be for certain queries given other system memory requirements

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.