CodeWithAsheville / CodeWithAsheville/jaildata
Shift application to a data lake architecture
Open
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
We don't yet have clear use cases around which to build APIs, design our DynamoDB table and so on.
Instead, we're going to store the data as Iceberg/Parquet files in S3 and use Athena for queries. We'll drop daily files throughout the month and then run a compaction job (`OPTIMIZE`) to put everything into a monthly file at EoM.
Partitioning will be by facility ('buncombe', 'wake', _etc._) and year+month.
Rework Terraform and Serverless Framework IaC definitions to accommodate this architecture.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.