Support the Open Meteo custom data format
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 366
- Forks
- 96
- Avg merge
- 24m
- Merged PRs (30d)
- 1
Description
OM is a custom format created by and for open-meteo.com. It’s a Zarr-like format that I bet would be easy to Kerchunkify.
There seems to be a considerable catalog of weather data curated by Open Meteo on AWS: https://github.com/open-meteo/open-data
Here’s some more info about the format:
- https://github.com/open-meteo/open-data?tab=readme-ov-file#file-format
- Reference implementation: https://github.com/open-meteo/open-meteo/blob/main/Sources/SwiftPFor2D/SwiftPFor2D.swift
Some potential sharp edges:
- “Each individual chunk varies in size”
- “As of now, the Open-Meteo API does not directly support partial reads from S3, but this functionality will be implemented in the future.”
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Open Meteo file-format description and the SwiftPFor2D.swift reference implementation linked in the issue, then review the open-data catalog to understand the target files. Done means kerchunk can support the custom format, including its variable-sized chunks and relevant S3 access constraints; the issue names no repository files or tests to run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python, swift
- Domain
- cloud, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100