spcl / spcl/serverless-benchmarks
Fetching external benchmarks data
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 199
- Forks
- 105
- PR merge metrics
- No merged PRs in 30d
Description
Currently, all data is assumed to be cloned from the benchmarks-data repository or downloaded from a GH release. However, we need to provide additional data that is (a) under a problematic license or (b) too large to fetch from GitHub. In particular, this problem appears in SeBS-Flow.
We should have a fetch system that:
- Runs after initializing benchmarks-data
- Runs all custom "fetch" scripts in the benchmarks-data.
- Clearly communicates this to the user - how much we fetch and from where (we should put size expectations to not surprise users).
- Adds a hidden marker to skip refetching.
Good place to install this feature: https://github.com/spcl/serverless-benchmarks/blob/master/sebs/utils.py#L737
CC: @lolamaspoli
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in sebs/utils.py around line 737 and inspect how benchmarks-data is initialized, then review the custom fetch scripts in the benchmarks-data repository. Define the fetch step, user-facing size and source information, and the hidden marker behavior. Done means initialization runs the scripts once and skips them on subsequent runs when the marker is present.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100