spcl / spcl/serverless-benchmarks
Complete and variable inputs for benchmarking
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 199
- Forks
- 105
- PR merge metrics
- No merged PRs in 30d
Description
Currently, not all benchmarks have a full set of inputs. Furthermore, even when we have all inputs, it would be great to use different inputs so that all invocations are not doing exactly the same input file. The input generation should still be deterministic.
This issue also connects with #298 and #300. For some benchmarks, batching support might be needed for a proper "large" input, e.g., see issue #296
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files or tests are named. Start by tracing how benchmark inputs are generated and review the related work in issues #298, #300, and #296; the work is done when every benchmark has complete inputs and deterministic generation can vary inputs between invocations, including batching where needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100