Investigate use of Hypothesis for testing
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 303
- Forks
- 102
- Avg merge
- 12h 39m
- Merged PRs (30d)
- 17
Description
Description
Given discussions with @Zac-HD at SciPy 2019, it would be interesting to investigate using Hypothesis with the pytest test suite.
As @lukasheinrich has pointed out, a good first example to try out would be ensuring that all CLs values are [0,1]. Also, as @kratsg has pointed out, with hypothesis-jsonschema it might be possible to generate workspaces on the fly using the JSON schemas.
For posterity and future reference I will also link to Zac's very nice SciPy 2019 talk: Test Research Code the Easy Way: By Generating Random Inputs with Hypothesis
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the pytest suite and the proposed CL-value example, checking that CL values are constrained to [0,1]. Then examine the JSON schemas under pyhf/schemas/1.0.0 and the hypothesis-jsonschema option for generating workspaces. Done means a concrete assessment or implementation path for using Hypothesis is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100