JuliaGaussianProcesses / JuliaGaussianProcesses/KernelFunctions.jl
Tests checking the metric are unhelpful
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 275
- Forks
- 41
- PR merge metrics
- No merged PRs in 30d
Description
There are quite a lot of tests that check that metric(kernel) returns a particular metric. It's not clear to me how much value these tests have.
It probably makes sense to ensure that SimpleKernels do implement metric, and return a valid type of metric, but I don't think that we should be testing that they return a specific metric, as this feels like writing a test is basically the same code as the code it's testing, which I believe generally gives a false sense of security.
I'm pro- testing that kernels yield the correct numerical values for certain specific inputs / parameters, or that they are the same as other kernels for particular settings (e.g. gamma-exponential and EQ / exponential) as with this this type of test you're typically not just writing out your source code in your tests.
What are people's thoughts?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Search the test suite for checks that metric(kernel) returns a particular metric, then review the SimpleKernel and metric discussion in this issue. The work is done when the project has a decided, consistent testing approach that preserves numerical and equivalence checks while resolving the disputed specific-metric tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100