TuringLang / TuringLang/SSMProblems.jl
Implemented CI-based unit tests
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 11
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
Ran into an issue today with one of the new prototype algorithms (RBPFBSi) where it looked like the posteriors were correct to high tolerance (rtol = 1e-3) but they were actually incorrect. This was only spotted when I whacked N_sample and N_particles up to massive numbers to test the runtime.
I noticed that this false negative could have been avoid if I had used the std of the mean estimates generated by the RBPF to see whether the true Kalman mean was within a 95% confidence interval. This is probably the better way to write tests going forward as it allows us to avoid setting arbitrary rtols and instead be model-driven.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test path is named. Start by locating the RBPFBSi implementation and existing unit tests that compare posteriors with rtol, then review how CI runs them. Done means the tests use RBPF mean-estimate standard deviations and a 95% confidence interval rather than arbitrary tolerances.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100