Full test reproducibility appears to be failing on internal tests
Open
Nobody has claimed this yet.
bug
- Dominant language
- R
- Stars
- 21
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
── Failure ('test-reproducibility.R:75:9'): Results are Reproducible ───────────
`drawobj` (`actual`) not equal to `drawobj2` (`expected`).
actual$samples[[1]]$beta | expected$samples[[1]]$beta
[1] 4.844 - 5.504 [1]
[2] 8.212 - 6.047 [2]
[3] 0.724 - -0.615 [3]
[4] 0.448 - -0.599 [4]
[5] 3.090 - 3.010 [5]
[6] 2.247 - 2.715 [6]
[7] -1.141 - 1.856 [7]
[8] -0.486 - 2.180 [8]
actual$samples[[1]]$sigma$A vs expected$samples[[1]]$sigma$A
[,1] [,2] [,3]
- actual$samples[[1]]$sigma$A[1, ] 3.8996258 1.81540872 0.18346780
+ expected$samples[[1]]$sigma$A[1, ] 4.9890982 1.04694767 -0.13357694
- actual$samples[[1]]$sigma$A[2, ] 1.8154087 1.88882220 -0.04483718
+ expected$samples[[1]]$sigma$A[2, ] 1.0469477 1.02407796 -0.03493389
- actual$samples[[1]]$sigma$A[3, ] 0.1834678 -0.04483718 0.37916568
+ expected$samples[[1]]$sigma$A[3, ] -0.1335769 -0.03493389 0.35201823
Haven't checked but I think this is from the FULL test suite that doesn't get checked via the GitHub actions pipelines
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked check-rbmi.html artifact and test-reproducibility.R at line 75, then compare how the full test suite is run with the GitHub Actions pipelines. Reproduce the mismatch and identify whether the failure is limited to the full suite. Done means the reproducibility test passes consistently or the differing test coverage is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100