JuliaDiff / JuliaDiff/ChainRulesTestUtils.jl
Stacktraces are hidden
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 50
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
When tests fail, I think this package is somehow helpfully capturing the error (with try) and hiding it:
test_rrule: partialsort on Vector{Float64},UnitRange{Int64}: Error During Test at /home/runner/.julia/packages/ChainRulesTestUtils/YbVdW/src/testers.jl:193
[232](https://github.com/JuliaDiff/ChainRulesCore.jl/runs/8017765323?check_suite_focus=true#step:6:233)
Got exception outside of a @test
[233](https://github.com/JuliaDiff/ChainRulesCore.jl/runs/8017765323?check_suite_focus=true#step:6:234)
Can this be removed?
The default behaviour is to print the stacktrace, which is useful information about what's actually caused the error, and where it was called from. (Not testers.jl:193.)
julia> @testset "outer" begin
@testset "inner" begin
@test 1+1==2
sqrt(-1) # outside test
end
end
inner: Error During Test at REPL[5]:2
Got exception outside of a @test
DomainError with -1.0:
sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
Stacktrace:
[1] throw_complex_domainerror(f::Symbol, x::Float64)
@ Base.Math ./math.jl:33
[2] sqrt
@ ./math.jl:675 [inlined]
[3] sqrt(x::Int64)
@ Base.Math ./math.jl:1480
...
Test Summary: | Pass Error Total Time
outer | 1 1 2 0.2s
inner | 1 1 2 0.2s
ERROR: Some tests did not pass: 1 passed, 0 failed, 1 errored, 0 broken.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in testers.jl at line 193, where the reported test failure indicates the exception is being handled. Reproduce the behavior with a failing test or the nested @testset example from the issue, then trace how the error is captured. Done means errors outside @test retain the originating stacktrace instead of only reporting testers.jl:193.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100