JuliaDiff / JuliaDiff/ChainRulesTestUtils.jl

Stacktraces are hidden

Open
#261 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
50
Forks
18
PR merge metrics
No merged PRs in 30d

Description

When tests fail, I think this package is somehow helpfully capturing the error (with try) and hiding it:

test_rrule: partialsort on Vector{Float64},UnitRange{Int64}: Error During Test at /home/runner/.julia/packages/ChainRulesTestUtils/YbVdW/src/testers.jl:193
[232](https://github.com/JuliaDiff/ChainRulesCore.jl/runs/8017765323?check_suite_focus=true#step:6:233)
  Got exception outside of a @test
[233](https://github.com/JuliaDiff/ChainRulesCore.jl/runs/8017765323?check_suite_focus=true#step:6:234)

Can this be removed?

The default behaviour is to print the stacktrace, which is useful information about what's actually caused the error, and where it was called from. (Not testers.jl:193.)

julia> @testset "outer" begin
         @testset "inner" begin
           @test 1+1==2
           sqrt(-1) # outside test
         end
       end
inner: Error During Test at REPL[5]:2
  Got exception outside of a @test
  DomainError with -1.0:
  sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
  Stacktrace:
    [1] throw_complex_domainerror(f::Symbol, x::Float64)
      @ Base.Math ./math.jl:33
    [2] sqrt
      @ ./math.jl:675 [inlined]
    [3] sqrt(x::Int64)
      @ Base.Math ./math.jl:1480
...
Test Summary: | Pass  Error  Total  Time
outer         |    1      1      2  0.2s
  inner       |    1      1      2  0.2s
ERROR: Some tests did not pass: 1 passed, 0 failed, 1 errored, 0 broken.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in testers.jl at line 193, where the reported test failure indicates the exception is being handled. Reproduce the behavior with a failing test or the nested @testset example from the issue, then trace how the error is captured. Done means errors outside @test retain the originating stacktrace instead of only reporting testers.jl:193.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.