SciML / SciML/FiniteVolumeMethod1D.jl

[CI Broken] Multiple test failures on Julia 1.0 and 1.6

Open
#16 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
8
Forks
2
Avg merge
36m
Merged PRs (30d)
3

Description

CI Health Check: Tests Failing

Summary

The CI is currently broken on the main branch. Automated health check PR #14 revealed multiple failures across different Julia versions.

Failed Jobs

  1. Julia 1.0 on ubuntu-latest: Threading configuration error
  2. Julia 1.6 on ubuntu-latest: Visual regression test failures (3 errored tests)

Detailed Analysis

Julia 1.0 Failure

Root Cause: Invalid threading configuration in CI workflow

Error Message:

ERROR: julia: -t,--threads=<n>[,auto|<m>]; n must be an integer >= 1
Package FiniteVolumeMethod1D errored during testing

Issue: The CI workflow file (.github/workflows/CI.yml) sets JULIA_NUM_THREADS: ${{ matrix.julia-threads }} on line 18, but matrix.julia-threads is never defined in the matrix configuration (lines 22-28). This passes an empty value to Julia's -t flag, causing the error.

Suggested Fix: Either:

  • Remove the JULIA_NUM_THREADS environment variable if threading is not needed
  • Add julia-threads to the matrix configuration with appropriate values
Julia 1.6 Failure

Root Cause: Visual regression test failures due to reference image mismatches

Test Results: 67 passed, 0 failed, 3 errored, 0 broken

Failed Tests:

  1. Porous-Medium - Reference image mismatch at docs/src/figures/porous_surface.png
  2. Robin Diffusion - Reference image mismatch at docs/src/figures/robin_diffusion_surface.png
  3. Reaction-Diffusion - Reference image mismatch at docs/src/figures/dirichlet_source_surface.png

Error Message:

LoadError: To update the reference images either run the tests interactively with 'include("test/runtests.jl")',
or to force-update all failing reference images set the environment variable `JULIA_REFERENCETESTS_UPDATE`
to "true" and re-run the tests via Pkg.

Possible Causes:

  • Upstream dependency changes affecting plot rendering
  • Julia version differences in plotting libraries
  • Platform-specific rendering differences

Suggested Fix Approach:

  1. Run tests locally on Julia 1.6 to reproduce the issue
  2. Review the actual vs reference images to determine if differences are acceptable
  3. If differences are minor/acceptable: Update reference images by setting JULIA_REFERENCETESTS_UPDATE=true
  4. If differences indicate a real regression: Investigate changes in dependencies or plotting code

Links

Priority

High - CI is broken on main branch, blocking development and future PRs.

cc @ChrisRackauckas

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .github/workflows/CI.yml lines 18 and 22-28 to inspect the Julia 1.0 threading matrix, then run the Julia 1.6 test suite and review the three reference-image mismatches under docs/src/figures/. Confirm whether the rendering differences are acceptable or indicate a regression, and verify that both Julia jobs pass in CI without unexplained image changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
ci-cd, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.