JuliaAI / JuliaAI/MLJTestIntegration.jl

failures are not very legible

Open
#22 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Julia
Stars
4
Forks
3
Avg merge
21m
Merged PRs (30d)
4

Description

For example

XGBoostClassifier: Test Failed at /home/expandingman/.julia/dev/MLJXGBoostInterface/test/runtests.jl:235
  Expression: isempty(failures)
   Evaluated: isempty(NamedTuple{(:name, :package_name, :test, :exception), NTuple{4, Any}}[NamedTuple{(:name, :package_name, :test, :exception), NTuple{4, Any}}(("XGBoostClassifier", "XGBoost", "fitted_machine", ErrorException("Call to XGBoost C function XGBoosterUpdateOneIter failed: [18:48:00] /workspace/srcdir/xgboost/src/objective/regression_obj.cu:36: Check failed: info.labels.Size() == preds.Size() (200 vs. 400) : Invalid shape of labels.\nStack trace:\n  [bt] (0) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(+0x32baa7) [0x7f9f5cfa9aa7]\n  [bt] (1) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(+0x32bd53) [0x7f9f5cfa9d53]\n  [bt] (2) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(xgboost::obj::RegLossObj<xgboost::obj::LogisticClassification>::GetGradient(xgboost::HostDeviceVector<float> const&, xgboost::MetaInfo const&, int, xgboost::HostDeviceVector<xgboost::detail::GradientPairInternal<float> >*)+0x29) [0x7f9f5cfbcf49]\n  [bt] (3) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(xgboost::LearnerImpl::UpdateOneIter(int, std::shared_ptr<xgboost::DMatrix>)+0x295) [0x7f9f5cef1f15]\n  [bt] (4) /home/expandingman/.julia/artifacts/501324fbf7eec8ade7ee93955843c5cbdbd43fdc/lib/libxgboost.so(XGBoosterUpdateOneIter+0x42) [0x7f9f5cd99ba2]\n  [bt] (5) [0x7fa0047cb7a8]\n  [bt] (6) [0x7fa0047cc32b]\n  [bt] (7) [0x7fa0047cd11b]\n  [bt] (8) [0x7fa0047cd1ac]\n\n")))])

It doesn't help that the Test stdlib doesn't have very good printing which I believe is producing this output, however the situation is exacerbated with these integration tests because it's harder to track down the exact test that failed.

I suggest that some try catching to print errors more legibly may be appropriate.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in test/runtests.jl around line 235 and inspect how the Julia Test stdlib prints the integration failure. Compare the current output with the requested legibility, including identifying the exact failing test and presenting the exception more clearly; done means failures are concise and actionable.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.