NixOS / NixOS/ofborg

Testing functions with "passthru" tests.

Open
#578 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
323
Forks
171
PR merge metrics
No merged PRs in 30d

Description

Functions producing derivations can have their own tests attribute, just like a derivation can have a tests attribute using mkDerivation's passthru.tests. While a plain lambda can't have attributes, a callable attrset with __functor, as produced by callPackage for example, can.

While some functions are already tested indirectly by having calls in the package tree (mkDerivation has quite some coverage this way...), some functions have surprisingly little. Oddly, the fetchers are among those, because they often don't even have to run, as their outputs are already available.

With https://github.com/NixOS/nixpkgs/pull/136022 we can use a trick to run fetchers whenever their implementations change, which is great for testing. However, ofborg can't run them yet. It uses nix-build (or a similar command) to run the tests, but it tries to invoke the __functor function instead of going directly for the attribute. nix build does not suffer from this problem, but alternatively, we can introduce a packageTests attribute in Nixpkgs as an alias for the expected behavior. I've elided it from that PR, but it could be reintroduced for ofborg specifically. See https://github.com/NixOS/nixpkgs/pull/136022#issuecomment-923940598

So it seems that all ofborg needs to do to support running these tests is to either
a. switch to nix build, or
b. switch from *.tests (eqv. *.passthru.tests if it insists?) to packageTests.*

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how ofborg invokes tests with nix-build and compare that behavior with nix build. Check how *.tests and *.passthru.tests are selected, then evaluate the two proposed paths: switching commands or introducing packageTests as an alias. Done means ofborg can run tests for callable attrsets and fetchers without invoking __functor.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
ci-cd, devtools
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.