scverse / scverse/mantispy

Add an iLISI and silhouette equivalence test against scib-metrics

Open
#53 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8
Forks
0
Avg merge
12h 18m
Merged PRs (30d)
26

Description

mantispy.metrics reimplements scib's definitions rather than depending on scib, and tests/test_api_guards.py asserts that src/ never imports scib, scib-metrics, pycytominer, pyod, cytominer-eval or scmorph, so an equivalence check cannot become circular.

The gap: nothing actually checks the reimplementation against a reference. tests/test_metrics.py used to claim "equivalence with scib where the definition is shared" in its module docstring while no test performed one; #51 corrects the docstring rather than adding the test.

Worth adding as a test-time-only dependency, in the spirit of the existing pycytominer and scmorph equivalence tests: compare iLISI, cLISI and the label/batch silhouettes against scib-metrics on a fixed synthetic object, asserting numerical agreement rather than a conceptual match. That would earn back the sentence that was removed.

Note the LISI implementation already claims agreement with harmonypy.lisi.compute_lisi to machine precision; a scib-metrics comparison would cover the silhouettes and PC regression too.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with tests/test_metrics.py and the existing pycytominer and scmorph equivalence tests, then review tests/test_api_guards.py to preserve the no-source-import rule. Add scib-metrics only for testing and compare iLISI, cLISI, label silhouettes, batch silhouettes, and PC regression on a fixed synthetic object. Done means the numerical comparisons pass and the API guard remains satisfied.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.