VH-Lab / VH-Lab/vhlab-library-python

Add CI, and symmetry tests that check the port against vhlab-library-matlab

Open
#7 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
1
Avg merge
1h 53m
Merged PRs (30d)
4

Description

Neither this repository nor vhlab-library-matlab has any CI. There is no .github/ directory in either, so nothing runs on push or on a pull request, and nothing checks that the MATLAB and Python implementations still agree.

A companion issue covers the MATLAB side: VH-Lab/vhlab-library-matlab#8.

Current state here

  • No workflows, no .github/.
  • No packaging metadata — no setup.py, no pyproject.toml. requirements.txt lists vhlab-toolbox-python, numpy, scipy, pyyaml.
  • tests/ now exists (added in #5) with 18 tests under unittest: tests/test_bridge.py validating the bridge contract files, and tests/test_neural_response_significance.py.
  • Run locally with python3 -m unittest discover -s tests.

1. CI for this repository

A GitHub Actions workflow on push and pull request that:

  • runs python3 -m unittest discover -s tests on a couple of Python versions;
  • installs vhlab-toolbox-python from GitHub rather than PyPI. This is the one thing that needs a decision before the workflow can be writtenvlt is not importable in a clean environment today, so any test that touches it is currently unwritable, which is part of why the ported modules have so few tests. Either pin a git URL in requirements.txt or add a CI step that clones it.
  • optionally runs a linter. Worth deciding whether to adopt one at all before wiring it up; a linter that fails on existing code on day one gets switched off.

2. Symmetry tests

The bridge contract files added in #5 make part of this mechanical. Two distinct kinds, worth keeping separate:

Structural symmetry — cheap, needs no MATLAB, can run in this repository's CI as soon as the MATLAB repo is checked out alongside:

  • every matlab_path in every vhlib_matlab_python_bridge.yaml exists in vhlab-library-matlab;
  • every .m file in a ported area has an entry, so a function added on the MATLAB side shows up as a gap rather than silently;
  • matlab_last_sync_hash is compared against the current MATLAB HEAD, and the test reports which ported functions have drifted since they were last checked. This is the field that makes drift detectable rather than assumed, and it is worthless unless something reads it.

Numerical symmetry — the same fixture through both implementations, asserting the same answer. Running MATLAB inside this repository's CI is the expensive option; the cheaper one is golden fixtures: a MATLAB script generates reference inputs and outputs, commits them as JSON, and the Python tests compare against them. The MATLAB half of that belongs in the companion issue.

Good first candidates, because they are pure functions with no dirstruct or file-system dependency:

  • neural_response_significance — has Python tests as of #5, but they check against an ANOVA computed from the definition, not against MATLAB's anova1. A golden fixture would close that gap.
  • cellname2nameref / nameref2cellname / cellname2date — string round-trips.
  • filter_by_index / filter_by_reference — where the 1-based to 0-based index decision could silently go wrong.
  • read_stimtimes_txt and write_stimtimes_txt — round-trip a file written by one and read by the other.

spiketriggeredaverage is the one place where symmetry is not expected: the Python signature deliberately diverges (raw arrays instead of measureddata/spikedata objects, three outputs instead of six), as recorded in vhlib/md/vhlib_matlab_python_bridge.yaml. Any symmetry harness needs a way to mark that, rather than reporting it as a failure forever.

Note

Whatever shape this takes, it should apply to vhlab-toolbox-python too, which has the same gap.

🤖 Generated with Claude Code

https://claude.ai/code/session_01QJXrVvcT4jif5nwu5ihLkV

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with requirements.txt, tests/, the existing unittest command, and vhlib/md/vhlib_matlab_python_bridge.yaml; the issue also requires checking the companion MATLAB issue. Resolve the dependency, linter, fixture, and cross-repository workflow decisions before implementation. Done means push and pull-request CI runs the tests and reports structural or numerical drift against vhlab-library-matlab.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.