AllenNeuralDynamics / AllenNeuralDynamics/aind-dynamic-foraging-bfm-dispatcher
[Cross-species transfer] Run study 09 benchmark and report transfer curves
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 2h 9m
- Merged PRs (30d)
- 31
Description
## Context
Parent: #32. The original Grossman, Chen, and Zid matched-half benchmark, author-aligned baselines, representative sessions, and embedding-space analysis were merged in #130. The ten-cohort Stage-A expansion is complete in draft PR #139.
## Stage-A expansion
- Draft implementation and provenance PR: #139
- Admitted with existing schemas: 10 new cohorts (9 v1, 1 v2), for 13 total with the three frozen starter cohorts.
- Skipped without adding schema v3: López-Yépez human, Shin, Alsiö cohort VI, Hattori, and Samejima; exact reasons are in the committed dataset survey.
- [x] Pin full-release sources, licenses/terms, checksums, inclusion rules, and exact cohort counts.
- [x] Generate deterministic v1/v2 manifests and pass exact-count, binary-choice/reward, and loader-contract validation.
- [x] Pass one real-subject local CPU Q smoke for v1 and v2.
- [x] Pass one D=614, seed-0 embedding-only GPU smoke for every newly admitted cohort.
- [x] Complete and freeze the 150-cell expansion GRU matrix on Beaker GPU (195/195 cells across all 13 cohorts).
- [ ] Complete all ten expansion common-Q fits on Allen HPC CPU as SLURM array `25581304`.
- [ ] Freeze exact ordered GRU/Q held-out trial-key parity and update Results 2 and 3.
- [ ] Review Stage-A results before selecting any new author model.
Draft PR #139 contains the completed common-Q fits, frozen parity evidence, and reports; those acceptance boxes remain open until the PR is merged.
## Stage-A decision result
At D=614, the unadjusted subject-paired result favors GRU for Grossman, Chen, Lebedeva, and López-Yépez mouse; it favors common Q for Zid, Kwak, Miller, Findling, and Eckstein; Beron, Tang, Alsiö, and Costa are unresolved. Every cohort improves in trial-pooled GRU likelihood from D=10 to D=614, although several curves peak earlier. Every external cohort is farther from the source embedding distribution than held-out AIND mice in all three seeds.
## Reports
- Result 2: `studies/09-gru-cross-species-transfer/analysis/reports/r2-author-aligned-baselines.md`
- Result 3: `studies/09-gru-cross-species-transfer/analysis/reports/r3-embedding-space.md`
## Remaining original scope
- [ ] Grossman/Chen K curves.
- [ ] Chance/bias and source-informed HB-Q controls when available.
- [ ] Full metric/bootstrap package.
Stage A does not add schema v3 or implement a new author-selected model.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.