AllenInstitute / AllenInstitute/AllenSDK
VisualBehaviorOphysProjectCache.from_lims() method does not accurately reflect what is in lims
@danielsf is already working on this.
Since Aug 12, 2021.
- Dominant language
- Jupyter Notebook
- Stars
- 398
- Forks
- 163
- PR merge metrics
- No merged PRs in 30d
Description
Describe the bug
When I run the following code, the table that is returned does not include the complete list of ophys_experiment_ids for each ophys_container_id. It appears to require that a given ophys_experiment_id is only represented once, however it is possible for an ophys_experiment_id to be associated with multiple ophys_container_ids in lims (some that have passed and some that were failed). The result of this is that sometimes an ophys_experiment_id is associated with a passed container, and other times with a failed container, meaning that the passing ophys_container_id does not have the correct list of ophys_experiment_ids associated with it.
To Reproduce
from allensdk.brain_observatory.behavior.behavior_project_cache import VisualBehaviorOphysProjectCache as bpc
cache = bpc.from_lims()
experiments = cache.get_ophys_experiment_table()
print('there are', len(experiments[experiments.ophys_container_id==1115959875]),
'experiments associated with container id', 1115959875, 'in the cache table')
Expected behavior
There are actually 6 experiments associated with that container ID in lims. This can be observed through direct lims queries, and by looking at the lims directory for that container:

Workaround
@djkapner identified that this part of the code in the SDK enforces that there should only be one unique ophys_experiment_id in the ophys_experiments table, and suggested the code block below as an alternative workaround. This block gives me the correct list of ophys_experiment_ids in lims for a given ophys_container_id, including cases where an ophys_experiment_id is associated with multiple containers. This is the ground truth that we need access to do QC and other tasks, like identifying candidate experiments for release.
experiments = cache.fetch_api.get_ophys_experiment_table()
print('there are', len(experiments[experiments.ophys_container_id==1115959875]),
'experiments associated with container id', 1115959875, 'in the cache table')
print('yay this is correct')
Environment (please complete the following information):
- AllenSDK version 2.11.0
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.