cortex-lab / cortex-lab/phy

Waveforms view only show mean waveforms

Open
#1,238 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: visualization status: confirmed type: bug
Dominant language
Python
Stars
430
Forks
187
Avg merge
2h 31m
Merged PRs (30d)
2

Description

Hi all,
I have an issue using the waveforms view. I can only display the mean waveform view and the button "toggle mean waveforms" does not have any effect.

Here is my phy folder : https://instituteicm-my.sharepoint.com/:f:/g/personal/anthony_pinto_icm-institute_org/EjKkyzBR4O5OvbYDfxM1R9sBHFJO2Mmn3hOWhgyv42xfYg?e=jwBj4v

Here is the log file if you want to have a quick look at it before downloading all the data.

16:13:31.396 [D] context:80           Create cache directory `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.396 [D] context:80           Create cache directory `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.806 [D] context:100          Initialize joblib cache dir at `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.806 [D] context:100          Initialize joblib cache dir at `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy`.
16:13:31.826 [D] context:101          Reducing the size of the cache if needed.
16:13:31.826 [D] context:101          Reducing the size of the cache if needed.
16:13:32.017 [D] base:102             Add filter `high_pass`.
16:13:32.017 [D] base:102             Add filter `high_pass`.
16:13:32.101 [D] config:31            Load config file `C:\Users\anthony.pinto\.phy\phy_config.py`.
16:13:32.101 [D] config:31            Load config file `C:\Users\anthony.pinto\.phy\phy_config.py`.
16:13:32.189 [D] plugin:145           Loading 0 plugins.
16:13:32.189 [D] plugin:145           Loading 0 plugins.
16:13:32.866 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\new_cluster_id.pkl` doesn't exist.
16:13:32.866 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\new_cluster_id.pkl` doesn't exist.
16:13:32.888 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl` doesn't exist.
16:13:32.888 [D] context:209          The file `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl` doesn't exist.
16:13:32.901 [D] clustering:237       Recompute spikes_per_cluster manually: this might take a while.
16:13:32.901 [D] clustering:237       Recompute spikes_per_cluster manually: this might take a while.
16:13:32.974 [D] context:185          Save data to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl`.
16:13:32.974 [D] context:185          Save data to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\spikes_per_cluster.pkl`.
16:13:33.257 [D] gui:463              Creating GUI.
16:13:33.257 [D] gui:463              Creating GUI.
16:13:33.333 [D] state:46             Load C:\Users\anthony.pinto\.phy\TemplateGUI\state.json for GUIState.
16:13:33.333 [D] state:46             Load C:\Users\anthony.pinto\.phy\TemplateGUI\state.json for GUIState.
16:13:34.035 [D] gui:718              Add view ClusterView to GUI.
16:13:34.035 [D] gui:718              Add view ClusterView to GUI.
16:13:34.186 [D] gui:718              Add view SimilarityView to GUI.
16:13:34.186 [D] gui:718              Add view SimilarityView to GUI.
16:13:34.293 [D] gui:718              Add view WaveformView to GUI.
16:13:34.293 [D] gui:718              Add view WaveformView to GUI.
16:13:34.372 [D] base:337             Set state for WaveformView.
16:13:34.372 [D] base:337             Set state for WaveformView.
16:13:34.465 [D] gui:718              Add view CorrelogramView to GUI.
16:13:34.465 [D] gui:718              Add view CorrelogramView to GUI.
16:13:34.536 [D] base:337             Set state for CorrelogramView.
16:13:34.536 [D] base:337             Set state for CorrelogramView.
16:13:34.623 [D] gui:718              Add view ISIView to GUI.
16:13:34.623 [D] gui:718              Add view ISIView to GUI.
16:13:34.693 [D] base:337             Set state for ISIView.
16:13:34.693 [D] base:337             Set state for ISIView.
16:13:34.784 [D] gui:718              Add view FeatureView to GUI.
16:13:34.784 [D] gui:718              Add view FeatureView to GUI.
16:13:34.856 [D] base:337             Set state for FeatureView.
16:13:34.856 [D] base:337             Set state for FeatureView.
16:13:34.954 [D] gui:718              Add view AmplitudeView to GUI.
16:13:34.954 [D] gui:718              Add view AmplitudeView to GUI.
16:13:35.074 [D] base:337             Set state for AmplitudeView.
16:13:35.074 [D] base:337             Set state for AmplitudeView.
16:13:35.111 [D] gui:718              Add view FiringRateView to GUI.
16:13:35.111 [D] gui:718              Add view FiringRateView to GUI.
16:13:35.125 [D] base:337             Set state for FiringRateView.
16:13:35.125 [D] base:337             Set state for FiringRateView.
16:13:35.444 [D] gui:519              Load the geometry state.
16:13:35.444 [D] gui:519              Load the geometry state.
16:13:38.599 [D] supervisor:817       Clusters selected: [13] (20)
16:13:38.599 [D] supervisor:817       Clusters selected: [13] (20)
16:16:12.074 [D] state:171            Update GUI state for ClusterView
16:16:12.074 [D] state:171            Update GUI state for ClusterView
16:16:12.139 [D] state:171            Update GUI state for WaveformView
16:16:12.139 [D] state:171            Update GUI state for WaveformView
16:16:12.205 [D] state:171            Update GUI state for CorrelogramView
16:16:12.205 [D] state:171            Update GUI state for CorrelogramView
16:16:12.268 [D] state:171            Update GUI state for ISIView
16:16:12.268 [D] state:171            Update GUI state for ISIView
16:16:12.387 [D] state:171            Update GUI state for FeatureView
16:16:12.387 [D] state:171            Update GUI state for FeatureView
16:16:12.428 [D] state:171            Update GUI state for AmplitudeView
16:16:12.428 [D] state:171            Update GUI state for AmplitudeView
16:16:12.448 [D] state:171            Update GUI state for FiringRateView
16:16:12.448 [D] state:171            Update GUI state for FiringRateView
16:16:12.455 [D] context:138          Save memcache for `phy.apps.base._get_mean_waveforms`.
16:16:12.455 [D] context:138          Save memcache for `phy.apps.base._get_mean_waveforms`.
16:16:12.494 [D] context:138          Save memcache for `phy.apps.base._get_template_waveforms`.
16:16:12.494 [D] context:138          Save memcache for `phy.apps.base._get_template_waveforms`.
16:16:12.527 [D] context:138          Save memcache for `phy.apps.base.get_mean_spike_template_amplitudes`.
16:16:12.527 [D] context:138          Save memcache for `phy.apps.base.get_mean_spike_template_amplitudes`.
16:16:12.562 [D] context:138          Save memcache for `phy.apps.base.get_template_counts`.
16:16:12.562 [D] context:138          Save memcache for `phy.apps.base.get_template_counts`.
16:16:12.601 [D] context:138          Save memcache for `phy.apps.base.get_template_for_cluster`.
16:16:12.601 [D] context:138          Save memcache for `phy.apps.base.get_template_for_cluster`.
16:16:12.634 [D] context:138          Save memcache for `phy.apps.template.gui.get_template_amplitude`.
16:16:12.634 [D] context:138          Save memcache for `phy.apps.template.gui.get_template_amplitude`.
16:16:12.742 [D] context:138          Save memcache for `phy.apps.base.get_cluster_amplitude`.
16:16:12.742 [D] context:138          Save memcache for `phy.apps.base.get_cluster_amplitude`.
16:16:12.837 [D] context:138          Save memcache for `phy.apps.base.get_mean_firing_rate`.
16:16:12.837 [D] context:138          Save memcache for `phy.apps.base.get_mean_firing_rate`.
16:16:12.912 [D] context:138          Save memcache for `phy.apps.base.get_best_channel`.
16:16:12.912 [D] context:138          Save memcache for `phy.apps.base.get_best_channel`.
16:16:12.967 [D] context:138          Save memcache for `phy.apps.template.gui.get_best_channels`.
16:16:12.967 [D] context:138          Save memcache for `phy.apps.template.gui.get_best_channels`.
16:16:13.030 [D] context:138          Save memcache for `phy.apps.base.get_channel_shank`.
16:16:13.030 [D] context:138          Save memcache for `phy.apps.base.get_channel_shank`.
16:16:13.084 [D] context:138          Save memcache for `phy.apps.base.get_probe_depth`.
16:16:13.084 [D] context:138          Save memcache for `phy.apps.base.get_probe_depth`.
16:16:13.139 [D] context:138          Save memcache for `phy.apps.base.peak_channel_similarity`.
16:16:13.139 [D] context:138          Save memcache for `phy.apps.base.peak_channel_similarity`.
16:16:13.229 [D] gui:600              Save the geometry state.
16:16:13.229 [D] gui:600              Save the geometry state.
16:16:13.240 [D] state:218            Save global GUI state to `C:\Users\anthony.pinto\.phy\TemplateGUI\state.json`.
16:16:13.240 [D] state:218            Save global GUI state to `C:\Users\anthony.pinto\.phy\TemplateGUI\state.json`.
16:16:13.256 [D] state:232            Save local GUI state to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\state.json`.
16:16:13.256 [D] state:232            Save local GUI state to `\\l2export\iss02.charpier\analyses\anthony.pinto\git_lustre\priam\data\1_2_postprocessed\multi_v2.0\02711_2019-04-19_10-00\curation\mHaT2\phy\.phy\state.json`.

I am a bit confused about the exact reason of this issue. I exported the data using the function from spikeinterface like that:

sexp.export_to_phy(
                waveform_extractor=waveforms,
                output_folder=os.path.join(output_folder, 'phy'),
                compute_pc_features=True,
                compute_amplitudes=True,
                copy_binary=True,
                peak_sign='neg',
                remove_if_exists=True,
                verbose=True,
            )

To add a bit more context, my folder is located at a very long path and it was the origin of a problem with the waveforms a few weeks ago as mentionned here https://github.com/cortex-lab/phy/issues/1219#issue-1880538192 but I found a workaround by doing that : https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/ so I avoided the windows limit of 260 characters. I though it solved my issue because with the previous issue the waveforms were not displaying at all and now I can see them.
I would like to find a workaround because I would like to avoid to move the location of the phy folder if possible.

Cheers,
Anthony

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with the exported phy folder and the provided spikeinterface export_to_phy call. Inspect the WaveformView behavior around the “toggle mean waveforms” control and compare it with the logged waveform-loading steps. Done means the control visibly switches between mean and non-mean waveforms for a reproducible dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-visualization
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.