Quality Metrics Seem Off - Spike locations don't match Kilosort4 output

Hi again,

I have some suspicions that my Kilosort4 output is not working very nicely with spikeinterface.

Firstly, all of my PCA-based quality metrics (except maybe isolation distance) are really quite low. Lower than I'd expect given how my units look in Phy (and their contamination percentages).

Through spikeinterface, all but a handful of my units have a nn_hit_rate of less than 0.02 (average: 0.002). I also have an average d' of 0.32. At first, I thought this was because I wasn't using the preprocessed temp_wh.dat data from kilosort. Below I'm showing the plot_spike_locations when I was using the raw data from Bonsai (all the units were stacked on a few points towards the base of the probe). I figured the lack of separation spatially was also causing issues for all my PCA-related metrics. I've since switched to using the preprocessed data in spikeinterface and at first glance (right side of image below), I thought this would have cleaned up my metrics. It has not - at least, not significantly.

I did further digging and picked out a few units in Phy that were mostly well isolated (from my mostly-untrained eye). I want to draw attention to the right two panels showing the spike positions along with the channels on the Neuropixel 1 that were picking up those units' activity (relatively low towards the tip of the probe).

I then tried to make a direct comparison of these 6 units based on the computed spike locations using spikeinterface (assuming these are comparable) and I'm getting something very strange. According to spike interface these are those same units (unit 28 and 126 are overlapping with 156 and 136, respectively).

unit_list = [28,59, 99, 126, 136, 156]
sw.plot_spike_locations(analyzer, unit_ids=unit_list, max_spikes_per_unit=None, unit_colors=colours, plot_all_units=False, plot_legend=True, backend="matplotlib", figsize=(6,8))

Any ideas why this might be and if it's possible this could be the tip of the iceberg for why some of my quality metrics on spikeinterface are a bit unusual? I thought it could be something to do with the probe file I'm importing, though, these seem to correspond quite closely. I've attached NP_probe.prb (used for kilosort; upload as a .txt) and np1-config.json (used with spikeinterface, which is the output from Open Ephys' Bonsai Onix package; https://open-ephys.github.io/bonsai-onix1-docs/articles/hardware/np1e/gui.html). The only other thought I had was the parameters for the PCA computation (e.g., whiten=True despite kilosort data already being pre-processed with whitening, CAR, drift correction etc.)?

np1-config.json
NP_probe.txt

Anyway, not sure what else it could be and I'm open to ideas; even the potential it's just a terrible recording. If it's helpful, my workflow is below. Thanks in advance!

---

Run raw data through kilosort with default settings (as per kilosort4 docs).

Load Probe.

probe_path = os.path.join(root_path, 'np1-config.json')
np1_config = probeinterface.io.read_probeinterface(probe_path)

Load kilosort folder.

sorting = read_phy(root_path + 'kilosort/')

Load recording.

ephys_recording = se.read_binary(
    file_paths=root_path + recording_path,
    sampling_frequency = 30000.0,
    num_channels=384,
    dtype=np.uint16,
    gain_to_uV=ap_scalar,
    offset_to_uV=-ap_offset
    )

Set probe to recording object.

ephys_recording.set_probe(np1_config.probes[0], group_mode='by_shank', in_place=True)

SortingAnalyzer: 384 channels - 948 units - 1 segments - binary_folder - sparse - has recording

Initialize Analyzer

analyzer = si.create_sorting_analyzer(
    sorting=sorting, 
    recording=recording, 
    format="binary_folder",
    folder=root_path + 'analyzer/',
    overwrite=True)

analyzer.compute("random_spikes", method="uniform", max_spikes_per_unit=1000, seed=2205) 
analyzer.compute("waveforms", ms_before=1.3, ms_after=2.6)
analyzer.compute("templates", operators=["average", "median", "std"])
analyzer.compute("noise_levels")
analyzer.compute("principal_components", n_components=3, mode="by_channel_global", whiten=True)
analyzer.compute("spike_amplitudes")
analyzer.compute("spike_locations")

Define metrics to run and their params (mostly left at default):

param_dict = {
    'num_spikes': {}, 
    'firing_rate': {}, 
    'presence_ratio': {'bin_duration_s': 60, 'mean_fr_ratio_thresh': 0.0}, 
    'snr': {'peak_sign': 'neg', 'peak_mode': 'extremum'}, 
    'isi_violation': {'isi_threshold_ms': 1.0, 'min_isi_ms': 0}, 
    'rp_violation': {'refractory_period_ms': 1.0, 'censored_period_ms': 0.0}, 
    'amplitude_cutoff': {'peak_sign': 'neg', 'num_histogram_bins': 100, 'histogram_smoothing_value': 3, 'amplitudes_bins_min_ratio': 5}, 
    'drift': {'interval_s': 60, 'min_spikes_per_interval': 10, 'direction': 'y', 'min_num_bins': 2}, 
    'nearest_neighbor': {'max_spikes': 1000, 'n_neighbors': 5}, 
    'isolation_distance':{},
    'd_prime':{}

Run metrics.

metrics = compute_quality_metrics(
    analyzer,
    metric_names=run_metrics,
    qm_params=param_dict,
    skip_pc_metrics=skip_pc
)

Start by comparing NP_probe.txt with np1-config.json and reviewing the workflow that loads the recording, sorting, and probe. Reproduce the mismatch with spike_locations and inspect the principal_components and quality-metric computations using the parameters shown. Done means identifying whether probe geometry, preprocessing, or metric configuration explains the discrepancy and documenting a concrete correction or confirmed recording issue.