AllenNeuralDynamics / AllenNeuralDynamics/aind-ephys-pipeline

ERROR on local (real) data file

未关闭
#61 4 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Nextflow
星标
41
派生
10
平均合并
25 分钟
30 天内合并 PR
2

描述

Expanding beyond the attempts to use the sample NWB data, I have some real data I tried to run this pipeline on...

I encounter the following error early in the process

```bash
N E X T F L O W ~ version 25.04.8

Launching `main_multi_backend.nf` [soggy_cajal] DSL2 - revision: 780c3e867c

DATA_PATH: /mnt/e/datalad/OdorSequence/code/aind-ephys-pipeline/pipeline/../../../sourcedata/nwb-raw/sub-M541/ses-2024-08-31/
RESULTS_PATH: /mnt/e/datalad/OdorSequence/code/aind-ephys-pipeline/pipeline/../../../derived/results
PARAMS: [n_jobs:8, executor:local, ecephys_path:/mnt/e/datalad/OdorSequence/code/aind-ephys-pipeline/pipeline/../../../sourcedata/nwb-raw/sub-M541/ses-2024-08-31/, params_file:null]
CONTAINER TAG: si-0.103.0
N JOBS: 8
Using RUNMODE: full
No parameters file provided, using command line arguments.
No sorter specified, defaulting to kilosort4
Using SORTER: kilosort4 with args:
executor > local (1)
[65/b57e82] job_dispatch (job-dispatch) | 0 of 1
[- ] preprocessing -
[- ] spikesort_kilosort4 -
[- ] postprocessing -
[- ] curation -
[- ] visualization -
[- ] results_collector -
[- ] quality_control -
[- ] quality_control_collector -
[- ] nwb_ecephys -
[- ] nwb_units -
ERROR ~ Error executing process > 'job_dispatch (job-dispatch)'

Caused by:
Process `job_dispatch (job-dispatch)` terminated with an error exit status (128)

Command executed:

#!/usr/bin/env bash
set -e

mkdir -p capsule
mkdir -p capsule/data
mkdir -p capsule/results
mkdir -p capsule/scratch

if [[ local == "slurm" ]]; then
echo "[job-dispatch] allocated task time: null"
fi

TASK_DIR=$(pwd)

echo "[job-dispatch] cloning git repo..."
git clone "https://github.com/AllenNeuralDynamics/aind-ephys-job-dispatch.git" capsule-repo
git -C capsule-repo -c core.fileMode=false checkout 82103472153da7ca25a36f48173e52826825e8cf --quiet
executor > local (1)
[65/b57e82] job_dispatch (job-dispatch) | 0 of 1 ✘
[- ] preprocessing -
[- ] spikesort_kilosort4 -
[- ] postprocessing -
[- ] curation -
[- ] visualization -
[- ] results_collector -
[- ] quality_control -
[- ] quality_control_collector -
[- ] nwb_ecephys -
[- ] nwb_units -
ERROR ~ Error executing process > 'job_dispatch (job-dispatch)'

Caused by:
Process `job_dispatch (job-dispatch)` terminated with an error exit status (128)

Command executed:

#!/usr/bin/env bash
set -e

mkdir -p capsule
mkdir -p capsule/data
mkdir -p capsule/results
mkdir -p capsule/scratch

if [[ local == "slurm" ]]; then
echo "[job-dispatch] allocated task time: null"
fi

TASK_DIR=$(pwd)

echo "[job-dispatch] cloning git repo..."
git clone "https://github.com/AllenNeuralDynamics/aind-ephys-job-dispatch.git" capsule-repo
git -C capsule-repo -c core.fileMode=false checkout 82103472153da7ca25a36f48173e52826825e8cf --quiet
mv capsule-repo/code capsule/code
rm -rf capsule-repo

echo "[job-dispatch] running capsule..."
cd capsule/code
chmod +x run
./run

MAX_DURATION_MIN=$(python get_max_recording_duration_min.py)

cd $TASK_DIR
echo "$MAX_DURATION_MIN" > max_duration.txt

echo "[job-dispatch] completed!"

Command exit status:
128

Command output:
[job-dispatch] cloning git repo...

Command error:
WARNING: Your kernel does not support memory swappiness capabilities or the cgroup is not mounted. Memory swappiness discarded.
[job-dispatch] cloning git repo...
Cloning into 'capsule-repo'...
fatal: detected dubious ownership in repository at 'capsule-repo'
To add an exception for this directory, call:

git config --global --add safe.directory capsule-repo

Work dir:
/mnt/e/datalad/OdorSequence/code/aind-ephys-pipeline/pipeline/work/65/b57e82b8dcc3bb527aab6ff0acebb8

Container:
ghcr.io/allenneuraldynamics/aind-ephys-pipeline-base:si-0.103.0

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

-- Check '/mnt/e/datalad/OdorSequence/code/aind-ephys-pipeline/pipeline/../../../derived/results/nextflow/nextflow.log' file for details
```

Normally I would set the `git config` to declare the repo as safe but this occurs automatically in the pipeline which is in a new work directory on every run. I also tried the local deployment through my own fork of this repo, but I suppose the things it is cloning do not know to adjust to my own forks of things?

Will continue to try to hack away to get this to work

cc @yarikoptic

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。