Running carla headless evaluation on determined AI cluster and failed due to segmentation fault(core dumped)

Open
#199 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
bash, python

Research direction

Start with b2d_scripts/run_evaluation_debug.sh and reproduce the command on the Determined AI cluster, noting that model loading succeeds before the crash during route execution. Check the CARLA launch output, CUDA initialization, and the reported GPU usage to isolate the segmentation fault. Done means the evaluation completes a route without the segmentation fault, or the issue identifies a reproducible root cause.

Written by the indexing model from the issue text.

Description

I run the bash file run_evaluation_debug.sh and get outputs as the following
_(simlingo) zhenwei@4dc93ea82e7a:/workspace$ bash b2d_scripts/run_evaluation_debug.sh
/workspace/software/carla0915/CarlaUE4.sh -RenderOffScreen -nosound -carla-rpc-port=30000 -graphicsadapter=0 None
4.26.2-0+++UE4+Release-4.26 522 0
Disabling core dumps.
sh: 1: xdg-user-dir: not found
WARNING: lavapipe is not a conformant vulkan implementation, testing use only.
load_world success , attempts=0
traffic_manager init success, try_time=0

========= Preparing RouteScenario_2286 (repetition 0) =========

Loading the world
Setting up the agent
CUDA initialized. GPU: NVIDIA GeForce RTX 4090
path to conf file: /workspace/checkpoints/epoch=013.ckpt/pytorch_model.pt+RouteScenario_2286_rep0_Town12_HighwayCutIn_1_18_09_23_10_44_35
Config path: /workspace/checkpoints/epoch=013.ckpt/pytorch_model.pt
Save path root: RouteScenario_2286_rep0_Town12_HighwayCutIn_1_18_09_23_10_44_35
Loading model...
Using /workspace/pretrained/InternVL2-1B as the image encoder.
Using PEFT model
trainable params: 17,596,416 || all params: 647,260,288 || trainable%: 2.7186
Loading model weights...
Model loaded successfully
===== Debug mode is ON. Saving images to: ./eval_v1/RouteScenario_2286_rep0_Town12_HighwayCutIn_1_18_09_23_10_44_35/debug_viz/workspace/iter_013.ckpt/Bench2Drive/bench2drive220_2025_09_23_10_46_10/images =====
Running the route
Segmentation fault (core dumped)_

I suppose it might not be mainly due to not enough memory because my command "gpustat" comes
"(simlingo) zhenwei@4dc93ea82e7a:~$ gpustat
4dc93ea82e7a Tue Sep 23 10:46:39 2025 535.183.01
[0] NVIDIA GeForce RTX 4090 | 38°C, 0 % | 4896 / 24564 MB |
"
and all the memory used was for loading my model.

Dominant language
Python
Stars
1.9k
Forks
144
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Thinklab-SJTU/Bench2Drive

All issues in Thinklab-SJTU/Bench2Drive

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.