mlcommons / mlcommons/inference

Nvidia 4.1 inference code is giving segmentation fault for RTX 4090 (4.0 code works fine)

Open
#1,847 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.6k
Forks
650
Avg merge
1d 22h
Merged PRs (30d)
6

Description

Trying to run Nvidia v4.1 implementation for stable diffusion on RTX 4090.

(mlperf) arjun@mlperf-inference-arjun-x86-64-24944:/work$ make generate_engines RUN_ARGS="--benchmarks=stable-diffusion-xl --scenarios=Offline"
make: *** [Makefile:37: generate_engines] Segmentation fault (core dumped)
make download_model BENCHMARKS="stable-diffusion-xl"

ran successfully and produced int8 model. Below are the custom configs used for 2x RTX4090.

class SPR(OfflineGPUBaseConfig):
    system = KnownSystem.spr

    # Applicable fields for this benchmark are listed below. Not all of these are necessary, and some may be defined in the BaseConfig already and inherited.
    # Please see NVIDIA's submission config files for example values and which fields to keep.
    # Required fields (Must be set or inherited to run):
    gpu_batch_size = {'clip1': 32 * 2, 'clip2': 32 * 2, 'unet': 32 * 2, 'vae': 1}
    offline_expected_qps: float = 1.0
    precision: str = 'int8'

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing make generate_engines RUN_ARGS="--benchmarks=stable-diffusion-xl --scenarios=Offline" and inspect the Makefile:37 generate_engines target. Compare the Nvidia v4.1 stable-diffusion engine-generation path with the v4.0 implementation that works on RTX 4090, using the supplied 2x RTX 4090 configuration. Done means v4.1 completes engine generation without a segmentation fault.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.