NVIDIA / NVIDIA/open-gpu-kernel-modules

Rocky Linux 8.8 crash

Open
#563 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
17.4k
Forks
1.9k
PR merge metrics
No merged PRs in 30d

Description

NVIDIA Open GPU Kernel Modules Version

525.47.04

Please confirm this issue does not happen with the proprietary driver (of the same version). This issue tracker is only for bugs specific to the open kernel driver.
  • I confirm that this does not happen with the proprietary driver package.
Operating System and Version

Rocky LInux 8.8

Kernel Release

linux-4.18.0-477.10.1.el8_8

Please confirm you are running a stable release kernel (e.g. not a -rc). We do not accept bug reports for unreleased kernels.
  • I am running on a stable kernel release.
Hardware: GPU

GPU 0:Tesla T4

Describe the bug

Pin and Unpin pages lead to host crash.
Nvidia driver will lock user pages by cuda API such as memory registry.
If pin pages failed by pin_user_pages for some reasons,it pin page using get_page which inc ref by 1.
However, driver will call unpin_user_page when unlock user pages regardless of page type,in this case, page ref will be decreased by 1024 not 1.
Host crashed because page ref less than zero.
it is reasonnable to record pages that pinned failed but inc by 1,then call put_page other than unpin_user_pages.

To Reproduce
  1. Map a page to file which is a special one.
  2. .register this page by CUDA API.
  3. Run TF benchmarks.
Bug Incidence

Always

nvidia-bug-report.log.gz

nvidia-bug-report.log.gz

More Info

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pin_user_pages, get_page, and unpin_user_page paths involved when CUDA registers a mapped special file page. Reproduce on Rocky Linux 8.8 with the Tesla T4 using the reported CUDA registration flow and TensorFlow benchmarks. Done means the host no longer crashes and page references remain balanced when pinning fails.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux
Domain
operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.