microsoft / microsoft/vscode-cpptools

Debugger fails under Linux if file size limits are in effect

Open
#13,768 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug debugger help wanted
Dominant language
TypeScript
Stars
6.2k
Forks
1.7k
Avg merge
14h 46m
Merged PRs (30d)
61

Description

Environment
  • OS and version: AlmaLinux 8.10
  • VS Code: 1.102.1 (and earlier)
  • C/C++ extension: ms-vscode.cpptools-1.23.5-linux-x64 and later
  • OS and version of remote machine (if applicable):
  • GDB / LLDB version: gdb-8.2-20.el8.x86_64
Bug Summary and Steps to Reproduce

Bug Summary:
LIke many sites, we set a default file size ulimit to prevent users' runaway processes from consuming unlimited disk resources. When attempting to debug a C program with any file size limit less than 2TiB in place (cf. "grep 'Max file size' /proc/$$/limits"), the debugger crashes.

Steps to reproduce:

  1. Set a reasonable file size limit (e.g., "ulimit -f 25165824" from bash)
  2. Launch "code" from the shell prompt
  3. Open a .c file, build it, and set a breakpoint
  4. Select Run -> Start debugging

The debugger controls will appear briefly and disappear; the OpenDebugAD7 process will crash with SIGXFSZ.

Debugger Configurations
Default settings.
Debugger Logs
No messages in Debug Console or Output windows.
Other Extensions

No response

Additional Information

Attaching a system call tracer reveals that the process is attempting to truncate the file returned from a memfd_create("doublemapper") call to 2TB:

1354933 memfd_create("doublemapper", MFD_CLOEXEC) = 8
1354933 ftruncate(8, 2199023255552) = -1 EFBIG (File too large)

This may have the same root cause as https://github.com/microsoft/vscode/issues/251037 .

Setting "C_Cpp_Runner.debuggerPath": "lldb" in settings.json may work around the issue (OpenDebugAD7 does not appear to run in this configuration), but this is not the default configuration.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the crash with the stated ulimit and trace the OpenDebugAD7 process, focusing on the memfd_create("doublemapper") and ftruncate call shown in the report. Compare the default debugger path with the documented LLDB workaround; done means starting a C debug session under a file-size limit without SIGXFSZ or a disappearing debugger.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, linux, vscode
Domain
devtools, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.