FEX-Emu / FEX-Emu/FEX

execve self-reexec fallback loses caller-supplied argv[0]

Open
#5,879 3 comments 0 reactions 0 assignees View on GitHub
ml-report
Dominant language
C++
Stars
8k
Forks
351
Avg merge
12h 31m
Merged PRs (30d)
102

Description

**Disclosure: this description is generated with GPT-5.6 Sol**

## Summary

When a guest calls `execve(pathname, argv, envp)` with an `argv[0]` different from `pathname`, FEX may replace the caller-supplied value with the executable pathname.

This happens in the fallback path where `ExecveHandler` cannot hand the executable back to a visible `binfmt_misc` registration and instead re-executes `/proc/self/exe`.

Both syscall registration files already acknowledge the limitation:

```cpp
// currently does not propagate argv[0] correctly
```

The problem is observable in mount-namespace and sandbox environments where the host's fixed binfmt interpreter remains usable, but the corresponding `/proc/sys/fs/binfmt_misc/FEX-x86_64` entry is not visible inside the namespace.

One concrete environment where this occurs is a Nix sandbox using FEX as an `x86_64-linux` extra platform.

## Related work

- #5069 and #5070 handled writes to argv[0] for FEX's emulated
`/proc/self/cmdline`.
- #5097 remapped the host kernel's cmdline range to FEX's guest
argument storage using `PR_SET_MM_MAP`.
- #5274 made host-side glibc invocation-name globals derive from
`ApplicationArgs[0]` for thunked libraries.
- #1647 discusses adjacent Nix/shebang/RootFS path-selection behavior
in `ExecveHandler`.

None of these changes preserve a caller-supplied argv[0] when the
self-reexec fallback constructs a new `ApplicationArgs` vector.

## Expected behavior

Linux preserves the caller-provided argument vector independently from the executable pathname:

```c
char *const args[] = {
"custom-process-name",
"sentinel",
NULL,
};

execve("/path/to/helper", args, environ);
```

The new process should observe:

```text
argv[0] = custom-process-name
argv[1] = sentinel
```

The pathname used to locate and load the executable must not replace `argv[0]`.

## Actual behavior

When FEX takes its `/proc/self/exe` fallback, the new guest observes:

```text
argv[0] = /path/to/helper
argv[1] = sentinel
```

The caller-provided `custom-process-name` value is lost.

## Relevant code

On current `main`, the shared fallback in `ExecveHandler` constructs the self-reexec argument vector approximately as follows:

```cpp
fextl::vector ExecveArgs =
SyscallHandler->GetCodeLoader()->GetExecveArguments();

ExecveArgs.emplace_back(Filename.c_str());

auto OldArgv = argv;

if (*OldArgv) {
// Skip filename argument
++OldArgv;

while (*OldArgv) {
ExecveArgs.emplace_back(*OldArgv);
++OldArgv;
}
}
```

This combines two values that need to remain independent:

1. `pathname` / `Filename`, used to locate the next executable.
2. `argv[0]`, supplied by the guest and installed in the new process image.

The direct binfmt path can preserve both because the kernel's `P` flag provides the resolved executable and original `argv[0]` separately.

The internal `/proc/self/exe` fallback currently has no equivalent handoff.

Relevant locations:

- `Source/Tools/LinuxEmulation/LinuxSyscalls/Syscalls.cpp`
- `Source/Tools/LinuxEmulation/LinuxSyscalls/x64/Thread.cpp`
- `Source/Tools/LinuxEmulation/LinuxSyscalls/x32/Thread.cpp`
- `Source/Tools/FEXInterpreter/FEXInterpreter.cpp`
- `Source/Tools/FEXInterpreter/ELFCodeLoader.h`

## Concrete failure: Nix Python binary wrapper

This affects more than cosmetic process naming.

Nix's `python3.withPackages` executable is a binary wrapper around base CPython. The wrapper intentionally preserves its own `argv[0]` when it calls `execv()` so that CPython discovers the correct environment prefix and `site-packages`.

Inside the same sandbox, direct execution works:

```text
$ ...-python3-3.14.7-env/bin/python3 -c \
'import sys, elftools; print(sys.prefix); print(elftools.__file__)'

...-python3-3.14.7-env
...-python3-3.14.7-env/lib/python3.14/site-packages/elftools/__init__.py
```

But executing a Python script through its shebang eventually causes the wrapper to execute base CPython through FEX's fallback:

```text
$ ...-auto-patchelf/bin/auto-patchelf --help

Traceback (most recent call last):
File "...-auto-patchelf/bin/auto-patchelf", line 18, in
from elftools.common.exceptions import ELFError
ModuleNotFoundError: No module named 'elftools'
```

Instrumentation shows that the resulting CPython process sees the base interpreter rather than the environment wrapper:

```text
sys.executable = ...-python3-3.14.7/bin/python3.14
sys.prefix = ...-python3-3.14.7
```

The environment's `site-packages` is therefore absent from `sys.path`.

Setting `PYTHONPATH` explicitly works around the failure, but the underlying problem is the lost `argv[0]`.

## Why the fallback is selected

The common path for a supported x86 ELF is:

```text
guest execve
-> host execveat
-> binfmt_misc
-> new FEX process
```

That path preserves `argv[0]`.

In a sandbox or mount namespace, FEX may be unable to see the expected registration under:

```text
/proc/sys/fs/binfmt_misc/FEX-x86_64
```

However, the host registration can still be active and its interpreter can remain available because the binfmt `F` flag pins it across mount namespaces.

FEX then considers the interpreter unavailable and uses:

```text
guest execve
-> execveat("/proc/self/exe", ...)
-> new FEX process
```

That second path loses the independent guest `argv[0]`.

## Suggested minimal reproducer

A regression test can use two guest binaries.

Driver:

```c
#include
#include
#include
#include

extern char **environ;

int main(int argc, char **argv) {
if (argc != 2) {
fprintf(stderr, "usage: %s /path/to/helper\n", argv[0]);
return 2;
}

char *const child_argv[] = {
(char *)"custom-process-name",
(char *)"sentinel",
NULL,
};

execve(argv[1], child_argv, environ);
perror("execve");
return errno;
}
```

Helper:

```c
#include
#include

int main(int argc, char **argv) {
if (argc != 2) {
fprintf(stderr, "unexpected argc: %d\n", argc);
return 1;
}

if (strcmp(argv[0], "custom-process-name") != 0) {
fprintf(stderr, "unexpected argv[0]: %s\n", argv[0]);
return 1;
}

if (strcmp(argv[1], "sentinel") != 0) {
fprintf(stderr, "unexpected argv[1]: %s\n", argv[1]);
return 1;
}

return 0;
}
```

The test must force the no-visible-binfmt/self-reexec path. Running it through a path that delegates back to a visible binfmt registration will not reproduce the bug.

## Proposed fix

The self-reexec protocol needs to carry the guest `argv[0]` separately from the executable pathname.

One possible implementation is an internal, removed-on-read environment handoff analogous to `FEX_EXECVEFD` and `FEX_SECCOMPFD`.

For example:

```text
FEX_EXECVEARGV0=
```

### Exec side

In `ExecveHandler`, when taking the `/proc/self/exe` fallback for a non-shebang ELF:

1. Keep `Filename` in `ExecveArgs` so the new FEX process can locate the executable.
2. Copy the caller-supplied `argv[0]` into an owned internal handoff.
3. Copy `envp` if necessary and add the private handoff entry.
4. Re-execute `/proc/self/exe` as today.

Conceptually:

```cpp
if (!IsShebang && argv && argv[0]) {
GuestArgv0Env =
fextl::fmt::format("FEX_EXECVEARGV0={}", argv[0]);

EnvpArgs.emplace_back(GuestArgv0Env.data());
}
```

This needs the same lifetime and environment-copy care as the existing FD and seccomp handoffs.

### Interpreter side

In `FEXInterpreter.cpp`:

1. Read and remove the internal handoff at startup.
2. Use the normal argument list to discover the executable pathname and application configuration.
3. After target discovery, but before constructing `ELFCodeLoader`, restore the carried value as the guest's `Args[0]`.
4. Do not expose the private variable to the guest.

Conceptually:

```cpp
auto GuestArgv0 =
StealFEXStringFromEnv("FEX_EXECVEARGV0");

auto Program =
FEX::Config::GetApplicationNames(
Args,
ExecutedWithFD,
FEXFD);

if (GuestArgv0 && !IsShebang) {
Args[0] = *GuestArgv0;
}
```

The actual location of the restoration may need to account for `InterpreterHandler` and configuration loading.

The important invariant is:

```text
Program.ProgramPath == pathname used to load the ELF
ApplicationArgs[0] == caller-supplied argv[0]
```

These values must not be derived from one another.

## Shebang behavior

The fix should not blindly restore the script caller's `argv[0]` after shebang processing.

Native Linux constructs the interpreter argument vector approximately as:

```text
interpreter argv[0]
optional shebang argument
script pathname
original argv[1..]
```

The script caller's original `argv[0]` is intentionally discarded.

This behavior should remain unchanged.

The Nix/Python failure still benefits from the proposed non-shebang fix:

1. The shebang correctly starts the Nix Python wrapper.
2. The wrapper calls `execv(base_python, argv)` while preserving the wrapper path as `argv[0]`.
3. That second operation targets a normal ELF.
4. The current fallback drops the wrapper path.
5. Preserving `argv[0]` for the ELF fallback lets CPython discover the intended environment.

## Alternative handoff designs

An internal command-line option could carry the value instead:

```text
FEX --internal-argv0 ... executable ...
```

However, this expands the argument parser's private surface and needs careful option termination and escaping.

A private environment handoff follows existing FEX mechanisms, but I am open to a different internal representation.

A marker plus a duplicated command-line entry may also avoid storing arbitrary argument data in an environment variable:

```text
FEX
```

The new interpreter would need an unambiguous internal marker telling it to consume the second entry as preserved guest `argv[0]`.

## Tests requested

The regression suite should cover both x86-64 and x86-32:

- `execve()` with `argv[0] != pathname`.
- Pathname-based `execveat()`.
- `argv == NULL`.
- `argv[0] == NULL`.
- `argv[0] == ""`.
- Arguments containing spaces and `=`.
- The private handoff not leaking into the guest environment.
- The no-visible-binfmt/self-reexec path specifically.
- Normal direct-binfmt execution remaining unchanged.
- Native-compatible shebang argument ordering.
- Shebang interpreter followed by an ELF wrapper that preserves a custom `argv[0]`.

A test that only runs with a visible binfmt registration is insufficient because it can bypass the affected fallback.

## Compatibility considerations

- Some applications may accidentally rely on the fallback replacing `argv[0]` with the executable pathname. Preserving the caller-provided value would restore native Linux behavior but may expose such dependencies.
- Shebang behavior is the main regression risk.
- An environment handoff temporarily duplicates the argument and consumes additional `ARG_MAX` space.
- The internal key must be overwritten or removed safely if the guest already supplied a variable with the same name.
- Empty and null argument-vector cases must remain distinct where the kernel/FEX currently distinguishes them.
- Both x86-32 and x86-64 syscall wrappers use the shared handler and should receive equivalent coverage.

## Proposed patch structure

Suggested commits:

```text
FEXLinuxTests: cover argv[0] across emulated execve
LinuxSyscalls: preserve argv[0] across FEX self-reexec
```

An optional cleanup can remain separate:

```text
FEXInterpreter: generalize private exec handoff parsing
```

## Environment where this was observed

```text
Host architecture: aarch64
Guest architecture: x86_64
FEX version: FEX-2608
Kernel page size: 4096
binfmt flags: POCF
Nix mode: x86_64-linux in extra-platforms
Nix sandbox: enabled
```

The same FIXME and fallback behavior are still present on FEX `main` as of commit:

```text
98964c552773b374676610776357a030a6825e53
```

Would an internal handoff like the above be acceptable, or is there a preferred way to represent a guest executable pathname and guest `argv[0]` independently across FEX self-reexec?

Contributor guide

Open the contributing guide

Research direction

Start by tracing the shared fallback and existing argv[0] FIXME in Source/Tools/LinuxEmulation/LinuxSyscalls/Syscalls.cpp, then follow startup argument handling in FEXInterpreter.cpp and ELFCodeLoader.h. Compare the x64 and x32 Thread.cpp paths and build the requested driver/helper regression case while forcing the no-visible-binfmt path. Done means non-shebang execve and execveat preserve caller-supplied argv[0] without leaking internal state, while argv edge cases and shebang behavior remain compatible.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.