intel / intel/libva

Leaking DRI fds when vaInitialize() is called again on same VADisplay after failure

Open
#741 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
C
Stars
784
Forks
314
PR merge metrics
No merged PRs in 30d

Description

`vaInitialize()` doesn't clean up properly on failure, so subsequent calls to `vaInitialize()` will lead to leaks of a `/dev/dri/card` or `/dev/dri/renderD` file descriptor. Eventually the fd limit is reached and no more files can be opened, leading to a crash with a message similar to:
```
XIO: fatal IO error 24 (Too many open files) on X server ":0"
after 6 requests (6 known processed) with 0 events remaining.
```

Attempting to call `vaInitialize()` again after a failure might seem contrived at first glance, but it is a reasonable thing to do for clients that might be using `vaSetDriverName()` to try to get one driver over another.

Either the documentation should clearly state that `vaTerminate()` must be called even after a _failed_ `vaInitialize()` call (which invalidates the whole `VADisplay`), or a failed `vaInitialize()` should leave the `VADisplay` in a clean state to allow another initialization attempt.

Reproducer:
```
gcc valeak.c -o valeak -lva -lva-x11 -lX11
```
```
#include
#include
#include

#include
#include

int main() {
Display *x11_disp = XOpenDisplay(NULL);
if (!x11_disp) {
fprintf(stderr, "Unable to open X11 display\n");
return -1;
}

VADisplay va_disp = vaGetDisplay(x11_disp);
if (!va_disp) {
fprintf(stderr, "Unable to open VA display\n");
return -1;
}

for (;;) {
int major, minor;
vaSetDriverName(va_disp, "NotARealDriver");
VAStatus status = vaInitialize(va_disp, &major, &minor);
if (status == VA_STATUS_SUCCESS) {
fprintf(stderr, "vaInitialize() succeded with bogus driver name?\n");
abort();
}
else {
fprintf(stderr, "vaInitialize() failed: %s\n", vaErrorStr(status));
}
}

vaTerminate(va_disp);
XCloseDisplay(x11_disp);
}
```

Running `lsof` on the reproducer will show hundreds of leaked `/dev/dri` fds:
```valeak 30432 cgutman 547u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 548u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 549u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 550u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 551u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 552u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 553u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 554u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 555u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 556u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 557u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 558u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 559u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 560u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 561u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 562u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 563u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 564u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 565u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 566u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 567u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 568u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 569u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 570u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 571u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 572u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 573u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 574u CHR 226,128 0t0 571 /dev/dri/renderD128
valeak 30432 cgutman 575u CHR 226,128 0t0 571 /dev/dri/renderD128
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.