apache / apache/nuttx

Unexpected behaviour with poll and tickless scheduler

Open
#11,189 10 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
4k
Forks
1.7k
Avg merge
1d 17h
Merged PRs (30d)
237

Description

Hello,

I'm using poll in conjunction with timerfds and the `POLLIN` event to schedule actions at different rates. I'm finding that when `CONFIG_SCHED_TICKLESS` is used, poll is returning with only a subset of my file descriptors in revents field, even though I'm specifying the timeout to occur at the same time.

Here's a working example which demonstrates the issue, in which I:

- Setup 8 timer file descriptors to have their first expiry after 3 seconds, and then expire at a 1 second rate.
- Poll continuously for `POLLIN` events.
- Report the expired events and print which fds expired as a hex value.

```c
#include
#include
#include
#include
#include
#include
#include

#define TIMERFD_COUNT 8

static int fds[TIMERFD_COUNT] = {-1};

int create_timers(void)
{
struct itimerspec its;

clock_gettime(CLOCK_MONOTONIC, &its.it_value);

// Set initial alarm for 3 seconds in the future
its.it_value.tv_sec += 3;
its.it_value.tv_nsec = 0;

// Set interval to 1 second
its.it_interval.tv_sec = 1;
its.it_interval.tv_nsec = 0;

for(int i = 0; i < TIMERFD_COUNT; i++)
{
fds[i] = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
if(fds[i] < 0)
{
return fds[i];
}

if (timerfd_settime(fds[i], TFD_TIMER_ABSTIME, &its, NULL) < 0)
{
return errno;
}
}

return 0;
}

void cleanup(void)
{
for(int i = 0; i < TIMERFD_COUNT; i++)
{
if(fds[i] > 0)
{
close(fds[i]);
}
}
}

void timer_read(int fd)
{
uint64_t expirations;
read(fd, &expirations, sizeof(expirations));
}

size_t wait(void)
{
struct pollfd pfds[TIMERFD_COUNT];
const int timeout_ms = 10000;
int ret;
size_t mask = 0;

for(int i = 0; i < TIMERFD_COUNT; i++)
{
pfds[i].fd = fds[i];
pfds[i].events = POLLIN;
}

ret = poll(pfds, TIMERFD_COUNT, timeout_ms);
if(ret < 0)
{
exit(EXIT_FAILURE);
}

for (int i = 0; i < TIMERFD_COUNT; i++)
{
if (pfds[i].revents == POLLIN)
{
mask |= (1 << i);
timer_read(pfds[i].fd);
}
}
return mask;
}

int main(int argc, char *argv[])
{
size_t mask;
int ret = create_timers();
atexit(cleanup);

if(ret < 0)
{
printf("Cannot create timers: %d\n", ret);
exit(EXIT_FAILURE);
}

printf("Start poll\n");
while(1)
{
mask = wait();
printf("mask 0x%zX\n", mask);
}

exit(EXIT_SUCCESS);
}
```

**Expected output.**

I've tried the following example on the following platforms:

- Linux: with `gcc poll_test_main.c -o poll_test && ./poll_test`
- `sim:nsh` with `CONFIG_TIMER_FD` added.
- `qemu-rv:knsh32`
- `arty_a7:knsh`

In each case I get:

```
nsh> poll
Start poll
mask 0xFF
mask 0xFF
mask 0xFF
mask 0xFF
mask 0xFF
mask 0xFF
mask 0xFF
```
Which is what I am expecting.

However, when I use a tickless scheduler, such as:

- `qemu-armv8a:nsh_smp_tickless`
- `arty_a7:knsh-tickless`

I get the following:

```
nsh> poll
Start poll
mask 0x80
mask 0x70
mask 0xF
mask 0x80
mask 0x70
mask 0xF
mask 0x80
mask 0x70
mask 0xF
mask 0x80
mask 0x78
mask 0x7
```

In this case poll needs to be called 2-3 times for all expiry events to be returned. It doesn't seem to miss any events, they just are not all returned at the same time.

I haven't looked into this too much yet, but I feel like the underlying cause may also be what is causing the issue in #9840.

If there's any interest, I can add this example program, as well as any additions needed to the mentioned default configurations in our NuttX fork, to ease testing.

I would appreciate and feedback or suggestions.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.