Data race or race condition during exiting/joining a thread

This is a program which spawns N threads. Each thread will then, M times, spawn a thread and join the thread. This is intended to exercise concurrently spawning threads.

#include <pthread.h>
#include <stdio.h>
#include <assert.h>

#define N 10
#define M 10000

static void *another_child(void *arg) {
  return arg;
}

static void* child(void *arg) {
  pthread_t child;
  for (int i = 0; i < M; i++) {
    int rc = pthread_create(&child, NULL, another_child, NULL);
    if (rc != 0)
      perror("failed to create thread");
    rc = pthread_join(child, NULL);
    if (rc != 0)
      perror("failed to join thread");
  }
  return arg;
}

int main() {
  pthread_t children[N];

  for (int i = 0; i < N; i++) {
    int rc = pthread_create(&children[i], NULL, child, NULL);
    if (rc != 0)
      perror("failed to spawn thread");
  }

  for (int i = 0; i < N; i++) {
    int rc = pthread_join(children[i], NULL);
    if (rc != 0)
      perror("failed to join thread");
  }
  printf("ok!\n");

}

This is then run with:

$ clang foo.c -o foo.wasm -O2 --target=wasm32-wasi-threads -pthread -Wl,--import-memory,--export-memory,--stack-first,-zstack-size=$((10 << 20)),--max-memory=$((3 << 30))
$ wasmtime run --wasi-modules experimental-wasi-threads --disable-cache --wasm-features threads -- ./foo.wasm

This will produce no output and exit with status zero, but not expectedly. The ok! is not printed at the end of the main function. The reason for this appears to be that this exit is reached because a thread confuses itself as the last remaining thread.

I did a bit of poking to see if this could be fixed. The __tl_lock() function, for example, is aliased to a dummy_0 in many files. I wasn't able to fix it, however.

While looking around in this file, however, I also noticed that free is used to deallocate a thread stack which I don't believe is correct because the thread stack is in use by the thread calling free, which means that after the memory has been deallocated some stack management may still be happening which may modify the free'd memory. I haven't run into this myself, though, and commenting out the free did not fix my issue above.

cc @abrown

Start with libc-top-half/musl/src/thread/pthread_create.c, especially the thread-count exit path and stack deallocation cited in the report. Build and run the provided wasm32-wasi-threads reproducer under wasmtime, then trace the thread bookkeeping and join/exit behavior. Done means the stress program completes normally and prints "ok!" without unsafe stack cleanup.