Azure / Azure/iot-hub-device-update

SIGSEGV on every failed startup: uninit_api_svc joins a thread that was never created

Open Beginner friendly
#911 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
61
Forks
54
Avg merge
2d 15h
Merged PRs (30d)
1

Description

### Summary
`AducIotAgent` crashes with `SIGSEGV` on **every failed startup**: whenever
`HealthCheck()` fails, the agent dies in the shutdown path instead of exiting with
a non-zero code. Seen on aarch64, 1.4.0; the same code is on `develop`.

Log right before the crash:
```
[E] Invalid connection info. [HealthCheck:630]
[E] Agent health check failed. [main:1161]
→ SIGSEGV, si_code=1 (SEGV_MAPERR), fault address 0xd0
```
Because the unit restarts the agent after a failure, this repeats until the
systemd start limit stops the service — on our devices 10 cores in 60 s while the
identity service could not hand out connection info.

### Root cause
`uninit_api_svc()` (`src/agent/api/src/apisvc.c`) joins `g_api_svc_thread`
unconditionally:

```c
atomic_store(&g_api_svc_thread_running, false);
int res = pthread_join(g_api_svc_thread, (void**)&threadRet);
```

`g_api_svc_thread` is `pthread_t g_api_svc_thread = { 0 };` and is only ever set by
`init_api_svc()`. On the health-check failure path `init_api_svc()` never runs:
`main()` does `goto done` before `StartupAgent()`, and `done:` calls
`ShutdownAgent()`, which calls `uninit_api_svc()`. So `pthread_join()` gets a NULL
thread descriptor.

glibc's join implementation dereferences the descriptor to validate it. On aarch64
that faults at `NULL + 0xd0`; x86-64 glibc happens to check the handle for NULL and
returns `ESRCH`, which is why the crash only shows on arm.

Evidence from a device core (two cores, identical):
- faulting instruction `ldr w0, [x0, #208]` with `x0 = 0`, inside the join
implementation in libc
- return address points right after `bl pthread_join@plt` in `uninit_api_svc()`

The analogous timer thread teardown already guards against this
(`_stop_thread()` in `src/utils/timer_utils/src/timer.c` only joins when
`threadCreated`); the API service thread does not.

### Reproduces when
Any shutdown that was not preceded by a successful `StartupAgent()`, i.e. every
`HealthCheck()` failure (identity service unavailable, unprovisioned device, bad
connection info). Deterministic on arm.

### Fix
Take the running flag with `atomic_exchange()` in `uninit_api_svc()` and skip the
teardown when it was already false; this also makes a repeat uninit a no-op. PR
attached.

Contributor guide

Open the contributing guide

Research direction

Start in src/agent/api/src/apisvc.c and trace main()'s done: path through ShutdownAgent() into uninit_api_svc(); compare its thread teardown with _stop_thread() in src/utils/timer_utils/src/timer.c. Reproduce a HealthCheck() failure and verify the agent exits nonzero without SIGSEGV, including safe repeated teardown.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
embedded-iot, operating-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.