getsentry / getsentry/sentry-javascript
Lambda extension stops polling after a 300s invocation, hanging every later invocation on that execution environment
- Dominant language
- TypeScript
- Stars
- 8.7k
- Forks
- 1.8k
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 523
Description
### Which SDK are you using?
`@sentry/aws-serverless`
### SDK Version
10.36.0 (Lambda layer `SentryNodeServerlessSDKv10:49`). Also present on `develop` (10.73.0,
layer `:88`) — the extension source is unchanged, so this is not fixed by upgrading.
### Framework Version
AWS Lambda, `nodejs22.x`
### Steps to Reproduce
No AWS account needed — the official runtime image runs external extensions.
```bash
cd packages/aws-serverless && yarn build:extension
mkdir -p /tmp/repro/task /tmp/repro/opt/extensions /tmp/repro/opt/sentry-extension
cat > /tmp/repro/task/index.js <<'EOF'
exports.handler = async event => {
await new Promise(r => setTimeout(r, Number(event.sleepMs || 0)));
return { ok: true };
};
EOF
cp build/lambda-extension/sentry-extension /tmp/repro/opt/extensions/
cp build/lambda-extension/index.mjs /tmp/repro/opt/sentry-extension/
# AWS_LAMBDA_FUNCTION_TIMEOUT matters: the emulator defaults to 300s, the same number
# under test, and a function timeout at 300s masks the bug entirely.
docker run -d --name repro -p 9000:8080 \
-e AWS_LAMBDA_FUNCTION_TIMEOUT=900 \
-v /tmp/repro/task:/var/task:ro -v /tmp/repro/opt:/opt:ro \
public.ecr.aws/lambda/nodejs:22 index.handler
URL=http://localhost:9000/2015-03-31/functions/function/invocations
curl -s -XPOST $URL -d '{"sleepMs":310000}' # {"ok":true} after ~310s
curl -s --max-time 60 -XPOST $URL -d '{"sleepMs":100}' # never returns
docker logs repro
```
### Expected Result
Both invocations return.
### Actual Result
The second invocation's handler finishes in ~110ms and the response never comes:
```
handler: done
INVOKE RTDONE(status: success, produced bytes: 0, duration: 107.806000ms)
<- nothing after this
```
`External agent sentry-extension ... registered` appears exactly once, so both invocations ran
on the same execution environment — the first one poisoned it. On real Lambda this ends as
`Status: timeout` with the handler long finished, and `PostRuntimeExtensionsDuration` equal to
the full function timeout.
Production numbers from one SQS worker (600s timeout, 155,083 invocations over 7 days):
| statistic | value |
| --- | --- |
| `PostRuntimeExtensionsDuration` p50 / p90 / p99.9 | 0 / 0 / **0.04 ms** |
| maximum | **599,949.55 ms** |
37 invocations hit `Status: timeout` in that window, against 33 that ran past 300s and did not
— about one poisoned environment each, which is what the mechanism predicts.
### Additional Context
**Mechanism.** `/event/next` both acknowledges the previous event and waits for the next one,
so the poll issued after each event stays open for the whole of the following invocation.
Node's `fetch` applies undici's `headersTimeout`, default 300,000 ms — measured against a
server that accepts the connection and never sends headers:
`rejected after 301.0 s -> UND_ERR_HEADERS_TIMEOUT`. Frozen time does not count against it
(undici's clock is tick-driven, `fastNow += TICK_MS`), so only a genuinely long invocation
triggers this, which is why the cliff sits exactly at 300s.
The rejection then escapes the loop, which has no `try`/`catch`:
```js
while (true) { await extension.next(); }
main().catch(err => { DEBUG_BUILD && debug.error('Error in Lambda Extension', err); });
```
**Why it is silent.** Lambda ends an invocation only once the runtime *and every registered
extension* have asked for the next event, so the long invocation itself completes normally and
later ones hang with nothing logged. And `debug` is only enabled from `Sentry.init`, which this
separate process never calls — `debug.error` here cannot print under any option or env var.
**Scope.** The extension is enabled by default for layer users, and `useLayerExtension: false`
does not help: the layer ships the binary in `/opt/extensions/`, which Lambda starts regardless
of SDK options. Reproduced identically with the published `SentryNodeServerlessSDKv10:88` layer,
and does not reproduce with a patched extension.
Contributor guide
Assessment
This issue has not been assessed yet.