S3Client missing requestHandler timeout configuration for external S3 storage
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 6h 51m
- Merged PRs (30d)
- 232
Description
Bug Description
Bug Description
When using external S3-compatible storage (like Wasabi) for session recordings, uploads fail with RequestTimeout errors even when SESSION_RECORDING_V2_S3_TIMEOUT_MS is set to a high value (e.g., 600000ms).
How to reproduce
Connect an external S3 to a self-hosted PostHog
Root Cause
The SESSION_RECORDING_V2_S3_TIMEOUT_MS environment variable is correctly read by the config (verified via node -e "require('/code/nodejs/dist/config/config.js').defaultConfig.SESSION_RECORDING_V2_S3_TIMEOUT_MS"), but it's only used for a JavaScript setTimeout in s3-session-batch-writer.js, NOT for the AWS SDK's socket/request timeout.
In /code/nodejs/dist/session-recording/consumer.js (lines 78-89), the S3Client is created without a requestHandler:
const s3Config = {
region: hub.SESSION_RECORDING_V2_S3_REGION,
endpoint: hub.SESSION_RECORDING_V2_S3_ENDPOINT,
forcePathStyle: true,
};
// ...
s3Client = new client_s3_1.S3Client(s3Config);
The AWS SDK v3 has its own default timeouts which are too short for external S3 services with higher latency.
Error Logs
"error":{"name":"RequestTimeout","$fault":"client","$metadata":{"httpStatusCode":400,"attempts":3,"totalRetryDelay":28},"message":"Your socket connection to the server was not read from or written to within the timeout period"}
"totalBytes":5595258,"msg":"session_batch_recorder_flush_error"
Note: totalRetryDelay:28 (28ms) indicates the SDK's internal timeout is very short.
Expected Behavior
SESSION_RECORDING_V2_S3_TIMEOUT_MS should configure the AWS SDK's requestTimeout and connectionTimeout.
Suggested Fix
const { NodeHttpHandler } = require("@smithy/node-http-handler");
const s3Config = {
region: hub.SESSION_RECORDING_V2_S3_REGION,
endpoint: hub.SESSION_RECORDING_V2_S3_ENDPOINT,
forcePathStyle: true,
requestHandler: new NodeHttpHandler({
requestTimeout: hub.SESSION_RECORDING_V2_S3_TIMEOUT_MS,
connectionTimeout: 30000,
}),
};
Environment
PostHog version: latest (self-hosted)
S3-compatible storage: Wasabi (eu-west-2)
Session recording data size: ~5.5MB batches
Debug info
- [ ] PostHog Cloud, Debug information: [please copy/paste from https://us.posthog.com/settings/project-details#variables or https://eu.posthog.com/settings/project-details#variables]
- [x] PostHog Hobby self-hosted with `docker compose`, version/commit: `posthog/posthog:latest` (image sha256:36eedca28008, built 2026-01-23)
- [ ] PostHog self-hosted with Kubernetes (deprecated, see [`Sunsetting Kubernetes support`](https://posthog.com/blog/sunsetting-helm-support-posthog)), version/commit: [please provide]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with nodejs/dist/session-recording/consumer.js, where the S3Client is created, and compare it with the timeout usage in s3-session-batch-writer.js and the value exposed by config/config.js. Confirm that SESSION_RECORDING_V2_S3_TIMEOUT_MS reaches the AWS SDK request handler and that external S3 uploads no longer fail with RequestTimeout errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, javascript, node.js
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100