cloudflare / cloudflare/workerd
🐛 Bug Report — httpx via Cloudflare Python Worker drops User-Agent headers
- Dominant language
- C++
- Stars
- 8.7k
- Forks
- 739
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 174
Description
After some debugging, I believe I have found some kind of bug or issue with how User-Agent headers are handled by httpx as provided within the Python Worker runtime.
The following snippet sets up a simple worker that makes a few requests, and outputs the result.
```
class Default(WorkerEntrypoint):
async def fetch(self, request):
# A manually constructed request for reference
req = Request.new("https://httpbin.org/headers")
req.headers.set("User-Agent", "some custom user-agent", "Custom-Header")
req.headers.set("Custom-Header", "another value")
result = await fetch(req)
print(await result.json())
# An httpx driven request (which does not function as expected)
print(
(
await httpx.AsyncClient().get(
"https://httpbin.org/headers",
headers={
"User-Agent": "some custom user-agent",
"Custom-Header": "another value",
},
)
).json()
)
```
The response for the first request is:
```
{'headers': {'Cf-Worker': 'worker.example.com', 'Custom-Header': 'another value', 'Host': 'httpbin.org', 'User-Agent': 'some custom user-agent', 'X-Amzn-Trace-Id': 'Root=1-6a9a557a-2bfffbe61affbf6b149be877'}}
```
For the second request, however, the response is:
```
{'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate', 'Cf-Worker': 'worker.example.com', 'Custom-Header': 'another value', 'Host': 'httpbin.org', 'X-Amzn-Trace-Id': 'Root=1-6a9a557a-37c5a13b304346096bb8e308'}}
```
Aside from the few extra headers, the request through httpx does not have the custom `User-Agent`. It appears to have been uniquely dropped at some point, since the other `Custom-Header` survives.
I have exhausted most other debugging options, and am left thinking that this must be either an intentional exemption in the runtime (preventing `User-Agent` from being overridden, but only sometimes?) or a bug internally with how httpx is bridged to Python Workers.
When htps is run in a local script outside of the Python Worker runtime, the correct User-Agent header is attached.
Environment info:
* Pywrangler: 1.16.7
* Node: v26.7.0
* httpx: 0.28.1
* Compatibility date: 2026-08-20
* Using the `python_workers` compatibility flag
Contributor guide
Research direction
Reproduce the discrepancy in a Python Worker using httpx.AsyncClient().get, comparing it with the manual Request.new and fetch path under the python_workers compatibility flag. Trace where the User-Agent header is bridged into the Worker runtime while Custom-Header survives. Done means the custom User-Agent reaches httpbin.org/headers in the Worker runtime, with a regression test covering the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100