agent fails to communicate with nginx listen sockets with proxy protocol enabled
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 371
- Forks
- 107
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 37
Description
Bug Overview
Agent is unable to communicate with nginx listeners that are configured with proxy protocol enabled, e.g. in a configuration like this:
server {
listen [::]:8888 default_server ipv6only=on proxy_protocol;
listen 8888 default_server proxy_protocol;
[..]
location = /stub_status {
stub_status on;
}
location /api {
api write=on;
}
[..]
}
Expected Behavior
Depending on a scenario, the following behavioral alternatives could be expected:
- If there are other listeners available to reach out to API/status endpoints, use those instead.
- If there are no other listeners, agent would seed proxy protocol payload while talking to specific sockets e.g. like curl does with
--haproxy-protocoloption (https://curl.se/docs/manpage.html#--haproxy-protocol).
Steps to Reproduce the Bug
- Set up nginx or nginx-plus with
proxy_protocollisteners (see the above example). - Install the agent.
- Run the agent.
- Check the nginx error log for entries like these:
2025/02/11 15:23:32 [error] 3881802#3881802: *889720 broken header: "GET /api HTTP/1.1" while reading PROXY protocol, client: ::1, server: [::]:8888
2025/02/11 15:23:32 [error] 3881803#3881803: *889721 broken header: "GET /api HTTP/1.1" while reading PROXY protocol, client: 127.0.0.1, server: 0.0.0.0:8888
2025/02/11 15:23:34 [error] 3881802#3881802: *889722 broken header: "GET /api HTTP/1.1" while reading PROXY protocol, client: ::1, server: [::]:8888
2025/02/11 15:23:34 [error] 3881802#3881802: *889723 broken header: "??/????I?In??Y?$?2??ↅK??? ?j,???p;ƫa?R?K?6Aijܜѝ?L^V?,?0?+?/̨̩??̪?????????$?(?#?'?" while reading PROXY protocol, client: ::1, server: [::]:8888
2025/02/11 15:23:34 [error] 3881803#3881803: *889724 broken header: "GET /api HTTP/1.1" while reading PROXY protocol, client: 127.0.0.1, server: 0.0.0.0:8888
2025/02/11 15:23:34 [error] 3881803#3881803: *889725 broken header: "??T;"%uB????(????#???9??," while reading PROXY protocol, client: 127.0.0.1, server: 0.0.0.0:8888
- Check agent log for entries like these:
time="2025-02-11T15:23:02Z" level=warning msg="Unable to perform Stub Status API GET request: Get \"http://[::1]:8888/stub_status\": EOF"
time="2025-02-11T15:23:02Z" level=warning msg="Unable to perform Stub Status API GET request: Get \"http://127.0.0.1:8888/stub_status\": EOF"
time="2025-02-11T15:23:02Z" level=warning msg="Unable to perform NGINX Plus API GET request: Get \"http://[::1]:8888/api\": EOF"
time="2025-02-11T15:23:02Z" level=warning msg="Unable to perform NGINX Plus API GET request: Get \"http://127.0.0.1:8888/api\": EOF"
Environment Details
- Target deployment platform: EC2 VM, local VM
- Target OS: Ubuntu 22.04.5 LTS
- Version of this project or specific commit: nginx-agent 2.39.0~jammy
- Version of any relevant project languages: irrelevant
- nginx version: nginx-plus-r33-p2
Additional Context
No response
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the agent entry points that issue Stub Status and NGINX Plus API GET requests, then reproduce the failure with the shown nginx proxy_protocol listeners and compare the request behavior with curl's --haproxy-protocol option. Done means the agent can successfully reach the relevant endpoints under the selected behavior and the nginx logs no longer report broken PROXY protocol headers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, nginx
- Domain
- api, backend, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100