NginxProxyManager / NginxProxyManager/nginx-proxy-manager

Custom location returns 502 after backend Docker container is recreated (stale upstream not re-resolved)

Open
#5,603 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
34.2k
Forks
3.9k
Avg merge
21h 12m
Merged PRs (30d)
20

Description

Checklist

  • I have searched existing issues (related: #302, and the "Use Docker DNS / resolver for dynamic upstream" feature request)
  • I am running the latest version of NPM
  • This is reproducible
Summary

A custom location whose forward host points to a Docker container (by service/container name) returns 502 Bad Gateway after that backend container is recreated (e.g. a redeploy / docker compose up -d --force-recreate / image update), even though the main proxy host location keeps working. The custom location keeps the old (now invalid) upstream IP and never re-resolves the container name until the proxy host is manually re-saved (which regenerates and reloads the nginx config).

This is the same root cause as #302 (stale upstream after container recreation), but it specifically affects custom locations and is easy to miss because the primary location often recovers on its own while the custom location stays broken.

Steps to reproduce
  1. Create a Proxy Host with a main forward to a Docker container, e.g. web on the same Docker network.
  2. Add a Custom Location (e.g. /api/unicore) forwarding to a different backend container, e.g. api:8080.
  3. Confirm both https://host/ and https://host/api/unicore/... return 200.
  4. Recreate only the backend (api) container (redeploy / new image / --force-recreate). The container comes back healthy with a (possibly) new IP on the Docker network.
  5. Request the custom location again: https://host/api/unicore/...502 Bad Gateway (server: openresty), while https://host/ still returns 200.
Expected behavior

After the backend container is recreated, the custom location should re-resolve the container name (Docker embedded DNS) and route to the new IP, the same way it would after a manual proxy-host re-save.

Actual behavior

The custom location keeps the stale upstream IP and returns 502 until the proxy host is edited and saved again (which forces nginx to regenerate config and re-resolve the upstream). A container restart of NPM itself, or re-saving the proxy host, fixes it.

Workaround

Re-save the affected proxy host (or reload/restart the NPM container) after every backend redeploy so nginx re-resolves the Docker upstream. In our case we now do an NPM proxy-host refresh as a mandatory post-deploy step.

Root cause / suggestion

nginx resolves proxy_pass upstream names at config load time and caches the IP for the life of the worker. When NPM emits a static proxy_pass http://container_name:port; for a custom location, a recreated container with a new IP is never picked up.

Possible fixes:

  • Use the Docker embedded DNS resolver with a variable in proxy_pass so nginx re-resolves at request time, e.g.:
    resolver 127.0.0.11 valid=10s;
    set $upstream http://container_name:port;
    proxy_pass $upstream;
    
    (applied to custom locations as well as the main location)
  • This overlaps with the existing feature request to use Docker DNS / a dynamic resolver for upstreams.
Environment
  • NPM: latest
  • Deployment: Docker / docker compose, all containers on the same user-defined bridge network
  • Reverse proxy reports server: openresty

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the nginx configuration generation for custom locations and compare it with the main proxy-host location, focusing on how proxy_pass upstream names are resolved. Reproduce the backend-container recreation on a Docker network; done means the custom location routes to the new container IP without re-saving or restarting NPM.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, nginx
Domain
infrastructure, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.