Backend server was not able to generate SSL

NPM SSL INCIDENT REPORT — LET'S ENCRYPT PROXY-HOST FAILURE ANALYSIS

Date: 2026-05-26
Environment type: production-testing
Proxy tier: Nginx Proxy Manager
Backend tier: Apache application server
Sanitized public hostname used in this report: dial.xyz.com
Sanitized backend hostname/IP examples:

• proxy: 192.168.1.40
• backend app server: 192.168.1.38

Executive summary

The SSL issuance and HTTPS activation problem was not caused by Let’s Encrypt itself and was not caused by the backend Apache certificate path.

The real failure was inside Nginx Proxy Manager runtime config generation:

• the proxy host existed in the NPM UI/database
• a certificate was issued/selected in NPM
• but NPM failed to generate or activate the live Nginx vhost because of a duplicate ACME challenge location block
• fallback Nginx behavior then rejected the TLS handshake for dial.xyz.com
• the browser showed ERR_SSL_UNRECOGNIZED_NAME_ALERT

A secondary contributing issue was stale duplicate proxy-host records for the same domain in the NPM database.

User-visible symptom

From browser:

• https://dial.xyz.com/ failed
• https://dial.xyz.com/.well-known/acme-challenge/test-token.txt failed
• browser error: ERR_SSL_UNRECOGNIZED_NAME_ALERT

At the same time:

• direct backend access on https://192.168.1.38/... worked
• backend-local challenge file serving worked

This created the false impression that Let’s Encrypt or backend Apache was broken, while the actual issue was the NPM front-door TLS host not becoming live.

What we tested first

We tested the full path in layers so we could isolate the failure point.

1. Backend application and challenge path test

We verified the backend server could serve the ACME challenge path directly.

Tests performed:

• direct HTTPS to backend IP
• direct HTTP/HTTPS challenge-file path from backend
• challenge-token file creation and retrieval

Findings:

• backend Apache served the challenge directory correctly
• challenge-token test file was reachable on backend
• backend certificate and application routing were functional

Conclusion:

• backend server was not the root cause

2. Public domain behavior through NPM

We then tested the public hostname through the proxy layer.

Tests performed:

• HTTPS request to https://dial.xyz.com/
• HTTPS request to challenge-token path
• TLS handshake probe against proxy

Findings:

• proxy TLS handshake failed before request reached backend
• browser showed ERR_SSL_UNRECOGNIZED_NAME_ALERT
• this indicated NPM did not have a working active TLS vhost for the hostname

Conclusion:

• front-door proxy config was the actual failing layer

3. NPM live config and DB inspection

We inspected NPM’s runtime-generated config and database entries.

Findings:

• the domain appeared in the NPM UI
• the host had a certificate selected
• but NPM was not successfully publishing an active live config for the host
• stale duplicate proxy-host rows existed for the same hostname
• a custom location for /.well-known/acme-challenge/ had been added manually

Critical runtime error found in NPM:

nginx: [emerg] duplicate location "/.well-known/acme-challenge/" in /data/nginx/proxy_host/<id>.conf
nginx: configuration file /etc/nginx/nginx.conf test failed

This was the key root-cause evidence.

Root cause

The root cause was a proxy-host configuration conflict inside NPM.

Primary root cause

A manual custom location was added for:

• /.well-known/acme-challenge/

NPM already manages ACME challenge routing for Let’s Encrypt internally.
When the manual custom location duplicated the same path, NPM generated an invalid Nginx config.

Effect:

• Nginx config test failed
• the proxy host could not be activated properly
• NPM fell back to its default TLS reject behavior
• HTTPS requests for the hostname failed with ERR_SSL_UNRECOGNIZED_NAME_ALERT

Secondary contributing cause

There were stale duplicate proxy-host records for the same hostname in the NPM database.

Effect:

• operational confusion
• increased risk of wrong backend mapping/cert association
• harder troubleshooting because UI state and runtime state did not line up cleanly

Why Let’s Encrypt looked broken even though it wasn’t

From the outside it looked like:

• Let’s Encrypt could not validate the host
• NPM was “not letting” Let’s Encrypt create SSL

What was actually happening:

• the challenge path on backend was okay
• the certificate object in NPM could exist
• but the active Nginx vhost for the hostname was not loading because config generation failed
• therefore public HTTPS traffic never reached the right live host

So this was primarily a proxy runtime activation bug/misconfiguration scenario, not a certificate authority failure.

Full scenario we verified

We validated the scenario step by step.

Scenario A: backend-only test

Expectation:

• if backend is healthy, direct backend challenge path should work

Observed:

• passed

Meaning:

• Apache/webroot/app routing okay

Scenario B: public hostname through proxy

Expectation:

• if NPM host is healthy, TLS should complete and request should forward to backend

Observed:

• failed before app response
• TLS name alert occurred

Meaning:

• proxy host not active or wrong runtime config

Scenario C: NPM runtime config generation

Expectation:

• saving the host in NPM should create one valid active Nginx vhost

Observed:

• config generation failed due to duplicate challenge location
• stale host rows also existed

Meaning:

• UI/database state was misleading; runtime state was not actually healthy

Scenario D: post-fix validation

Expectation:

• after removing conflict and duplicates, active TLS vhost should load and public hostname should work

Observed:

• active 443 ssl host generated
• proper certificate attached
• HTTPS requests returned 200
• challenge-token path returned 200

Meaning:

• issue resolved at proxy runtime layer

Exact fix applied in NPM

We fixed the issue by cleaning the NPM configuration, not by changing Let’s Encrypt behavior.

Fix 1: remove custom ACME challenge location

Removed the manually added custom location:

• /.well-known/acme-challenge/

Reason:

• NPM already handles this internally for its own Let’s Encrypt workflow
• manual duplication caused invalid Nginx config

Fix 2: remove stale duplicate proxy-host records

Removed/cleaned duplicate host entries for the same sanitized domain.

Reason:

• keep exactly one authoritative active proxy host per domain
• reduce config ambiguity and stale references

Fix 3: keep one clean active proxy host

Final clean proxy host shape:

• domain: dial.xyz.com
• forward host: backend app server
• forward port: 80
• SSL certificate attached in NPM
• Force SSL enabled only after host became healthy

Fix 4: reload/restart NPM runtime

After cleanup, NPM runtime was restarted so the cleaned host could be regenerated into live config.

Post-fix evidence

After the fix:

• active live Nginx host existed for dial.xyz.com
• listen 443 ssl was present in live config
• certificate binding was present in live config
• public HTTPS returned 200
• challenge-token path returned 200
• backend application opened correctly through NPM

Final technical conclusion

The SSL incident was caused by NPM config-generation failure triggered by a duplicate manual ACME challenge location, with stale duplicate domain rows worsening the situation.

It was not primarily caused by:

• Let’s Encrypt CA outage
• backend Apache challenge path failure
• backend application routing failure
• backend certificate storage failure

Recommended NPM bug-list entry

Title:

• Proxy host saved in UI but live TLS vhost not activated when duplicate ACME custom location exists

Problem statement:

• NPM allowed a proxy host to appear healthy in UI with a certificate selected, while runtime config generation failed because a manual /.well-known/acme-challenge/ custom location duplicated NPM’s internal ACME handling.
• The failure state was misleading because the hostname appeared configured, but live Nginx did not serve the expected TLS vhost and instead returned TLS name-alert rejection.

Suggested product improvements:

1. Prevent users from adding a custom location that conflicts with NPM-managed ACME challenge paths.
2. Show config-generation failure prominently in the proxy-host UI, not only in logs.
3. Detect duplicate active proxy hosts for the same domain and block or warn.
4. Show distinction between:
   • host saved in database
   • host config generated successfully
   • host loaded into live Nginx runtime
5. Add a health indicator for TLS activation success per proxy host.

Operational prevention guidance

To avoid recurrence:

• do not manually add /.well-known/acme-challenge/ in NPM custom locations when NPM manages Let’s Encrypt for that host
• keep only one active proxy host per domain
• after SSL changes, validate both:
  • NPM UI state
  • live generated Nginx config/runtime state
• if browser shows ERR_SSL_UNRECOGNIZED_NAME_ALERT, check NPM runtime config generation before blaming backend Apache or Let’s Encrypt

Clean summary for support

Issue summary:

• dial.xyz.com HTTPS failed at NPM despite backend being healthy.
• Root cause was duplicate ACME challenge location plus duplicate/stale host rows in NPM.
• Removing the conflicting custom location and stale duplicate proxy-host rows allowed NPM to generate the live TLS vhost correctly.
• After cleanup, Let’s Encrypt-backed HTTPS worked normally.

Start by reproducing the Nginx Proxy Manager proxy-host save flow with a custom /.well-known/acme-challenge/ location, then inspect the generated live Nginx config and duplicate proxy-host records described here. Done means the conflict is prevented or clearly reported, duplicate hosts are detected, and the UI distinguishes saved, generated, and live TLS states; validate with the stated HTTPS and challenge-path checks.