kubernetes / kubernetes/node-problem-detector

Flag --enable-k8s-exporter=false causes loss of /healthz, /conditions, and /debug/pprof endpoints

Open
#1,332 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
3.5k
Forks
702
Avg merge
1d 6h
Merged PRs (30d)
11

Description

When running NPD outside of Kubernetes (e.g. on a bare host or in a VM without a cluster), passing `--enable-k8s-exporter=false` is required to avoid a panic at startup. However, this flag also silently removes the HTTP diagnostic server on `--port` (default `20256`), losing access to `/healthz`, `/conditions`, and `/debug/pprof/`.

## Root cause

`startHTTPReporting()`, the function that starts the `/healthz`, `/conditions`, and pprof handlers, is called exclusively inside `k8sexporter.NewExporterOrDie()`.
When `--enable-k8s-exporter=false`, that function returns `nil` immediately and the HTTP server is never started, even though `--port` is set to a non-zero value.

## Steps to reproduce

```sh
❯ docker run --rm --name npd --privileged -v /dev/kmsg:/dev/kmsg:ro registry.k8s.io/node-problem-detector/node-problem-detector:v1.36.0 \
--enable-k8s-exporter=false \
--prometheus-port=20257 \
--config.custom-plugin-monitor=config/custom-plugin-monitor.json

❯ docker exec -ti npd bash
# Prometheus metrics work:
$ curl -Is http://127.0.0.1:20257/metrics
HTTP/1.1 200 OK
Content-Type: text/plain; version=0.0.4; charset=utf-8; escaping=underscores
Date: Mon, 03 Aug 2026 12:06:56 GMT
Content-Length: 1255

# But the HTTP server is gone:
$ curl http://127.0.0.1:20256/healthz
curl: (7) Failed to connect to 127.0.0.1 port 20256 after 0 ms: Couldn\'t connect to server
$ curl http://127.0.0.1:20256/conditions
curl: (7) Failed to connect to 127.0.0.1 port 20256 after 0 ms: Couldn\'t connect to server
$ curl http://127.0.0.1:20256/debug/pprof/
curl: (7) Failed to connect to 127.0.0.1 port 20256 after 0 ms: Couldn\'t connect to server
$
```

## Expected behavior

`/healthz`, `/conditions`, and `/debug/pprof` should be available on `--port` regardless of whether `--enable-k8s-exporter` is true or false.
The Prometheus metrics endpoint (problem_counter, problem_gauge, etc.) already works without Kubernetes because those metrics are written directly by the problem daemons via OpenCensus; the HTTP diagnostic server should behave the same way.

## Fix

Move the HTTP server out of the K8s exporter into a standalone httpexporter that tracks conditions in-memory via `ExportProblems` and is started independently of `--enable-k8s-exporter`.

Contributor guide

Open the contributing guide

Research direction

Trace startHTTPReporting() and its call from k8sexporter.NewExporterOrDie(), then inspect how --enable-k8s-exporter and --port are handled. Separate the diagnostic HTTP server into the proposed standalone exporter, including in-memory ExportProblems condition tracking. Validate with --enable-k8s-exporter=false by checking /healthz, /conditions, and /debug/pprof/ on port 20256 while Prometheus metrics continue to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.