lightninglabs / lightninglabs/lndmon
Performance issue with /metrics endpoint
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 160
- Forks
- 53
- PR merge metrics
- No merged PRs in 30d
Description
I am trying to use lnd+lndmon on a rock64 board (similar to rpi, with arm64 and 4GB RAM) but Grafana only shows data points coming directly from lnd (Go Runtime + Performance dashboard). Everything supposed to come from lndmon is not there.
I noticed that when running simple queryes with PromQL I immediately got the error: "the queries returned no data for a table". Then went to Explore section and checked for `up`, there I can see how the lndmon process is reported to be down, which is not true.
After that I tried to get the metrics directly and I realized I was getting slow response times on the metrics endpoint (between 10s and 12s usually):
```
$ time curl -s --output /dev/null localhost:9092/metrics
real 0m10.717s
user 0m0.022s
sys 0m0.015s
```
I haven't investigated this deeply yet but the instance has more than enough Ram, and the CPU usage and load average don't look that bad.
Will try to spend more time in another moment but wanted to report soon just in case it's happening to more people.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported 10–12 second response from the /metrics endpoint with curl and compare it with the lndmon process status shown in Prometheus. Trace the endpoint's request path and metric collection behavior; done means lndmon metrics are exposed reliably, the process is reported up, and the Grafana data is available without the observed delay.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, grafana, prometheus
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100