lightninglabs / lightninglabs/lndmon

latency seems high for /metric endpoint

Open
#77 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
160
Forks
53
PR merge metrics
No merged PRs in 30d

Description

on both v0.13.3-beta and v0.14.1-beta if nodes aren't compacted often it seems that latency starts creeping up for the /metrics endpoint. It gets pretty bad, over 10 seconds. My scrape interval is 15s and my timeout is 10s, so when nodes start to fail to return after 10s, I know it's time to restart the node.

Is there something wrong with LND / LNDMON / or my setup?

I would imagine a prometheus metric endpoint would take less than 1s to return data on average. Even after restart/compacting the node it still takes several seconds to retrieve data from `/metrics`

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the latency on the /metrics endpoint with the reported v0.13.3-beta and v0.14.1-beta versions, comparing behavior before and after node compaction or restart. Determine whether the delay comes from lndmon, LND, or the reported setup, and document the conditions and expected response time once the cause is established.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, prometheus
Domain
backend, observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.