prometheus-community / prometheus-community/PushProx
Error polling - EOF
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 811
- Forks
- 154
- PR merge metrics
- No merged PRs in 30d
Description
up to yesterday evening (~2024-11-27 18:00:00 UT) my configuration
- client (debian12)
- PushProx
- several exporter like Prometheus-node-exporter (from Debian repository)
- server (debian12)
- Prometheus (from Debian repository)
- PushProx
works perfectly
Without any changes at the configuration and the firewalls now I got in the log of the PushProxy on the client
pushprox-client[7039]: ts=2024-11-27T11:35:29.979Z caller=main.go:197 level=error msg="Error polling:" err="Post \"http://server:8082/poll\": EOF"
and of curse on the server some time out error due to this:
pushprox-proxy[33701]: ts=2024-11-27T11:40:29.429Z caller=main.go:179 level=error msg="Error scraping:" err="Timeout reached for \"http://client:9101/metrics\": context canceled" url=http://client:9101/metrics
I have check access to the ports and also with curl by
curl -d 'test' http://server:8082/poll
which is received on the server
Nov 27 12:48:04 server pushprox-proxy[33701]: ts=2024-11-27T11:48:04.779Z caller=coordinator.go:136 level=info msg=WaitForScrapeInstruction fqdn=test
Nov 27 12:48:04 server pushprox-proxy[33701]: ts=2024-11-27T11:48:04.779Z caller=main.go:148 level=info msg="Error WaitForScrapeInstruction:" err="request is expired"
I have also pulled the latest version form PushProxy now and checked the system times - same errors.
Sending to the PushProxyon the clientvia curl -d 'test' http://localhost:9101/polls generates a return of
# HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
# TYPE go_gc_duration_seconds summary
go_gc_duration_seconds{quantile="0"} 0
go_gc_duration_seconds{quantile="0.25"} 0
go_gc_duration_seconds{quantile="0.5"} 0
....
My configs:
- server
- pushproxy:
--web.listen-address=:8082 - prometheus: /etc/prometheus/prometheus.yml
- pushproxy:
- job_name: push_proxy
proxy_url: http://localhost:8082/
scrape_interval: 120s
scrape_timeout: 120s
tls_config:
insecure_skip_verify: true
static_configs:
- targets: ['client:9101'] # push proxy (etc, metrics); Presuming the FQDN of the client is "client".
- targets: ['client:9732'] # nftables (/home)
- targets: ['client:9633'] # smartctl (/home)
- targets: ['client:9902'] # smartctl (/home)
- client:
--proxy-url=http:/server:8082/ --log.level=debug --metrics-addr=:9101
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Trace the polling failure from client main.go:197 through the server main.go:179 and coordinator.go:136, using the provided client and server configurations to reproduce the EOF and timeout logs. Review the polling and scrape interaction first; done means identifying the cause of the connection failure and documenting a verified resolution or reproduction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- debian, go, prometheus
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100