smartcontractkit / smartcontractkit/chainlink
[NODE] Missing job requests.
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 8.2k
- Forks
- 2k
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 202
Description
Description
The node receives heads but missing jobs.
Basic Information
While running release V1.1.0 the following occurred twice. The node was set up with a minimal env file specifying only the bare minimum and setting the chain nodes etc through the GUI to be round robined. The node will run fine for a while but suddenly will start missing jobs. The logs display the node receiving heads and processing them from multiple endpoints, in this case, 4 different RPC/wss connections. This will result in LINK being locked in the oracle contract and jobs staying unfulfilled. There are no errors in the logs and the only way to spot these malfunctions is constantly monitoring the incoming and outgoing transactions for the oracle.
- Network: Polygon
- Operating System: Docker Ubuntu 20.04 LTS
- Commit: v1.1.0
- Hosting Provider: self-hosted
- Startup Command: cd ~/.chainlink && sudo docker run --network chainlink --name chainlink --restart unless-stopped -p X:X -v ~/.chainlink:/chainlink -it --env-file=.env smartcontract/chainlink:1.1.0 local n -p /chainlink/.psw/.password -a /chainlink/.psw/.api
Environment Variables
ROOT=/chainlink
ETH_CHAIN_ID=137
LOG_LEVEL=debug
CHAINLINK_TLS_PORT=
SECURE_COOKIES=true
ALLOW_ORIGINS=*
DATABASE_URL=
DATABASE_TIMEOUT=0
TLS_CERT_PATH=/chainlink/.tls/server.crt
TLS_KEY_PATH=/chainlink/.tls/server.key
Steps to Reproduce
Run the node and get transactions.
Additional Information
No errors are logged. All chain connections seem in sync (4 different endpoints). Node is processing heads. A restart fixes the issue. Can be detected by monitoring incoming link payments vs outgoing tx'es.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the v1.1.0 Docker startup command and minimal environment shown in the issue, then reproduce the node receiving heads from four RPC/WSS endpoints while jobs go missing. Compare incoming heads and LINK payments with outgoing transactions and debug logs; done means the missed-job condition is explained and no longer occurs silently, or produces a useful error signal.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, go
- Domain
- backend, blockchain, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100