cloudfoundry / cloudfoundry/nodejs-buildpack

Supply step do not retry when error is connection reset by peer

Open
#626 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

unscheduled
Dominant language
Go
Stars
174
Forks
389
PR merge metrics
No merged PRs in 30d

Description

What version of Cloud Foundry and CF CLI are you using? (i.e. What is the output of running cf curl /v2/info && cf version?

{
...
   "version": 0,
   "min_cli_version": null,
   "min_recommended_cli_version": null,
   "api_version": "2.203.0",
   "osbapi_version": "2.15",
}
cf version 7.5.0+0ad1d63.2022-06-04

What version of the buildpack you are using?

We have multiple applications, the below versions are used currently.
Version: 1.8.5, 1.8.7 and 1.8.12

If you were attempting to accomplish a task, what was it you were attempting to do?

Package staging randomly fails at supply/finalize step when downloading Go without attempting any retry.

What did you expect to happen?

The curl command already has retry mechanism, and it should retry if the attempts encounter any network connection problems.

What was the actual behavior?

If the error is "connection reset by peer", the curl command do not retry and immediately return error

see https://stackoverflow.com/questions/42873285/curl-retry-mechanism

Sample log output from running the same curl command

   2023-07-18T18:17:02.90+0000 [APP/TASK/eab4dfa6/0] ERR % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
   2023-07-18T18:17:02.90+0000 [APP/TASK/eab4dfa6/0] ERR Dload  Upload   Total   Spent    Left  Speed
   2023-07-18T18:17:03.07+0000 [APP/TASK/eab4dfa6/0] ERR 0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
   2023-07-18T18:17:03.74+0000 [APP/TASK/eab4dfa6/0] ERR 0  151M    0  142k    0     0   812k      0  0:03:10 --:--:--  0:03:10  809k
   2023-07-18T18:17:03.75+0000 [APP/TASK/eab4dfa6/0] ERR 33  151M   33 50.1M    0     0  59.3M      0  0:00:02 --:--:--  0:00:02 59.3M
   2023-07-18T18:17:03.75+0000 [APP/TASK/eab4dfa6/0] ERR curl: (56) OpenSSL SSL_read: Connection reset by peer, errno 104
   2023-07-18T18:17:03.75+0000 [APP/TASK/eab4dfa6/0] OUT Exit: 56, Elapsed: 1

To reproduce, create a sample app(doesn't matter what app), then create a run-task that perpetually download the go binary

cf run-task app-1 --command 'while true; do start_time=$(date +%s); curl https://buildpacks.cloudfoundry.org/dependencies/go/go_1.19_linux_x64_cflinuxfs3_7e231ea5.tgz --location --retry 15 --retry-delay 2 --output /tmp/go.tgz; echo -n "Exit: $?, "; end_time=$(date +%s); elapsed=$(( end_time - start_time )); echo "Elapsed: $elapsed"; sleep 10; done'

Watch the app log output for errors.

Can you provide a sample app?

N/A

Please confirm where necessary:

  • I have included a log output
  • My log includes an error message
  • I have included steps for reproduction

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the supply/finalize step that downloads Go and inspect the curl retry behavior described in the issue. Reproduce the failure with the provided cf run-task command and confirm that a connection-reset error is retried; done means the download no longer fails immediately on that error.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.