jfrog / jfrog/jfrog-cli

jfrog-cli is unable to gracefully recover from connection closed by peer on Apple platforms.

Open
#3,406 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
595
Forks
307
Avg merge
4d 17h
Merged PRs (30d)
17

Description

Describe the bug

jfrog-cli is unable to gracefully recover from connection closed by peer on Apple platforms. Despite any successive retries does succeed.

Current behavior

I have a situation where a large +100Mb upload on a 10Gb network between the build-servers and the artifactory server times up during the upload (nginx and Artifactory sets HTTP 408 - Request Timed Out), the connection is closed. We believe the root cause here is some performance issues with nginx under the hood of the artifactory deployment. However, it doesn't explain the behavior for jfrog-cli (or jenkins-jfrog-plugin).

jfrog-cli is configured to retry up to 5 times with a wait-retry of 30s. That is all working fine, and will work, but
the pipeline still fails as jfrog-cli is returning 1 (should return 0). Also, it spews out 100s of messages: 11:17:38 [Error] io: read/write on closed pipe

[apple] $ /Users/buildops/tools/io.jenkins.plugins.jfrog.JfrogInstallation/jfrog-cli/jf -v
[apple] $ /Users/buildops/tools/io.jenkins.plugins.jfrog.JfrogInstallation/jfrog-cli/jf rt upload --spec ../../build/jfrog/secure-access-apple-upload.json --retries 5 --retry-wait-time 30s --spec-vars=${SPEC_VARS}
11:16:38 [Info] [Thread 2] Uploading artifact: sa-generic-dev/secure-access/builds/lynyrd/15976.Mar30.lynyrd/apple/ios/Debug/Mobility.xcarchive.zip
11:17:37 [Warn] [Thread 0] (Attempt 1) - Failure occurred while uploading to https://artifactory.failsafe.net/artifactory/sa-generic-dev/secure-access/builds/lynyrd/15976.Mar30.lynyrd/apple/ios/Debug/Secure%20Access.ipa;build.branch=lynyrd;build.change=365976;build.config=%7B2%7D;build.id=15976.Mar30.lynyrd;build.platform=macos;build.stamp=NONE;build.version=14.60.15976;type=ipa;build.name=Secure+Access+-+Apple+-+%28IPHONE%2C+DEBUG%29+-+lynyrd;build.number=15976.Mar30.lynyrd;build.timestamp=1774894597112: server response: 408 
<!doctype html><html lang="en"><head><title>HTTP Status 408 – Request Timeout</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 408 – Request Timeout</h1></body></html>
11:17:38 [Error] io: read/write on closed pipe
11:17:38 [Warn] [Thread 2] (Attempt 1) - Failure occurred while uploading to https://artifactory.failsafe.net/artifactory/sa-generic-dev/secure-access/builds/lynyrd/15976.Mar30.lynyrd/apple/ios/Debug/Mobility.xcarchive.zip;build.branch=lynyrd;build.change=365976;build.config=%7B2%7D;build.id=15976.Mar30.lynyrd;build.platform=macos;build.stamp=NONE;build.version=14.60.15976;type=macos-app;build.name=Secure+Access+-+Apple+-+%28IPHONE%2C+DEBUG%29+-+lynyrd;build.number=15976.Mar30.lynyrd;build.timestamp=1774894597112: server response: 408 
<!doctype html><html lang="en"><head><title>HTTP Status 408 – Request Timeout</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 408 – Request Timeout</h1></body></html>
11:17:38 [Error] io: read/write on closed pipe
...
11:17:38 [Error] io: read/write on closed pipe
11:18:09 [Error] io: read/write on closed pipe
{
  "status": "failure",
  "totals": {
    "success": 2,
    "failure": 0
  }
}
11:18:09 [Info] Trace ID for JFrog Platform logs: 5005513f40df719b
11:18:09 [Error] upload finished with errors. Review the logs for more information
Found unhandled java.lang.RuntimeException exception:
Couldn't execute 'jf' command. RuntimeException: Running 'jf' command failed with exit code 1
	PluginClassLoader for jfrog//io.jenkins.plugins.jfrog.JfStep$Execution.run(JfStep.java:134)
	PluginClassLoader for jfrog//io.jenkins.plugins.jfrog.JfStep$Execution.run(JfStep.java:92)
	PluginClassLoader for workflow-step-api//org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:49)
	java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
	java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	java.base/java.lang.Thread.run(Thread.java:1583)
Reproduction steps

Retry a bunch of large uploads and over a faulty nginx deployment that terminates the connection

Expected behavior

The jfrog-cli should recover gracefully and return 0 despite the read/write errors on a closed peer connection. That way, my build job can continue when a retry attempt was successful.

JFrog CLI version

2.97.0

Operating system type and version

MacOS 26.2 on macmini

JFrog Artifactory version

7.133.15

JFrog Xray version

N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the jf rt upload command and reproduce repeated large uploads through the faulty nginx deployment using the reported retry settings. Trace the upload retry and error-reporting path around HTTP 408 responses and closed pipes. Done means a successful retry returns exit code 0 without flooding the output with closed-pipe errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, macos, nginx
Domain
cli, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.