apache / apache/lucene

buildAndPushRelease.py: scp to people.apache.org stalls during transfer [LUCENE-5556]

Open
#6,618 2 comments 0 reactions 0 assignees View on GitHub
legacy-jira-priority:Minor tool:build type:enhancement
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

When I first tried to build the first RC for 4.7.1 using `dev-tools/scripts/buildAndPushRelease.py`, after preparing both the Lucene and Solr releases, the `scp` command to transfer the Lucene release tarball to people.apache.org stalled after about 180MB had transferred out of 220MB. After an hour or so, I `kill -9`'d the `scp` process and started over. Thankfully, everything went smoothly the second go-around.

But I'd like to see if we can reduce the likelihood of this happening again.

I did some searching for causes/solutions to this problem and came across this recommendation to use `scp`'s `-l` option to limit bandwidth in order to prevent stalls: - the recommendation there limits bandwidth to 1MB/sec (`-l 8192` is 8192Kbit/sec = 8Mbit/sec = 1MB/sec).

---
Migrated from [LUCENE-5556](https://issues.apache.org/jira/browse/LUCENE-5556) by Steven Rowe (@sarowe), updated Mar 27 2014
Attachments: [LUCENE-5556.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5556/LUCENE-5556.patch)

Contributor guide

Open the contributing guide

Research direction

Start with dev-tools/scripts/buildAndPushRelease.py and trace the scp command used after the Lucene and Solr releases are prepared. Review the attached LUCENE-5556.patch and the scp bandwidth-limiting recommendation, then verify that the Lucene release tarball transfer completes without stalling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devops, release
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.