buildAndPushRelease.py: scp to people.apache.org stalls during transfer [LUCENE-5556]
- Dominant language
- Java
- Stars
- 3.6k
- Forks
- 1.4k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 88
Description
When I first tried to build the first RC for 4.7.1 using `dev-tools/scripts/buildAndPushRelease.py`, after preparing both the Lucene and Solr releases, the `scp` command to transfer the Lucene release tarball to people.apache.org stalled after about 180MB had transferred out of 220MB. After an hour or so, I `kill -9`'d the `scp` process and started over. Thankfully, everything went smoothly the second go-around.
But I'd like to see if we can reduce the likelihood of this happening again.
I did some searching for causes/solutions to this problem and came across this recommendation to use `scp`'s `-l` option to limit bandwidth in order to prevent stalls: - the recommendation there limits bandwidth to 1MB/sec (`-l 8192` is 8192Kbit/sec = 8Mbit/sec = 1MB/sec).
---
Migrated from [LUCENE-5556](https://issues.apache.org/jira/browse/LUCENE-5556) by Steven Rowe (@sarowe), updated Mar 27 2014
Attachments: [LUCENE-5556.patch](https://apache.github.io/lucene-jira-archive/attachments/LUCENE-5556/LUCENE-5556.patch)
Contributor guide
Research direction
Start with dev-tools/scripts/buildAndPushRelease.py and trace the scp command used after the Lucene and Solr releases are prepared. Review the attached LUCENE-5556.patch and the scp bandwidth-limiting recommendation, then verify that the Lucene release tarball transfer completes without stalling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, release
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100