[Packaging][Release] Binary verification downloads loads of packages which aren't necessary for the verification
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
Verification of binaries when executing the following:
```
TEST_DEFAULT=0 TEST_BINARY=1 ./dev/release/verify-release-candidate.sh 23.0.1 0
```
Takes a really long time.
I've noticed we seem to be downloading a lot of binaries that don't seem related to the actual release being verified, a really small snippet example:
```
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_12-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_12-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_12-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_12-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_12-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_12-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_13-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_13-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_13-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_13-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_13-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_13-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_14-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_14-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_14-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_14-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_14-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_14-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_15-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_15-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_15-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_15-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_15-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_15-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_16-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_16-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_16-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_16-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_16-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_16-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_17-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_17-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_17-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_17-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_17-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_17-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_18-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_18-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_18-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_18-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_18-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_18-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_19-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_19-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_19-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_19-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_19-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_19-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_20-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_20-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_20-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_20-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_20-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_20-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_21-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_21-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_21-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_21-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_21-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_21-1_amd64.deb.sha512
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_22-1_amd64.deb to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_22-1_amd64.deb
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_22-1_amd64.deb.asc to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_22-1_amd64.deb.asc
Downloading debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_22-1_amd64.deb.sha512 to /tmp/arrow-23.0.1.EbVUu/binaries/debian-rc/pool/bookworm/main/a/apache-arrow-adbc/libadbc-driver-snowflake-dev_22-1_amd64.deb.sha512
```
In this case those are adbc and we download several versions of them which are unnecessary.
The problem gets worst when for connectivity issues the verification fails and we have to restart again.
I'll try to time it to get a real number of how long it takes but I am sure it takes ~2 hours.
### Component(s)
Release, Packaging
Contributor guide
Assessment
This issue has not been assessed yet.