[BUG] Downloading and decompressing cuvs_conda_cpp_libcuvs_x86_64_cu12 occasionally takes a very long time
Nobody has claimed this yet.
- Dominant language
- Cuda
- Stars
- 854
- Forks
- 236
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 62
Description
Describe the bug
I keep noticing cuvs-java and cuvs-lucene builds that occasionally take a very long time to complete. It looks like the majority of the time is spent on these two lines (depending on the type of the build):
https://github.com/NVIDIA/cuvs/blob/fdd404f063639afb5738ad13d6868d03f50f1fe4/ci/build_java.sh#L29
https://github.com/NVIDIA/cuvs/blob/fdd404f063639afb5738ad13d6868d03f50f1fe4/ci/build_lucene.sh#L34
Most of the time these lines take just a couple of minutes. For example, in https://github.com/NVIDIA/cuvs/actions/runs/32545903651/job/96968831522?pr=2499#step:13:72 this line is executed pretty fast:
RAPIDS logger » [08/22/26 02:58:18]
┌────────────────────────────────────────────────┐
| Downloading artifacts from previous jobs |
└────────────────────────────────────────────────┘
[rapids-download-from-github] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Assuming artifacts should be fetched from this same run of 'pr.yaml'
Downloading and decompressing cuvs_conda_cpp_libcuvs_x86_64_cu13 from Run ID 32545903651 into /tmp/tmp.m7yJdKDcm7
RAPIDS logger » [08/22/26 03:00:40]
However, in other cases such as https://github.com/NVIDIA/cuvs/actions/runs/32545903651/job/96968831551?pr=2499#step:13:72 it can take almost 30 minutes
RAPIDS logger » [08/22/26 02:59:48]
┌────────────────────────────────────────────────┐
| Downloading artifacts from previous jobs |
└────────────────────────────────────────────────┘
[rapids-download-from-github] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Assuming artifacts should be fetched from this same run of 'pr.yaml'
Downloading and decompressing cuvs_conda_cpp_libcuvs_x86_64_cu12 from Run ID 32545903651 into /tmp/tmp.nC3Cq1kuVk
RAPIDS logger » [08/22/26 03:28:29]
And occasionally it can take almost hour and a half https://github.com/NVIDIA/cuvs/actions/runs/32545903651/job/96973240844?pr=2499
RAPIDS logger » [08/22/26 03:36:51]
┌────────────────────────────────────────────────┐
| Downloading artifacts from previous jobs |
└────────────────────────────────────────────────┘
[rapids-download-from-github] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Assuming artifacts should be fetched from this same run of 'pr.yaml'
Downloading and decompressing cuvs_conda_cpp_libcuvs_x86_64_cu13 from Run ID 32545903651 into /tmp/tmp.0HAanNDBql
[rapids-download-from-github] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Detected CI=true. Skipping running 'gh auth status'.
[rapids-github-run-id] Assuming artifacts should be fetched from this same run of 'pr.yaml'
Downloading and decompressing cuvs-java-cuda13.3.0 from Run ID 32545903651 into /tmp/tmp.bgbx0u4QWB
RAPIDS logger » [08/22/26 05:02:21]
I actually watched this one and it was stuck on cuvs_conda_cpp_libcuvs_x86_64_cu13 for most of that time.
Steps/Code to reproduce bug
Start a build in CI that builds cuvs-java and cuvs-lucene.
Expected behavior
Download is performed in reasonable and predictable time.
Environment details (please complete the following information):
CI
Additional context
N/A
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ci/build_java.sh line 29 and ci/build_lucene.sh line 34, then reproduce a CI build for cuvs-java and cuvs-lucene. Compare the artifact download logs for the cuvs_conda_cpp_libcuvs_x86_64_cu12 and cu13 artifacts across runs. Done means these downloads complete in a reasonable and predictable time.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, shell
- Domain
- build-system, ci-cd
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100