NVIDIA / NVIDIA/NemoClaw

[Install] OpenShell release asset download has no timeout or retry, so a stalled connection hangs the installer indefinitely

Open Beginner friendly
#11,281 0 comments 0 reactions 0 assignees View on GitHub
area: install area: onboarding platform: arm64 platform: dgx-spark
Dominant language
TypeScript
Stars
22.5k
Forks
3.1k
Avg merge
1d 1h
Merged PRs (30d)
715

Description

## Investigation Summary

- `download_with_curl()` in `scripts/install-openshell.sh` fetches every OpenShell release asset with no `--max-time`, no `--connect-timeout`, and no `--retry`.
- A connection that stalls mid-transfer therefore never fails and never retries; the installer waits forever with a progress bar frozen at a partial byte count.
- Observed on a DGX Spark aarch64 host: the transfer stopped at 8,199,110 bytes and stayed at 0 B/s for over an hour while unrelated `curl` requests from the same host to the same URL succeeded at ~71 KB/s, so host egress was healthy.
- The same file already guards its other GitHub download: `openshell_homebrew_formula` uses `--connect-timeout 10 --max-time 30 --retry 3 --retry-delay 1 --retry-all-errors`. Only the release-asset path is unguarded.
- Because `nemoclaw onboard` auto-installs OpenShell during preflight, the hang presents to the user as onboarding hanging with no output, not as a download failure.

## Description

`scripts/install-openshell.sh` downloads the OpenShell release assets (`openshell-*`, `openshell-gateway-*`, `openshell-sandbox-*` and their checksum files) with a bare `curl -fL`:

```bash
# scripts/install-openshell.sh:1028
curl -fL "${curl_progress[@]}" "https://github.com/NVIDIA/OpenShell/releases/download/${RELEASE_TAG}/$name" \
-o "$tmpdir/$name"
```

There is no transfer timeout and no retry. When a TCP connection stalls after partially transferring an asset, `curl` keeps waiting indefinitely, so the installer -- and any `nemoclaw onboard` that triggered it -- hangs with no error and no progress.

**Expected:** a stalled or slow transfer either recovers through a retry or fails with a clear error after a bounded time, so the installer can report the failure and the user can retry.

**Actual:** the installer blocks indefinitely. The only visible symptom is a progress bar that stops advancing.

Note the inconsistency inside the same script -- the Homebrew formula download is already protected:

```bash
# scripts/install-openshell.sh:787
curl --proto '=https' --tlsv1.2 -fL -sS \
--connect-timeout 10 --max-time 30 --retry 3 --retry-delay 1 --retry-all-errors \
"https://github.com/NVIDIA/OpenShell/releases/download/${release_tag}/openshell.rb" \
-o "$output"
```

## Reproduction Steps

1. On a host with no `openshell` binary installed (for example after `nemoclaw uninstall --yes --destroy-user-data --all-gateway-ports`, which removes it), and on a network where the connection to `objects.githubusercontent.com` can stall.
2. Run `bash scripts/install-openshell.sh` (or any `nemoclaw onboard`, which installs OpenShell during preflight).
3. Watch the size of the partially downloaded asset while the progress bar is displayed:

```bash
watch -n10 'ls -l /tmp/tmp.*/openshell-*'
```

4. When a transfer stalls, the byte count stops changing and the process never exits.

A stall can be forced deterministically by dropping the established connection's packets rather than resetting it, for example with an `iptables` DROP rule on the `curl` process's destination while the transfer is in flight.

## Environment

- OS: Ubuntu 24.04.4 LTS
- Hardware: DGX Spark (GB10), aarch64
- Kernel: 7.0.0-1010-nvidia
- Node.js: v22.23.2
- npm: 10.9.8
- Docker: 29.2.1
- OpenShell: 0.0.106 (the release being installed)
- NemoClaw: `main` at v0.0.114-520-gb117f414d
- Affected file is platform-independent, so this is not specific to DGX Spark.

## Debug Output

`nemoclaw debug --quick`, System section (remainder omitted -- the failure is in a shell script that runs before any sandbox exists, so sandbox diagnostics are not relevant):

```text
Linux 7.0.0-1010-nvidia #10spark3 SMP PREEMPT_DYNAMIC aarch64 GNU/Linux
total used free shared buff/cache available
Mem: 124608 4557 95123 19 26044 120050
Swap: 16383 0 16383
```

## Logs

Installer output while stalled -- the byte counter stops and nothing further is printed:

```text
[install] Detected Linux (aarch64)
[install] Installing OpenShell from release 'v0.0.106'...
[install] Downloading OpenShell release assets (this may take a minute)...
###### 1.6%
```

Evidence that the stall was in the transfer, not the host:

```text
# partially downloaded asset, unchanged across 60s
bytes: 8199110 -> 8199110 rate=0 B/s

# a separate request to the same URL from the same host at the same time
downloaded=1779234 speed=71171 B/s
```

## Suggested Fix

Apply the same guards the Homebrew path already uses, plus a low-speed abort so a connection that technically dribbles does not hang either:

```bash
curl -fL "${curl_progress[@]}" \
--connect-timeout 10 --retry 3 --retry-delay 2 --retry-all-errors \
--speed-limit 1024 --speed-time 60 \
"https://github.com/NVIDIA/OpenShell/releases/download/${RELEASE_TAG}/$name" \
-o "$tmpdir/$name"
```

A fixed `--max-time` is deliberately avoided here: the assets total roughly 46 MB and legitimately take a long time on slow links. `--speed-limit`/`--speed-time` bounds the stall while still allowing a slow-but-progressing download to finish.

## Checklist

- [x] I confirmed this bug is reproducible
- [x] I searched existing issues and this is not a duplicate

Contributor guide

Open the contributing guide

Research direction

Start in scripts/install-openshell.sh at download_with_curl() around line 1028, then compare its curl options with the guarded openshell_homebrew_formula download around line 787. Run bash scripts/install-openshell.sh and verify release-asset downloads retry and abort after sustained low speed instead of hanging indefinitely, while slow progressing transfers can finish.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
devops
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.