Smoke tests: interactive shell access and TTY features
- Dominant language
- Python
- Stars
- 2
- Forks
- 0
- Avg merge
- 15h 59m
- Merged PRs (30d)
- 9
Description
## Summary
The current smoke tests only verify basic SSH connectivity by running `echo "connection-ok"` over SSH. We should add tests that exercise `remo shell` — the primary interactive entry point — to catch regressions in shell access, environment setup, and port-forwarding tunnels.
### Black-box testing with `expect` or alternatives
Several of these scenarios involve interactive/TTY behavior that cannot be tested with simple `ssh ... echo`. For CI, we should evaluate:
- **[`expect`](https://core.tcl-lang.org/expect/index)** — classic tool for driving interactive programs; available via `apt install expect` on Ubuntu runners
- **[`unbuffer`](https://linux.die.net/man/1/unbuffer)** (from the `expect` package) — wraps a command with a pseudo-TTY so it behaves as if interactive
- **Python [`pexpect`](https://pexpect.readthedocs.io/)** — modern alternative, already available in our venv; good for more complex assertions
The basic approach: spawn `remo shell` (or `remo shell`) under `expect`/`pexpect`, wait for a shell prompt, send commands, and assert on output. This lets us test the full "black box" user experience without mocking internals.
## Test scenarios
- [ ] **Basic shell access** — `remo shell` (or `remo shell `) drops into a working shell; run a command (`whoami`, `hostname`) and verify output
- [ ] **Shell environment** — verify expected env vars are set inside the shell (e.g., `TERM`, timezone propagation via `TZ`)
- [ ] **Port forwarding** — `remo shell -L 8080` establishes a tunnel; start a listener inside the instance (`python3 -m http.server 8080 &`), then `curl localhost:8080` from the runner to verify the tunnel works
- [ ] **Multiple tunnels** — `remo shell -L 8080 -L 3000` forwards both ports correctly
- [ ] **`--no-open` flag** — `remo shell -L 8080 --no-open` does not attempt to open a browser (no error on headless CI)
- [ ] **Shell picker** — when multiple instances exist, `remo shell` (no args) presents a picker; verify with `expect` that the picker appears and an entry can be selected
## Implementation notes
- Start with the Incus provider since it runs locally on the runner (no cloud cost, fastest feedback)
- Port forwarding tests need the shell to stay open in the background while we `curl`; `expect` or `pexpect` can handle this by spawning the process and interacting asynchronously
- Consider adding a small helper script (`tests/expect_helpers/` or similar) to keep the workflow YAML readable
- These tests should run **after** the existing create + connectivity steps, before teardown
Contributor guide
Research direction
Locate the existing smoke-test workflow after the create and connectivity steps, using the Incus provider as the first entry point. Exercise `remo shell` with expect or pexpect across the listed scenarios, then verify the workflow passes before teardown, including interactive access, environment variables, tunnels, `--no-open`, and picker behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, cli, testing-qa
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100