[BUG] `npm test` is non-hermetic: integration tests trigger real npm installs and fail due to stdout/stderr pollution
- Dominant language
- TypeScript
- Stars
- 272
- Forks
- 375
- Avg merge
- 3h 22m
- Merged PRs (30d)
- 8
Description
## Describe the bug
Running `npm test` in the AsyncAPI CLI repository fails on a clean or typical local setup, even when the CLI code itself has not been changed.
A large number of integration tests fail because some CLI commands trigger **real npm dependency resolution / installation steps** during test execution. This causes npm to print progress output (for example `idealTree` spinners and dependency logs) to `stdout`.
Many tests expect:
- empty output (`''`), or
- an exact, fixed string
Because of this, the additional npm output breaks assertions and causes widespread test failures that are **not related to actual CLI behavior or regressions**.
---
## Current behavior
- `npm test` produces many failing integration tests.
- Failures contain unexpected output such as:
- `idealTree:*`
- npm progress spinners
- dependency resolution logs
- Test results vary depending on:
- npm version
- local environment
- network conditions
- OS setup
This makes the test suite **non-deterministic** and hard to trust.
---
## Expected behavior
- `npm test` should pass on a clean local setup without requiring special environment tweaks.
- CLI integration tests should:
- not rely on real npm installs, or
- mock / silence npm output during tests, or
- assert exit codes and behavior instead of exact stdout content.
- Test results should be deterministic and environment-independent.
---
## Steps to reproduce
1. Clone the repository:
```bash
git clone https://github.com/asyncapi/cli.git
cd cli
```
2. Install dependencies:
```
npm install
```
3. Run the test suite:
```
npm test
```
4. Observe:
- Multiple integration test failures
- Unexpected npm output (idealTree, spinners, logs) appearing in assertions
- Tests failing even though CLI logic is unchanged
Contributor guide
Assessment
This issue has not been assessed yet.