npm / npm/cli

[BUG] npm audit JSON output is not deterministic/stable

Open
#4,366 3 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug Priority 2 Release 8.x
Dominant language
JavaScript
Stars
10.1k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
19

Description

Is there an existing issue for this?
  • I have searched the existing issues
This issue exists in the latest npm version
  • I am using the latest npm
Current Behavior

npm audit --json produces different results in each of several consecutive runs on the same input file.

The differences are mainly in the fixAvailable, effects and sometimes the via fields of the vulnerabilities key.

Expected Behavior

When running npm audit several times with the same package.json and package-lock.json I would only expect changes to the output if some new vulnerability is reported, details are added to one or a new fixed version is released.

This makes it hard to e.g. create or update issue tracker tickets based on the output since it leads to spurious updates.

Steps To Reproduce

On Linux
Internal project's npm config but

  • "via" seems to change with cssnano <=4.1.11, usually a single postcss-* value is removed and one is added
  • fixAvailable seems to sometimes just return true and sometimes css-loader 6.6.0 and sometimes switch between karma 6.3.13 and protractor 7.0.0 or remap-istanbul 0.13.0 and protractor 7.0.0 or webpack-dev-server 4.7.4 and karma 6.3.13
  • effects seems to be sometimes empty and sometimes contain "cssnano"

The call is just

npm audit --json

Environment
  • npm: 8.1.2
  • Node.js: 16.13.1
  • OS Name: Gentoo Linux
  • System Model Name: AMD Ryzen 9 3900X 12 Core, 64GB RAM
  • npm config: irrelevant, just shows node binary location and my pwd and HOME

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing npm audit --json repeatedly with the same package.json and package-lock.json, comparing the via, fixAvailable, and effects fields described in the report. Done means identical output across consecutive runs unless vulnerability data or fixed-version details have changed.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.