[BUG] npm does not play nicely with pnpm/yarn@2 populated package trees (link targets appear to not resolve correctly)

In some specific enviroment the code [D:\Program Files\nodejs\node_modules\npm\node_modules@npmcli\arborist\lib\node.js:718:35] will throw a null error when we call npm ls -g or npm i -g because some old dependency links in the node_modules can not be parsed and lost target they linked to.

In my case it's C:\Users\QQ370\AppData\Roaming\npm\node_modules\babel-cli\node_modules\_babel-messages@6.23.0@babel-messages, I update the babel-cli and problem solved.

In fact, it is not a bug caused by arborist, but I suggest to add the this.realpath of node to the error logs so that developers could resolve it quickly. Now we got only a single-line message said npm:ERR Cannot read properties of null (reading 'package') and had no clue for what happend. I finally figure it out by debug npm in VSCode, it cost much time for me (I spent lot of time to search online, clear npm caches, change npm root, reinstall the npm/nodejs. downgrade it from v8.5.0 to v7.24.2).

And after I resolved it, I can not recover the enviroment and I have not record the specific version of babel-cli. I'm sorry for that.

There do be someone got the same issue, such as the replier of ISSUE 3194.

Here is the key code:

        if (this.isLink) {
          const target = node.target
          this[_target] = target
          // Errors occured here cuz the target of sub node (node.target) is null
          // But it's obviously that *this* node is a file-link, and the sub node of it should own a target
          this[_package] = target.package
          target.linksIn.add(this)
          // reload edges here, because now we have a target
          if (this.parent) {
            this.parent[_reloadNamedEdges](this.name)
          }
          break
        } else {

Here is the log:

0 verbose cli [
0 verbose cli   'D:\\Program Files\\nodejs\\node.exe',
0 verbose cli   'D:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
0 verbose cli   'ls',
0 verbose cli   '-g'
0 verbose cli ]
1 info using npm@8.3.1
2 info using node@v16.14.0
3 timing npm:load:whichnode Completed in 1ms
4 timing config:load:defaults Completed in 1ms
5 timing config:load:file:D:\Program Files\nodejs\node_modules\npm\npmrc Completed in 3ms
6 timing config:load:builtin Completed in 3ms
7 timing config:load:cli Completed in 2ms
8 timing config:load:env Completed in 0ms
9 timing config:load:project Completed in 2ms
10 timing config:load:file:C:\Users\QQ370\.npmrc Completed in 1ms
11 timing config:load:user Completed in 1ms
12 timing config:load:file:C:\Users\QQ370\AppData\Roaming\npm\etc\npmrc Completed in 0ms
13 timing config:load:global Completed in 0ms
14 timing config:load:validate Completed in 1ms
15 timing config:load:credentials Completed in 1ms
16 timing config:load:setEnvs Completed in 0ms
17 timing config:load Completed in 13ms
18 timing npm:load:configload Completed in 14ms
19 timing npm:load:setTitle Completed in 0ms
20 timing config:load:flatten Completed in 4ms
21 timing npm:load:display Completed in 6ms
22 verbose logfile C:\Users\QQ370\AppData\Local\npm-cache\_logs\2022-02-15T05_08_47_962Z-debug-0.log
23 timing npm:load:logFile Completed in 7ms
24 timing npm:load:timers Completed in 0ms
25 timing npm:load:configScope Completed in 0ms
26 timing npm:load Completed in 28ms
27 timing arborist:ctor Completed in 1ms
28 silly logfile start cleaning logs, removing 2 files
29 timing command:ls Completed in 7420ms
30 verbose stack TypeError: Cannot read properties of null (reading 'package')
30 verbose stack     at Link.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:718:35)
30 verbose stack     at Node.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:784:20)
30 verbose stack     at Node.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:784:20)
30 verbose stack     at Node.set root [as root] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:592:22)
30 verbose stack     at Node.set parent [as parent] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:1235:15)
30 verbose stack     at new Node (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\node.js:226:17)
30 verbose stack     at Arborist.[newNode] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\load-actual.js:322:9)
30 verbose stack     at D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\load-actual.js:283:68
30 verbose stack     at async Promise.all (index 0)
30 verbose stack     at async Arborist.[findMissingEdges] (D:\Program Files\nodejs\node_modules\npm\node_modules\@npmcli\arborist\lib\arborist\load-actual.js:446:7)
31 verbose cwd C:\WINDOWS\system32
32 verbose Windows_NT 10.0.19042
33 verbose argv "D:\\Program Files\\nodejs\\node.exe" "D:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "ls" "-g"
34 verbose node v16.14.0
35 verbose npm  v8.3.1
36 error Cannot read properties of null (reading 'package')
37 verbose exit 1
38 timing npm Completed in 8073ms
39 verbose code 1
40 error A complete log of this run can be found in:
40 error     C:\Users\QQ370\AppData\Local\npm-cache\_logs\2022-02-15T05_08_47_962Z-debug-0.log

Start in @npmcli/arborist/lib/node.js around the Link root setter at line 718, then trace the npm ls -g and npm i -g entry paths shown in the report. Reproduce the null-target case if possible and inspect how the existing error is logged. Done means the failure log includes the affected node's realpath and the null error remains diagnosable.