[DAP] Stepping messed up after restart request

Describe the bug
Using neovim, debugging itself works all fine, unless I restart the session (using nvim-dap extension with require('dap').restart()). When I restart the session, it hits the first breakpoint and kind of detaches(?): i.e. I can click step over and it will just do nothing (although indicating that execution is not stopped at the breakpoint anymore). I have to click "terminate session" twice to see [Process terminated] in the terminal. I tried the same launch.json in vscode and restarting works fine

To Reproduce
Steps to reproduce the behavior:

1. Have a file like test.js

const a = 1;
console.log(a);
const b = 2;
console.log(b);

2. Add breakpoint to the second line
3. Start the debugger with this config

      require("dap").adapters["pwa-node"] = {
        type = "server",
        host = "localhost",
        port = "${port}",
        executable = {
          command = "node",
          args = {
            vim.call("stdpath", "data") .. "/lazy/vscode-js-debug/out/src/dapDebugServer.js",
            "${port}",
          },
        }
      }

And launch config:

          {
            type = "pwa-node",
            name = "Node",
            request = "launch",
            program = "${file}",
            cwd = "${workspaceFolder}",
            console = "integratedTerminal",
          },

4. Hit breakpoint, step over fine.
5. Restart the session via dap.restart(), hit the breakpoint, click Step Over and observe session detaching...

Log File
Log from nvim-dap, includes: start the session, hitting breakpoint, stepping over, restarting, hitting breakpoint, stepping over, terminating, terminating again. loadedSource event and stackTrace commands removed as too lengthy

Version I built dapDebugServer from the latest commit

Additional context
Sorry if nvim-dap issue, but I use like the most basic config