USB device removal appears to cause problems in Windows.

Using the midi example code run from mpremote as per the instructions
Verify that the example logs transmit events over MIDI
Connect the Cockos Reaper DAW (most recent version, 7.20) [this can be downloaded for free]
Observe that the Board USB device appears in Reaper's MIDI device list
Create a track, assigning the board USB device as the input
Verify that MIDI events are correctly received
Exit Reaper
Unplug the Pico from the USB port
Plug the Pico back into the USB port
Attempt to connect via mpremote. The COM port previously used is now marked 'in use'
Attempt to restart the computer. It will hang during restart and require power to be cycled
The current USB implementation appears to leave the device in an undetermined state. Reaper is a very mature and stable product and is highly unlikely to be the cause here. Presumably some event which ought to occur when a USB device is disconnected does not occur for the Pico, leaving resources locked and causing shutdown and port acquisition to both fail.
I do appreciate that when the Pico was unplugged the midi example code was still running but for any practical implementation this will be the case as users are accustomed to unplugging USB devices at any time.

Reproduce the failure on Windows using the MIDI example through mpremote, Reaper 7.20, and the unplug/replug sequence described in the issue. Then trace the USB disconnect and reconnect handling; done means mpremote can reacquire the COM port and Windows can restart normally after the board is unplugged while the example is running.