`output.clean : true` is not working with dev server

Bug report

What is the current behavior?

output.clean: true is not working with webpack dev server, when building for the first time, or when project is rebuilding due to sources change.

Description

I was trying to configure webpack to clean the dis folder for one of my projects, and decided to use brand new output.clean configuration as I'm using "webpack": "^5.21.2" and this config is supported starting from version 5.20

Config works fine in production build, but when I start webpack dev server with writeToDisk: true webpack doesn't clean the dist/ directory neither upon the start of the dev server nor upon project rebuild.

Initially I though that it might be not supposed to work with dev server, but changing

output: {
    ...
    clean: true
  },

to

output: {
    ...
    clean: {
      dry: true,
    }
  },

does log what files it should delete when rebuilding project:

If the current behavior is a bug, please provide the steps to reproduce.

1. clone my sandbox created for debugging this issue https://github.com/vovkvlad/webpack_clean-test
2. run npm i in the root of the repository
3. run npm start to start the application
4. go to src/App.js and make any change so that webpack catch it up and rebuild the project

Expected behavior:

old files from dist/ folder should me removed, and dist/ folder should contain one instance of each bundled file

Current behavior:

dist/ folder is not being cleaned, and contains several files for each bundled files:

What is the expected behavior?
old files from dist/ folder should me removed, and dist/ folder should contain one instance of each bundled file

Other relevant information:
webpack version: webpack 5.27.1
Node.js version: v14.15.0
Operating System: macOs Catalina Version 10.15.7
Additional tools:

---

This issue was moved from webpack/webpack#12949 by @alexander-akait. Original issue was by @vovkvlad.

Reproduce the issue using the linked sandbox: run npm i, npm start, then edit src/App.js and inspect the dist/ output with writeToDisk enabled. Compare output.clean: true with the dry-run behavior and verify that rebuilds leave only one instance of each bundled file in dist/.