[colorManipulator] Inconsistent output values with the `darken` function

Current behavior 😯

When using MUI's internal darken function (material-ui/packages/mui-system/src/colorManipulator.js) we see very different color outputs depending on the format of the input color value. For example:

/* All these values are the same color and written in different formats */
let hex = "#7B61FF";
let rgb = "rgb(123, 97, 255)";
let hsl = "hsl(250, 100%, 69%)";

let coefficient = 0.1; // Can be any value

darken(hex, coefficient) // returns rgb(110, 87, 229)
darken(rgb, coefficient) // returns rgb(110, 87, 229)
darken(hsl, coefficient) // returns hsl(250, 100%, 62.1%) which is very different than the first two

Preview (Left column is inputs, right column is outputs):

This is due to the way that the darken function manipulates colors. If it detects an hsl color, it will lower the lightness by the coefficient. However, if it is RGB (or is cast to RGB) the function will adjust each separate color component by the coefficient. The latter is a more naïve approach to darkening since the result is usually desaturated.

Expected behavior 🤔

Color adjustments should output a consistent color value regardless of input format. TinyColor's darken function is a good example; it always casts an input value to HSL and then makes the adjustment, outputting the same results for every color format.

Steps to reproduce 🕹

Live example: https://codesandbox.io/s/magical-dubinsky-zrehcj?file=/src/index.js

Context 🔦

This is causing issues with keeping our design files in sync with our Figma mockups because not all color values are in the same base color format.

Your environment 🌎

No response

Start in packages/mui-system/src/colorManipulator.js and reproduce the reported darken examples from the linked CodeSandbox using hex, RGB, and HSL inputs. Compare the outputs with the expected consistent color adjustment described in the issue; done means equivalent input colors produce consistent darken results.