Steps scheduling features for LR Schedulers?

As far as I can see after an extensive research (discussions, not code), the options for steps scheduling during training are currently very limited.

**Cosine with restarts** can be tuned with _**lr_scheduler_num_cycles**_ for the restarts to happen more (or less) often than every epoch (default), which is great.

But what about **Constant** steps scheduling? How do I set a LR of 5e-5 for the first 1000 steps, LR of 5e-6 for the next 1000 and LR of 5e-7 till the end? A syntax like **_5e-5:1000, 5e-6:2000, 5e-7_** would've been so helpful! I actually can't believe there are no settings for Constant or Linear steps schedules. Am I missing something?

**P.S.** Cosine LR scheduler curve settings would've been incredible, but I'm asking too much already I think!

Start by tracing the training loop's existing learning-rate scheduler handling and the `lr_scheduler_num_cycles` option. Compare how Constant, Linear, and Cosine schedules are configured, then define how step-based rate changes should be represented and verified. Done means the requested multi-step schedule behavior is supported without breaking existing scheduler options.

Written by the indexing model from the issue text.