Implementation of new optimizer: Sophia

Github: https://github.com/Liuhong99/Sophia

According to the paper on arxiv: https://arxiv.org/pdf/2305.14342.pdf It has a 2 time speed up compared to AdamW.

Start by reading the linked Sophia repository and the arXiv paper to understand the optimizer's required behavior. Then inspect sd-scripts' existing optimizer integration points and tests, if present. Done means Sophia is integrated with the project's training workflow and its behavior is covered by appropriate validation.

Written by the indexing model from the issue text.