Change the Optimization Method (AdamW --> diffGrad)

Adamw was replaced by a new optimization method - diffgrad. What do you think about it? Will you add it to increase the accuracy in the experiment?

Change

    optim_g = torch.optim.AdamW(
        net_g.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )
    optim_d = torch.optim.AdamW(
        net_d.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )

To

pip install torch-optimizer
...

import torch_optimizer as optimizer
...

    optim_g = optimizer.DiffGrad(
        net_g.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )
    optim_d = optimizer.DiffGrad(
        net_d.parameters(),
        hps.train.learning_rate,
        betas=hps.train.betas,
        eps=hps.train.eps,
    )

Source diffGrad: https://github.com/shivram1987/diffGrad

Locate the two AdamW optimizer initializations for net_g and net_d shown in the issue, then review the project's dependency setup before considering torch-optimizer. Verify that DiffGrad can be installed and used with the existing parameters, and define completion by both optimizers using it with the training experiment still running successfully.