akanimax / akanimax/Variational_Discriminator_Bottleneck

Resizing support/progressive growing support

Aperta
#2 4 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
Python
Stelle
151
Fork
31
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Hiya. I've been experimenting with generating anime faces with GANs for years now, and I've been trying out your GAN implementation to compare with another (more complex) VGAN implementation by nsheppard (currently called PokeGAN), using ~137k anime faces extracted from my [Danooru2017](https://www.gwern.net/Danbooru2017) dataset. The results so far are as good as anything I've gotten aside from Nvidia's ProGAN implementation; samples from a few minutes ago after about 2 days:

![Epoch 37 sample of anime faces with akanimax's VGAN](https://user-images.githubusercontent.com/352559/48752213-1a74b980-ec56-11e8-8bb9-f54dfc0ea0ea.png)

For speed, I am using 128px. At some point when it's converged, it'd be nice to switch to 256px and then 512px without restarting from scratch. Some support for adding on additional layers would be great; the exact 'blending' ProGAN does might be a bit hard to implement, but almost as good would be freezing the original layers for a while and training only the new ones; or even just slapping on more layers would be better than restarting from scratch (given how minibatches go down drastically with increased resolution).

---

On a side note, there's one trick nsheppard found which might be useful. We don't know if the official VGAN code does this because it seems kind of obvious in retrospect but they haven't released source AFAIK. But in the bottleneck loss with the KL loss, it seems that occasionally `i_c` can be larger than the KL loss, in which case the loss is then negative? It's not clear what this means or how it might be desirable, so he just got rid of it by bounding it to zero:

```.diff
- bottleneck_loss = (th.mean(kl_divergence) - i_c)
+ bottleneck_loss = max(0, (th.mean(kl_divergence) - i_c)) # EDIT: per nsheppard, ensure that the loss can't be negative
```

It seems to permit stable training with much higher values of `i_c`.

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.