[Chapter 17] Question : Does an autoencoder based on advanced CNN such as SE-ResNET make any sense ?
- Dominant language
- Jupyter Notebook
- Stars
- 30k
- Forks
- 13.1k
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I was going to work on exercise 9 from Chapter 17 (denoising autoencoder), and wanted to try using the best classifier I had trained so far on MNIST digits which is an SE-ResNET, as a basis for the encoder.
Here are my questions :
- Does it make any sense trying to do an encoder using such complex NN as a basis ? I would, of course, change the last layer of the encoder with a wider `Dense` layer with _relu_ activation (instead of _softmax_). I just fear that this kind of classifier NN tends to remove essential data for further decoding steps ... ?
- Is it viable or counterproductive to simplify the decoder with a simple `Dense`/`Conv2DTranspose` combination (i.e. having the autoencoder non-symmetrical) ? or do I need to implement a reverse-ish the SE-ResNet for the decoder ?
Thanks for your answers/insights.
After posting this question I found two part-answers but without explanation on why is it good or bad :
- ResNet used as an encoder here : https://towardsdatascience.com/u-nets-with-resnet-encoders-and-cross-connections-d8ba94125a2c
- I also found that thread in reddit, but does not seem clear about the result of that kind of autoencoder implementation : https://www.reddit.com/r/MachineLearning/comments/54tdww/autoencoders_using_residual_networks/
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.