karpathy / karpathy/deep-vector-quantization
Missing 1x1 convolutions at the beginning of the decoder
- Dominant language
- Jupyter Notebook
- Stars
- 655
- Forks
- 60
- PR merge metrics
- No merged PRs in 30d
Description
I believe that there is at least one 1x1 conv missing. In the paper on p. 3 they mention the crucial importance of those but I could only find a projection here prior to the bottleneck.
https://github.com/karpathy/deep-vector-quantization/blob/c3c026a1ccea369bc892ad6dde5e6d6cd5a508a4/dvq/model/quantize.py#L93
As a side question: What is the reason that many Autoencoder architectures do away completely with normalization layers in both the encoder and the decoder? I tried to reseach this question but couldnt find a proper answer. Also does the size and complexity of both directly relate to the reconstruction quality? I have seen huge encoder/decoder structures which did not perform significantly better than the modest form you have in this repo or Phils simple architecture for that matter
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with dvq/model/quantize.py at the linked projection near line 93 and compare the decoder structure with the paper's description on page 3. Determine whether the decoder is missing the reported 1x1 convolutions, then verify the architectural change against the repository's available model or reconstruction workflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100