karpathy / karpathy/deep-vector-quantization

Missing 1x1 convolutions at the beginning of the decoder

Open
#6 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
655
Forks
60
PR merge metrics
No merged PRs in 30d

Description

I believe that there is at least one 1x1 conv missing. In the paper on p. 3 they mention the crucial importance of those but I could only find a projection here prior to the bottleneck.

https://github.com/karpathy/deep-vector-quantization/blob/c3c026a1ccea369bc892ad6dde5e6d6cd5a508a4/dvq/model/quantize.py#L93

As a side question: What is the reason that many Autoencoder architectures do away completely with normalization layers in both the encoder and the decoder? I tried to reseach this question but couldnt find a proper answer. Also does the size and complexity of both directly relate to the reconstruction quality? I have seen huge encoder/decoder structures which did not perform significantly better than the modest form you have in this repo or Phils simple architecture for that matter

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with dvq/model/quantize.py at the linked projection near line 93 and compare the decoder structure with the paper's description on page 3. Determine whether the decoder is missing the reported 1x1 convolutions, then verify the architectural change against the repository's available model or reconstruction workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.