huggingface/nanoVLM

View on GitHub

The simplest, fastest repository for training/finetuning small-sized VLMs.

Stars: 5k
Forks: 510
Open beginner issues: 0
Indexed issues: 36

Dominant language: Python
License: Apache-2.0
Last GitHub push: Oct 27, 2025
Latest indexed: Sep 17, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

36 open issues indexed

Support for TRL / GRPO training? Open

huggingface/nanoVLM#10 · 3 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
Enable streaming mode to avoid full dataset download when using data_cutoff_idx Open

huggingface/nanoVLM#54 · 1 comment · 2 reactions · 0 assignees · Indexed Sep 17, 2026
`torch.compile` does not consistently work with DDP Open

huggingface/nanoVLM#61 · 13 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Add optional NdLinear support for standard Linear layers in ModalityProjector and ViTMLP Open

huggingface/nanoVLM#68 · 2 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
add qwen2-VL model Open

huggingface/nanoVLM#78 · 2 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
scaled_dot_product_attention throws an error when both attn_mask and is_causal are set Open

huggingface/nanoVLM#80 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
VLMEvalKit support for nanoVLM Open

huggingface/nanoVLM#89 · 8 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
KV Cache getting prefilled and emptied during training phase Open

huggingface/nanoVLM#92 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Mixture of Modality-Experts (MoME) Integration Open

huggingface/nanoVLM#96 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
[Proposal] Contribute to nanoALM – Add Audio Modality Support Open

huggingface/nanoVLM#99 · 6 comments · 8 reactions · 0 assignees · Indexed Sep 17, 2026
Is fsdp currently supported? Open

huggingface/nanoVLM#117 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
AttributeError: 'GPT2TokenizerFast' object has no attribute 'image_token_id' Open

huggingface/nanoVLM#136 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
nanoVLM.ipynb configuration is out of sync with library, causing TypeError Open

huggingface/nanoVLM#144 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Dataset curation / smollm3 Open

huggingface/nanoVLM#150 · 1 comment · 2 reactions · 0 assignees · Indexed Sep 17, 2026
Bug at ModalityProjector _init_weights method Open

huggingface/nanoVLM#158 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
about the model Open

huggingface/nanoVLM#160 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
lm_eos_token_id Open

huggingface/nanoVLM#166 · 4 comments · 2 reactions · 0 assignees · Indexed Sep 17, 2026
Help needed: NaN observed in training loss Open

huggingface/nanoVLM#167 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
NEED HELP! Empty Batches (([], [], [], [])) During Validation Open

huggingface/nanoVLM#171 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Why is padding applied to the left of text inputs? Open

huggingface/nanoVLM#172 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
lusxvr/nanoVLM-222M image patching issue Open

huggingface/nanoVLM#174 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Can I Load SmolVLM using nanoVLM and Do Post-training? Open

huggingface/nanoVLM#176 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Wrong answer for assets/image.png Open

huggingface/nanoVLM#177 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Is it possible to include video models such as SmolVLM2? Open

huggingface/nanoVLM#180 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
AttributeError: 'GPT2TokenizerFast' object has no attribute 'r1c1' Open

huggingface/nanoVLM#183 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Recreate README Training Result Curves in Similar Time Open

huggingface/nanoVLM#186 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
nan loss when I run the scripts of colab in my own environment. Open

huggingface/nanoVLM#187 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
can not load datasets Open

huggingface/nanoVLM#191 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
SmolVLM 2 Open

huggingface/nanoVLM#194 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Training Curves Open

huggingface/nanoVLM#195 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Is there a reason my model isn't stopping generation even after producing a complete response? Open

huggingface/nanoVLM#197 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Training Config for lusxvr/nanoVLM-230M-8k Open

huggingface/nanoVLM#198 · 13 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
VQADataset.iter_for_worker() lacks explicit worker sharding, causing data duplication in multi-worker DataLoader Open

huggingface/nanoVLM#200 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Are position IDs handled correctly for packed sequences during training? Open

huggingface/nanoVLM#201 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
out of memory Open

huggingface/nanoVLM#202 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
Another Post on NanoVLM Open

huggingface/nanoVLM#208 · 1 comment · 1 reaction · 0 assignees · Indexed Sep 17, 2026

huggingface/nanoVLM

Get new issues in your inbox