huggingface / huggingface/computer-vision-course

add notebook examples

Open
#294 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
881
Forks
242
Avg merge
2m
Merged PRs (30d)
1

Description

https://github.com/johko/computer-vision-course/blob/main/chapters/en/unit4/multimodal-models/transfer_learning.mdx

| Task | Description | Model | Notebook |
| ----------- | ---------------------------------------------------------------- | ------------------------------------------------- | ----------- |
| Fine-tune CLIP | Fine-tuning CLIP on a custom dataset | [openai/clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32) | [CLIP notebook](https://) |
| VQA | Answering a question in natural
language based on an image | [dandelin/vilt-b32-finetuned-vqa](https://huggingface.co/dandelin/vilt-b32-finetuned-vqa) | [VQA notebook](https://) |
| Image-to-Text | Describing an image in natural language | [Salesforce/blip-image-captioning-large](https://huggingface.co/Salesforce/blip-image-captioning-large) | [Text 2 Image notebook](https://) |
| Open-set object detection | Detect objects by natural language input | [Grounding DINO](https://github.com/IDEA-Research/GroundingDINO) | [Grounding DINO notebook](https://) |
| Assistant (GTP-4V like) | Instruction tuning in the multimodal field | [LLaVA](https://github.com/haotian-liu/LLaVA) | [LLaVa notebook](https://) |

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.