Multimodal Embeddings?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 10.5k
- Forks
- 1.2k
- Avg merge
- 4m
- Merged PRs (30d)
- 1
Description
I'd like to generate embeddings using a multimodal model such as llama3.2-vision or minicam-v for images and text, for instance a pdf document with embedded images.
As far as I can tell this isn't supported, or at least it isn't documented.
Can someone explain to me what is needed here?
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by reviewing the current embedding API and its documented model capabilities, then determine whether multimodal inputs such as images embedded in PDFs are supported. Done should be a decided implementation or documentation change with coverage for the supported input and model behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ollama, python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100