microsoft / microsoft/TypeChat
Feat: Add support for Vision API and Input Images
Open
Nobody has claimed this yet.
In Discussion
Suggestion
- Dominant language
- TypeScript
- Stars
- 8.7k
- Forks
- 414
- Avg merge
- 3h 8m
- Merged PRs (30d)
- 3
Description
Hello,
We are trying to extract features from an Image using Vision API of OpenAI.
Currently, TypeChat only supports text information. It would be much appreciated if we can allow users to pass extra content like images.
I would like to work on adding this feature.
Approach
- In the
translatefunction we can allow users to pass extra content parameters like images... this can be extended to passing context to the model too... like if user wants to pass some user prompt or whole context of previous conversation
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the translate function and the linked OpenAI Vision API documentation. Clarify whether the scope is limited to image input or also includes user prompts and prior conversation context; done means users can pass image content through the supported API.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100