joshsoftware / joshsoftware/hackathon-team-5
Llama Vision Model Integration for Actions
Open
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Integrate the Llama Vision Model to interpret UI screenshots and generate browser actions.
1. Set up Llama Vision Model in the FastAPI backend to process UI screenshots.
2. Create an endpoint to accept images, predict coordinates, and return actions (e.g., click, scroll).
3. Write logic to determine coordinates and actions for Puppeteer based on Llama Vision predictions.
4. Test basic tasks like clicking buttons or scrolling on sample pages.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.