posit-dev / posit-dev/pointblank
Multimodal inputs in the `DraftValidation` class
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 490
- Forks
- 32
- Avg merge
- 14h 24m
- Merged PRs (30d)
- 2
Description
Prework
- Read and abide by the Pointblank code of conduct and contributing guidelines.
- Search for duplicates among the existing issues (both open and closed).
Proposal
Continuing on the theme of multimodal inputs, I thought it'd be really cool if the DraftValidation class could support multimodal inputs (images and PDF), just as Validate.prompt() now does. A rough sketch of how that would work, drawing inspiration from the How DraftValidation Works steps:
- A statistical summary of your data is generated using the DataScan class.
- This summary is converted to JSON format and sent to your selected LLM provider.
- NEW: You provide the
DraftValidationclass with an image or PDF file describing what the data should look like. For example, the image could be a schema diagram, while the PDF could outline the project configuration elements that the data should conform to. - The LLM uses the summary and multimodal input along with knowledge about Pointblank’s validation capabilities to generate a validation plan.
- The result is returned as executable Python code that you can use directly or modify as needed
Happy to get feedback from you @rich-iannone, and iterate. I'd love to implement this.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the DraftValidation and Validate.prompt() entry points, then review DataScan and the linked How DraftValidation Works guide to understand the existing flow. Done means defining and implementing support for image or PDF inputs so the LLM can use them with the data summary to produce executable validation code.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100