posit-dev / posit-dev/pointblank
Add a method to get error_reports per_column in a dataframe
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 490
- Forks
- 32
- Avg merge
- 14h 24m
- Merged PRs (30d)
- 2
Description
Prework
- [ X] Read and abide by the Pointblank code of conduct and contributing guidelines.
- [X ] Search for duplicates among the existing issues (both open and closed).
Proposal
I want to build an integration between pointblank validations and buckaroo. This will allow viewing the original dataframe inline, and highlighting errors. For reference Buckaroo is an interactive dataframe UI for notebooks, it lazily loads data into the browser and can scroll infinitely
To do this, I could use some help from pointblank.
Can you add a method that returns the original dataframe with "error_columns"
There could be a couple of approaches, but basically Buckaroo needs an additional column for each original column with a predictable name. This column should be null everywhere there isn't an error, when there is an error, just include the text explaining the error for that cell.
For pandas a multi-index of columns would be pretty cool, so make each column ('column_name', 'orig'), ('column_name', 'errors').
For polars you could make similar tuple columns.
Initially though just adding columns with a predictable name like 'column_name1__errors', 'column_name_foo__errors', will be easier for buckaroo. But that will inevitably run into some type of escaping error. Buckaroo can easily support this column name format right now. I'm very close to releasing multi-index column support and tuple column support.
What other cosiderations would go into a feature like that from your'alls side?
Here is a similar tool I built for pandera on top of Buckaroo
https://marimo.io/p/@paddy-mullen/buckaroo-pandera
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing pointblank’s existing validation and error-reporting APIs, then compare the linked Buckaroo/Pandera integration. Clarify the pandas and polars output contract, including predictable error-column naming, null cells, and escaping. Done means an agreed API design with coverage for the supported dataframe backends.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100