corvus-dotnet / corvus-dotnet/Corvus.Python

Add convert_dataframeschema_to_structtype function for Pandera to PySpark

Open
#26 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

We should add a function to convert a Pandera DataFrameSchema to a PySpark StructType.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no source files, tests, or entry points. Start by locating the Pandera and PySpark integration code and existing schema-conversion utilities, then determine the supported DataFrameSchema-to-StructType mappings and add coverage that verifies the conversion is complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.