Quantco / Quantco/dataframely

Allow to pass through columns

Open
#100 8 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
618
Forks
21
Avg merge
14h 34m
Merged PRs (30d)
7

Description

I have a dataframe with a lot of columns (>100) and would like to not need to specify all of them.
Would it be possiuble to support a strict = False mode and/or to specify types for multiple columns?

Maybe such an api:

class FeaturesForwardModelSchema(dy.Schema):
    claim = dy.String(primary_key=True, nullable=False)
    cost = dy.Int64(primary_key=True, nullable=False)
    dataframely_options: DataframelyOptions(strict=False)

df = pl.DataFrame({
    "claim": ["a"],
    "cost": [1],
    "extra_col": ["foo"],
})

df_validated = FeaturesForwardModelSchema.validate(df)
df_validated["extra_col"]

alternatively, maybe something like this?

FeaturesForwardModelSchema.validate(df, strict=False)

Although this could be quite annoying, needing to specify it every time

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the FeaturesForwardModelSchema.validate entry point and the proposed DataframelyOptions configuration. Compare the two strict=False API alternatives and define how validation should retain unspecified columns; done means the example preserves extra_col while still validating claim and cost, with tests covering the chosen API.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.