pydata / pydata/patsy

inf treated as a number, which is very weird

Open
#118 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
990
Forks
106
Avg merge
7d 34m
Merged PRs (30d)
1

Description

As noted here: https://stackoverflow.com/questions/48371747/how-to-modify-a-liner-regression-in-python-3-6

This formula causes patsy to raise an error:

patsy.ModelDesc.from_formula("inf ~ x")

the problem is that in patsy.parse_formula._read_python_expr, patsy tries to figure out whether an arbitrary Python expression is a numeric literal, and the way it does this is by calling int(...) and float(...) on the expression, and seeing if they work.

In this case, float("inf") does work, so patsy decides that the Python expression inf is a numeric literal. Whoops.

The same thing probably happens if you try to use nan as a variable name in a formula.

I guess a more reliable way of checking for numeric literals would be to check the tokenize output: if an expression is a single token, and that token has type tokenize.NUMBER, then it's a numeric literal.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in patsy.parse_formula._read_python_expr and reproduce the failure with patsy.ModelDesc.from_formula("inf ~ x"). Inspect the tokenize output for the expression and compare it with the current int() and float() checks. Done means inf and nan can be used as formula variable names without being classified as numeric literals.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.