QuantEcon / QuantEcon/QuantEcon.py

random.draw: size dispatch diverges between the Python and jitted paths for numpy integers

Open
#918 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug discuss
Dominant language
Python
Stars
2.4k
Forks
2.3k
Avg merge
3d 3h
Merged PRs (30d)
3

Description

quantecon.random.draw decides whether size requests an array of draws or a single scalar draw in two different places, using two different tests that do not agree:

Path Test Location
Pure Python body isinstance(size, int) quantecon/random/utilities.py
@overload implementation isinstance(size, types.Integer) quantecon/random/utilities.py

A NumPy integer is not a Python int, but it is a types.Integer to Numba. A Python bool is the reverse: it passes isinstance(x, int) but Numba types it as Boolean, not Integer. So the same call returns different things depending on whether the call site is jit-compiled — with no error either way.

Reproducing

Verified on numba 0.62.1 / numpy 2.3.5 / python 3.13.9, against current main:

import numpy as np
from numba import njit
from quantecon.random import draw

@njit
def draw_jit(cdf, size):
    return draw(cdf, size)

cdf = np.cumsum([0.4, 0.6])

draw(cdf, np.int64(10))       # np.int64(1)                    <- ONE scalar
draw_jit(cdf, np.int64(10))   # array([1, 0, 0, 1, 1, 1, 1, 1, 0, 0])   <- 10 draws

draw(cdf, True)               # TypeError
draw_jit(cdf, True)           # 1        <- a scalar draw

Full dispatch table:

size isinstance(size, int) Numba typeof isinstance(_, types.Integer) Python result Jitted result
10 True int64 True 10 draws 10 draws
np.int64(10) False int64 True 1 scalar 10 draws
np.int32(10) False int32 True 1 scalar 10 draws
np.uint8(10) False uint8 True 1 scalar 10 draws
True True bool False TypeError 1 scalar
np.bool_(True) False bool False 1 scalar 1 scalar
10.0 False float64 False 1 scalar 1 scalar

The numpy-integer rows are the ones that matter in practice. Nothing warns, nothing raises — the caller gets a scalar where they asked for an array, and downstream code sees a shape it did not expect.

Why this is easy to hit

A NumPy integer is what you get from most numpy-valued expressions, so size arrives as np.int64 without the caller doing anything unusual: an element of an integer array (counts[i]), a reduction (arr.sum(), arr.argmax()), a // result involving an array, or a value read out of a structured/record array. Plain len(x) and arr.shape[0] return Python ints and are fine, which is part of why this survives casual testing.

Suggested fix

Align the Python body with the overload:

if isinstance(size, (int, np.integer)) and not isinstance(size, bool):

The not isinstance(size, bool) clause is needed because bool subclasses int in Python but Numba types it as Boolean; without it the two paths would still disagree for size=True. I verified this predicate agrees with the overload's isinstance(t, types.Integer) on every case in the table above, including None.

There is a compatibility judgement for the team here rather than a purely mechanical fix. draw(cdf, np.int64(10)) currently returns a scalar from Python, and the fix changes that to a 10-element array. That is the behaviour the caller asked for and the behaviour the jitted path already gives, but it is a silent behaviour change for anyone who relied on the current result. It may deserve a release note, and possibly a decision on whether size=True should raise rather than quietly mean "one draw".

Provenance

Found while implementing #916 (PR #917), which adds a random_state argument to draw. This divergence predates that work and is untouched by it — deliberately left out so the behaviour change would not ride along in a feature PR and muddy the release note and the bisect story. probvec and sample_without_replacement are not affected; they build size internally and never dispatch on its type.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in quantecon/random/utilities.py at draw's pure Python body and its @overload implementation. Reproduce the listed NumPy integer and bool cases in both paths, then add regression coverage showing that their dispatch agrees; confirm the intended compatibility decision for size behavior and any release-note requirement.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.