microsoft / microsoft/qlib

Bug: Rsquare(N=0) expanding path leaks inf/garbage on near-constant windows (rolling path is guarded)

Open Beginner friendly
#2,297 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
48.7k
Forks
7.7k
PR merge metrics
No merged PRs in 30d

Description

Description

Rsquare in qlib/data/ops.py computes R² via a Cython kernel as
num / sqrt(var_x * var_y). For a near-constant window var_y ≈ 0, so
floating-point cancellation yields inf or a spurious finite value instead of
NaN (a degenerate 0/0 regression).

Rsquare._load_internal guards against this by masking windows whose std is ≈0
to NaN — but only on the rolling (N != 0) branch:

def _load_internal(self, instrument, start_index, end_index, *args):
    _series = self.feature.load(instrument, start_index, end_index, *args)
    if self.N == 0:
        series = pd.Series(expanding_rsquare(_series.values), index=_series.index)
        # <-- no guard here
    else:
        series = pd.Series(rolling_rsquare(_series.values, self.N), index=_series.index)
        series.loc[np.isclose(_series.rolling(self.N, min_periods=1).std(), 0, atol=2e-05)] = np.nan
    return series

The expanding (N == 0) branch is unguarded, so Rsquare($feature, 0) returns
inf/garbage on near-constant windows. Because ops.py sets
np.seterr(invalid="ignore"), no warning is emitted — the bad values silently
propagate into features (e.g. Alpha158/Alpha360) and downstream models.

Reproduction

Near-constant series [100, 100, 100, 100.000001, 100, 100]:

expanding_rsquare (N==0 path): [nan, nan, nan, inf, 0.01717987, inf]
rolling_rsquare(4) after mask: [nan, nan, nan, nan, nan, nan]

The expanding path leaks inf and a spurious 0.0172; the rolling path is
correctly NaN.

Fix

Apply the same std≈0 → NaN mask on the expanding branch (using expanding std).
Slope/Resi are unaffected — they divide by the x-variance (index 1..N),
which is always well-conditioned; only Rsquare divides by the y-variance.

PR incoming.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in qlib/data/ops.py at Rsquare._load_internal and compare the guarded rolling branch with the unguarded N == 0 expanding branch. Check the expanding standard deviation behavior on the reproduction series. Done means near-constant expanding windows return NaN rather than inf or spurious finite values, without changing the rolling behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, pandas, python
Domain
data, fintech-quant
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.