Bug: Rsquare(N=0) expanding path leaks inf/garbage on near-constant windows (rolling path is guarded)
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 48.7k
- Forks
- 7.7k
- PR merge metrics
- No merged PRs in 30d
Description
Description
Rsquare in qlib/data/ops.py computes R² via a Cython kernel as
num / sqrt(var_x * var_y). For a near-constant window var_y ≈ 0, so
floating-point cancellation yields inf or a spurious finite value instead of
NaN (a degenerate 0/0 regression).
Rsquare._load_internal guards against this by masking windows whose std is ≈0
to NaN — but only on the rolling (N != 0) branch:
def _load_internal(self, instrument, start_index, end_index, *args):
_series = self.feature.load(instrument, start_index, end_index, *args)
if self.N == 0:
series = pd.Series(expanding_rsquare(_series.values), index=_series.index)
# <-- no guard here
else:
series = pd.Series(rolling_rsquare(_series.values, self.N), index=_series.index)
series.loc[np.isclose(_series.rolling(self.N, min_periods=1).std(), 0, atol=2e-05)] = np.nan
return series
The expanding (N == 0) branch is unguarded, so Rsquare($feature, 0) returns
inf/garbage on near-constant windows. Because ops.py sets
np.seterr(invalid="ignore"), no warning is emitted — the bad values silently
propagate into features (e.g. Alpha158/Alpha360) and downstream models.
Reproduction
Near-constant series [100, 100, 100, 100.000001, 100, 100]:
expanding_rsquare (N==0 path): [nan, nan, nan, inf, 0.01717987, inf]
rolling_rsquare(4) after mask: [nan, nan, nan, nan, nan, nan]
The expanding path leaks inf and a spurious 0.0172; the rolling path is
correctly NaN.
Fix
Apply the same std≈0 → NaN mask on the expanding branch (using expanding std).
Slope/Resi are unaffected — they divide by the x-variance (index 1..N),
which is always well-conditioned; only Rsquare divides by the y-variance.
PR incoming.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in qlib/data/ops.py at Rsquare._load_internal and compare the guarded rolling branch with the unguarded N == 0 expanding branch. Check the expanding standard deviation behavior on the reproduction series. Done means near-constant expanding windows return NaN rather than inf or spurious finite values, without changing the rolling behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, pandas, python
- Domain
- data, fintech-quant
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 75/100