BUG: Lowess extrapolation missing output values.

Open

#7,337 opened on Feb 20, 2021 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty: 3/5
Estimated time: 1-2 days
Newbie friendliness: 35/100

Issue type: Bug
Clarity: Mostly clear
Activity status: Stale
Tech stack: python
Domain: data

Research direction

Start by running the supplied examples for statsmodels.nonparametric.smoothers_lowess.lowess, especially the xvals and frac combinations. Inspect the lowess implementation and add regression coverage for these cases; done means output is defined at least across the input X range and the reported NaN behavior is addressed.

Written by the indexing model from the issue text.

Description

Bug description

Suppose input values are defined on [5, 14], Then if you use lowess and use xvals to get values on a bigger range of Xs (for example [0, 19]), the output may contain undefined values even on the input range of Xs. Additionally, different frac values produce different behaviours that are not obvious.

In the last example, output is undefined even if you use the same output X values as input X values.

Code example

from statsmodels.nonparametric.smoothers_lowess import lowess
import numpy as np
import sys

np.set_printoptions(precision=6, linewidth=sys.maxsize, suppress=True, threshold=sys.maxsize)

x = np.arange(5, 15).astype(np.float)
y = np.arange(5, 15).astype(np.float)
out_x = np.arange(0, 20).astype(np.float)

lowess(y, x, xvals=out_x, frac=0.6)
# array([-0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19.])

lowess(y, x, xvals=out_x, frac=0.5)
# array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., nan, nan, nan, nan, nan, nan, nan, nan])

lowess(y, x, xvals=out_x, frac=0.4)
# array([ 0.,  1.,  2.,  3.,  4.,  5.,  6.,  7.,  8.,  9., 10., 11., 12., 13., 14., 15., 16., 17., 18., 19.])

lowess(y, x, xvals=out_x, frac=0.3)
# array([nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan, nan])

lowess(y, x, xvals=x, frac=0.5)
# array([ 5.,  6.,  7.,  8.,  9., 10., 11., nan, nan, nan])

Expected output

Ideally, lowess output should be defined at least on the range of input Xs.

Also, values do not become NaNs gradually (for example undefined below some frac value and defined above).
Because of that it looks like a bug and not like an expected behaviour.

Output of `import statsmodels.api as sm; sm.show_versions()`


INSTALLED VERSIONS
------------------
Python: 3.6.10.final.0
OS: Linux 5.4.95-1-MANJARO #1 SMP PREEMPT Thu Feb 4 00:38:02 UTC 2021 x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

statsmodels
===========

Installed: 0.12.2 (/home/timofey/.miniconda3/lib/python3.6/site-packages/statsmodels)

Required Dependencies
=====================

cython: 0.29.21 (/home/timofey/.miniconda3/lib/python3.6/site-packages/Cython)
numpy: 1.19.1 (/home/timofey/.miniconda3/lib/python3.6/site-packages/numpy)
scipy: 1.5.2 (/home/timofey/.miniconda3/lib/python3.6/site-packages/scipy)
pandas: 1.1.5 (/home/timofey/.miniconda3/lib/python3.6/site-packages/pandas)
    dateutil: 2.8.1 (/home/timofey/.miniconda3/lib/python3.6/site-packages/dateutil)
patsy: 0.5.1 (/home/timofey/.miniconda3/lib/python3.6/site-packages/patsy)

Optional Dependencies
=====================

matplotlib: Not installed
cvxopt: Not installed
joblib: Not installed

Developer Tools
================

IPython: 7.16.1 (/home/timofey/.miniconda3/lib/python3.6/site-packages/IPython)
    jinja2: 2.11.3 (/home/timofey/.miniconda3/lib/python3.6/site-packages/jinja2)
sphinx: Not installed
    pygments: 2.7.4 (/home/timofey/.miniconda3/lib/python3.6/site-packages/pygments)
pytest: Not installed
virtualenv: 16.7.5 (/home/timofey/.miniconda3/lib/python3.6/site-packages)

Dominant language: Python
Stars: 11.6k
Forks: 3.6k
Avg merge: 7h 37m
Merged PRs (30d): 96

Contributor guide

Open the contributing guide

First steps

Read the whole issue, then the project's contributing guide.
Comment on the issue to say you are picking it up — it saves two people doing the same work.
Fork the repository and make your change on a branch.
Open a pull request that references the issue number.

Similar issues

[Bug]: notebooks/06_visualization_quickstart.ipynb fails: SHAPVisualizer.plot_waterfall raises IndexError Open

Difficulty 2/5 1-3 hours Newbie friendliness 74/100

bancolombia/sentinel#23 · Opened Sep 20, 2026
test md Open

CI

Difficulty 2/5 1-3 hours Newbie friendliness 74/100

sympy/sympy#30547 · 1 comment · Opened Sep 20, 2026
quickjs: support `timeout=None` (no per-call deadline) in `CodeInterpreterMiddleware` Open

integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature

Difficulty 2/5 1-3 hours Newbie friendliness 74/100

langchain-ai/deepagents#6450 · Opened Sep 20, 2026
OpenAISamplingHandler drops ToolResultContent when a user message's list content also carries text/image blocks Open

bug client

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

PrefectHQ/fastmcp#5182 · 1 comment · Opened Sep 20, 2026
last-mile drop-off on this SKILL.md Open

Difficulty 2/5 1-3 hours Newbie friendliness 74/100

wshobson/agents#717 · Opened Sep 20, 2026

More Python issues