polyfit LinAlgError: SVD did not converge in Linear Least Squares
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 32.8k
- Forks
- 12.8k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 197
Description
After running my code on a new computer I had an issue with polyfit, which raises an error that should not happen.
Initially I ran my code on a Mac, with numpy 1.15.1.
The new computer is a laptop running on windows 10.
I tried several versions of numpy (1.19.1, 1.18.5, 1.17.5) but it didn't help and I could not dowlod 1.16.6 or 1.15.1 (because they are too old I guess)
My initial code was ;
import numpy as np
...
p0=np.polyfit(omega[nInit:nFin]-CentOmega,phi1[nInit:nFin],2,w=Spectrum[nInit:nFin])
omega, phi1 and Spectrum are numpy arrays (shape (4096,) ), nInit and nFin are within range and CentOmega is a scalar. I checked that there is no NaN values or infinites and that spectrum is positive.
Error message:
LinAlgError Traceback (most recent call last)
in calcCS2()
285 #print(np.min(phi1[nInit:nFin]))
286 #Here we remove the GDD to find the pulse shape with only higher order disp
--> 287 p0=np.polyfit(omega[nInit:nFin]-CentOmega,phi1[nInit:nFin],2,w=Spectrum[nInit:nFin]) #fitting
288 while True:
289 try:
<array_function internals> in polyfit(*args, **kwargs)
c:\users\pierre-marc\appdata\local\programs\python\python38\lib\site-packages\numpy\lib\polynomial.py in polyfit(x, y, deg, rcond, full, w, cov)
629 scale = NX.sqrt((lhs*lhs).sum(axis=0))
630 lhs /= scale
--> 631 c, resids, rank, s = lstsq(lhs, rhs, rcond)
632 c = (c.T/scale).T # broadcast scale coefficients
633
<array_function internals> in lstsq(*args, **kwargs)
c:\users\pierre-marc\appdata\local\programs\python\python38\lib\site-packages\numpy\linalg\linalg.py in lstsq(a, b, rcond)
2266 # lapack can't handle n_rhs = 0 - so allocate the array one larger in that axis
2267 b = zeros(b.shape[:-2] + (m, n_rhs + 1), dtype=b.dtype)
-> 2268 x, resids, rank, s = gufunc(a, b, rcond, signature=signature, extobj=extobj)
2269 if m == 0:
2270 x[...] = 0
c:\users\pierre-marc\appdata\local\programs\python\python38\lib\site-packages\numpy\linalg\linalg.py in _raise_linalgerror_lstsq(err, flag)
107
108 def _raise_linalgerror_lstsq(err, flag):
--> 109 raise LinAlgError("SVD did not converge in Linear Least Squares")
110
111 def get_linalg_error_extobj(callback):
LinAlgError: SVD did not converge in Linear Least Squares
NumPy 1.17.5 / Python 3.8.6
I found a way to avoid this error by doing the following:
while True:
try:
p0=np.polyfit(omega[nInit:nFin]-CentOmega,phi1[nInit:nFin],2,w=Spectrum[nInit:nFin]) #fitting
break
except:
continue
I saw this on stackoverflow for people who had the same issue. They noticed running the same code a second time works.
This is really a stupid code, and should not work (but it does). All it says is keep doing the same thing until you don't get an error
I don't understand why successive calls to polyfit with the same parameters would give a different result.
I didn't find this bug in your list of known issues but maybe someone else reported it?
I hope you can fix this in later versions.
Best regards,
Pierre-Marc Dansette
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the np.polyfit call and inspect the traceback locations in numpy/lib/polynomial.py and numpy/linalg/linalg.py. First obtain a minimal reproducible set of omega, phi1, Spectrum, nInit, nFin, and CentOmega that triggers the Windows failure, then compare repeated calls across the reported NumPy versions. Done means the cause is isolated or the failure is reproducibly characterized.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100