Specutils doesn't open (gildas) CLASS-generated FITS spectra

Hi all,
I'm working with millimeter-radio spectra from the IRAM 30m telescope, and my FITS spectra were generated by CLASS (part of the gildas package). When I try to load a CLASS-exported FITS file into specutils, it chokes on a SingularMatrixError. ([Further details in this gist](https://gist.github.com/tomr-stargazer/7a4efed0f8a1f5d410066ca87515f692), including the FITS header in question.)

The spectrum is attached (and zip'd):
[spectrum.zip](https://github.com/astropy/specutils/files/1324303/spectrum.zip)

@keflavich recommended I file an issue here so here it is. (:

Reproduce the failure with the attached spectrum.zip and review the linked gist, especially the CLASS-generated FITS header. Trace where specutils raises SingularMatrixError while loading the spectrum. Done means the attached CLASS-generated FITS spectrum opens successfully in specutils.

Written by the indexing model from the issue text.