import of specutils is slow

Importing `spectuils` is slow, taking 1.3-2 seconds. See attached screenshot, captured with `python -X importtime -c "import specutils" 2> imports.txt` and shown with tuna. For users developing packages that want to use and build off `Spectrum1D` it's unfortunate that importing `Spectrum1D` adds upwards of a second to the import time. If you drill down it looks like a big slowdown comes from importing `pkg_resources` in `astropy.modeling`, as well as `astropy.coordinates` within `astropy.nddata`. Has there been any discussion on ways of improving this?

Reproduce the report with `python -X importtime -c "import specutils" 2> imports.txt` and inspect the result with tuna. Trace the import paths involving `Spectrum1D`, `pkg_resources` in `astropy.modeling`, and `astropy.coordinates` within `astropy.nddata`; done means establishing a concrete import-time improvement without changing the public behavior.

Written by the indexing model from the issue text.