Clean Up Includes

I can break the code if I change the sort of the `#include` directives. We should fix that.

For now, I would say just cleaning up unused includes where we can. Once we get rid of a bunch of the leftover cruft it should be much easier to address. We will probably have to hit the god files with all of those includes in such a gross way, but changing that god file _may_ require us to fix up the swig file which needs to be done anyways.