Do not change type of python exceptions when returning to Python

Issue description

If Python raises an exception, pybind11 handles that in C++ as pybind11::error_already_set and in PYBIND11_CATCH_INIT_EXCEPTIONS this is always sent to Python as an ImportError with the text of the pre-existing exception added to the ImportError.

Unfortunately, this means that the nuance of the Python exception is completely lost when calling py::import::module(). For example ModuleNotFoundError is rebranded as an ImportError and in particular ImportWarning or RuntimeWarning are forced to be ImportError. I feel that Pybind11 should behave more closely to Python.

As an example, here is what happens if warnings are turned fatal and py::import::module() imports a package that generates a warning:

Traceback (most recent call last):
  File "tests/test.py", line 3, in <module>
    import cmake_example as m
ImportError: ImportWarning: Not importing directory /Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/mpl_toolkits: missing __init__

At:
  <frozen importlib._bootstrap_external>(426): _find_module_shim
  /Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py(2111): _handle_ns
  /Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py(2194): declare_namespace
  /Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py(2633): activate
  /Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py(3102): <genexpr>

This does include the warning message but it is reported as an ImportError. The stack trace does not tell me anything about where the warning is coming from because it is truncated and also reversed from the python standard.

With this patch (ie, let python keep its exception as is):

diff --git a/include/pybind11/detail/common.h b/include/pybind11/detail/common.h
index 3c67228..d38c279 100644
--- a/include/pybind11/detail/common.h
+++ b/include/pybind11/detail/common.h
@@ -229,7 +229,7 @@ extern "C" {
 
 #define PYBIND11_CATCH_INIT_EXCEPTIONS \
         catch (pybind11::error_already_set &e) {                               \
-            PyErr_SetString(PyExc_ImportError, e.what());                      \
+            e.restore();                                                       \
             return nullptr;                                                    \
         } catch (const std::exception &e) {                                    \
             PyErr_SetString(PyExc_ImportError, e.what());                      \

the stack trace becomes much more informative and Pythonic, and the trace is presented in the expected order and is complete:

Traceback (most recent call last):
  File "tests/test.py", line 3, in <module>
    import cmake_example as m
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/astropy/__init__.py", line 118, in <module>
    _check_numpy()
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/astropy/__init__.py", line 107, in _check_numpy
    requirement_met = minversion(numpy, __minimum_numpy_version__)
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/astropy/utils/introspection.py", line 147, in minversion
    from pkg_resources import parse_version
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3074, in <module>
    @_call_aside
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3058, in _call_aside
    f(*args, **kwargs)
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3102, in _initialize_master_working_set
    for dist in working_set
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3102, in <genexpr>
    for dist in working_set
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2633, in activate
    declare_namespace(pkg)
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2191, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2111, in _handle_ns
    loader = importer.find_module(packageName)
  File "<frozen importlib._bootstrap_external>", line 426, in _find_module_shim
ImportWarning: Not importing directory /Users/timj/work/lsstsw3/miniconda/envs/lsst-scipipe/lib/python3.6/site-packages/mpl_toolkits: missing __init__

Finally, if you import a pybind11 module, that imports some python that imports some pybind11 that imports a python module that fails, you end up with ImportError: ImportError: ImportWarning as the ImportErrors keep stacking up.

Reproducible example code

This is a simple change that I made to cmake_example to demonstrate the issue. Any import that generates a warning will suffice (such as formatter in the standard distribution; although that doesn't have such a large stack trace so doesn't give the full effect). I have tested on python3.6 and 3.7.

diff --git a/src/main.cpp b/src/main.cpp
index 86ab582..ad93c6c 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -7,6 +7,8 @@ int add(int i, int j) {
 namespace py = pybind11;
 
 PYBIND11_MODULE(cmake_example, m) {
+
+    py::module::import("astropy.units");
     m.doc() = R"pbdoc(
         Pybind11 example plugin
         -----------------------
diff --git a/tests/test.py b/tests/test.py
index cea38c4..747cf0e 100644
--- a/tests/test.py
+++ b/tests/test.py
@@ -1,3 +1,5 @@
+import warnings
+warnings.simplefilter("error")
 import cmake_example as m
 
 assert m.__version__ == '0.0.1'

Start in include/pybind11/detail/common.h at PYBIND11_CATCH_INIT_EXCEPTIONS, then reproduce the behavior with the cmake example in src/main.cpp and tests/test.py. Done means Python exceptions retain their original type and complete traceback when returned from a pybind11 module, including warning exceptions raised during imports.