BUG: np.vectorize (interally _parse_gufunc_signature) can't parse a NEP 20 signature

Hello,

The Debian numpy maintainer filed a a bug against my packaging of Dask (Debian Bug 918204) because some of dasks's tests failed. One of the failures seems to be related to the changes for https://www.numpy.org/neps/nep-0020-gufunc-signature-enhancement.html

(relevant traceback from the dask tests: here (search for "test_matmul ")

    @pytest.mark.skipif(sys.version_info < (3, 5),
                        reason="Matrix multiplication operator only after Py3.5")
    def test_matmul():
        x = np.random.random((5, 5))
        y = np.random.random((5, 2))
        a = from_array(x, chunks=(1, 5))
        b = from_array(y, chunks=(5, 1))
        assert_eq(operator.matmul(a, b), a.dot(b))
        assert_eq(operator.matmul(a, b), operator.matmul(x, y))
>       assert_eq(operator.matmul(a, y), operator.matmul(x, b))

/usr/lib/python3/dist-packages/dask/array/tests/test_array_core.py:849: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/dask/array/core.py:1054: in __array_ufunc__
    **kwargs)
/usr/lib/python3/dist-packages/dask/array/gufunc.py:264: in apply_gufunc
    input_coredimss, output_coredimss = _parse_gufunc_signature(signature)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

signature = '(n?,k),(k,m?)->(n?,m?)'

The signature '(n?,k),(k,m?)->(n?,m?)' looks like one of the examples from NEP 20, but I tried looking at Numpy's _parse_gufunc_signature to see how to update Dask's version and discovered that numpy's doesn't seem to parse '(n?,k),(k,m?)->(n?,m?)' either.

I looked for a unit test and couldn't find the above signature test, so I added it to commit 30eae3dd24402ca07cd56cae6db646ad61e5d7f1 (not in NumPy: mattip)

--- a/numpy/lib/tests/test_function_base.py
+++ b/numpy/lib/tests/test_function_base.py
@@ -1362,6 +1362,8 @@ class TestVectorize(object):
                      ([('x',)], [('y',), ()]))
         assert_equal(nfb._parse_gufunc_signature('(),(a,b,c),(d)->(d,e)'),
                      ([(), ('a', 'b', 'c'), ('d',)], [('d', 'e')]))
+        assert_equal(nfb._parse_gufunc_signature('(m?,n),(n,p?)->(m?,p?)'),
+                     ([('m?', 'n'),('n','p?')],[('m?', 'p?')]))
         with assert_raises(ValueError):
             nfb._parse_gufunc_signature('(x)(y)->()')
         with assert_raises(ValueError):

and python3 runtests.py fails.

___________________ TestVectorize.test_parse_gufunc_signature ___________________

self = <numpy.lib.tests.test_function_base.TestVectorize object at 0x7f8de58c1f60>

    def test_parse_gufunc_signature(self):
        assert_equal(nfb._parse_gufunc_signature('(x)->()'), ([('x',)], [()]))
        assert_equal(nfb._parse_gufunc_signature('(x,y)->()'),
                     ([('x', 'y')], [()]))
        assert_equal(nfb._parse_gufunc_signature('(x),(y)->()'),
                     ([('x',), ('y',)], [()]))
        assert_equal(nfb._parse_gufunc_signature('(x)->(y)'),
                     ([('x',)], [('y',)]))
        assert_equal(nfb._parse_gufunc_signature('(x)->(y),()'),
                     ([('x',)], [('y',), ()]))
        assert_equal(nfb._parse_gufunc_signature('(),(a,b,c),(d)->(d,e)'),
                     ([(), ('a', 'b', 'c'), ('d',)], [('d', 'e')]))
>       assert_equal(nfb._parse_gufunc_signature('(m?,n),(n,p?)->(m?,p?)'),
                     ([('m?', 'n'),('n','p?')],[('m?', 'p?')]))

self       = <numpy.lib.tests.test_function_base.TestVectorize object at 0x7f8de58c1f60>

numpy/lib/tests/test_function_base.py:1365: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

signature = '(m?,n),(n,p?)->(m?,p?)'

    def _parse_gufunc_signature(signature):
        """
        Parse string signatures for a generalized universal function.
    
        Arguments
        ---------
        signature : string
            Generalized universal function signature, e.g., ``(m,n),(n,p)->(m,p)``
            for ``np.matmul``.
    
        Returns
        -------
        Tuple of input and output core dimensions parsed from the signature, each
        of the form List[Tuple[str, ...]].
        """
        if not re.match(_SIGNATURE, signature):
            raise ValueError(
>               'not a valid gufunc signature: {}'.format(signature))
E           ValueError: not a valid gufunc signature: (m?,n),(n,p?)->(m?,p?)

signature  = '(m?,n),(n,p?)->(m?,p?)'

numpy/lib/function_base.py:1794: ValueError

Should _parse_gufunc_signature handle the signature with question marks? If yes, should there be a test for that case?

Edit: noted that commit is not in NumPy (mattip)

Start with _parse_gufunc_signature in numpy/lib/function_base.py and the existing TestVectorize.test_parse_gufunc_signature in numpy/lib/tests/test_function_base.py. Review the NEP 20 signature example and run the focused test or python3 runtests.py. Done means the question-mark signature behavior is resolved and covered by an appropriate test.