apache / apache/beam

beam crash when annotating function with typing.Protocol

Open
#20,986 0 comments 0 reactions 0 assignees View on GitHub
bug core P3 python types
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

When annotating a function with a https://docs.python.org/3/library/typing.html#typing.Protocol:

```

class Data(typing.Protocol):
def f(self) -> int:
...

def fn(x: Data) -> Data:

...

pipeline = (
...
| beam.Map(fn)
...
)

```

Beam will crash when `pipeline_type_check=True` because isinstance(x, Data) raise an error.
Indeed, by default, Protocol are not runtime checkable.

```

```

File "/usr/local/google/_blaze_epot/e52e0fa21f849d3c6badad5aaa8158a8/execroot/google3/blaze-out/k8-fastbuild/bin/third_party/py/tensorflow_datasets/scripts/tools/compute_split_info.runfiles/google3/third_party/py/apache_beam/transforms/ptransform.py",
line 935, in type_check_inputs
hint):
File "/usr/local/google/_blaze_epot/e52e0fa21f849d3c6badad5aaa8158a8/execroot/google3/blaze-out/k8-fastbuild/bin/third_party/py/tensorflow_datasets/scripts/tools/compute_split_info.runfiles/google3/third_party/py/apache_beam/typehints/typehints.py",
line 1172, in is_consistent_with
return issubclass(sub, base)
File "/usr/local/google/_blaze_epot/e52e0fa21f849d3c6badad5aaa8158a8/execroot/google3/blaze-out/k8-fastbuild/bin/third_party/py/tensorflow_datasets/scripts/tools/compute_split_info.runfiles/google3/third_party/py/typing_extensions/src_py3/typing_extensions.py",
line 1265, in __subclasscheck__
raise TypeError("Instance and class checks can only be used with"
TypeError:
Instance and class checks can only be used with @runtime protocols

```

```

Even if protocol are not runtime checkable, this is a perfectly valid python annotation, so beam should not raise error. For example, beam could ignore the annotation if isinstance raise error

Imported from Jira [BEAM-12457](https://issues.apache.org/jira/browse/BEAM-12457). Original Jira may contain additional context.
Reported by: conchylicultor.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.