apache / apache/beam

Check that DoFn outputs are valid iterables when using the performance runtime type check feature

Open
#20,404 0 comments 0 reactions 0 assignees View on GitHub
core improvement P3 python
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

Currently, when using the performance runtime type check feature, you can yield invalid iterables from a DoFn and not have an error thrown. This is incorrect behavior.

 

We should throw a TypeCheckError if the iterable type is any of the following: dict, str, unicode, or bytes. 

 

Previously, we tried accomplishing this using an isinstance check in process_outputs of common.py, however this harms performance significantly. We should search for a cheaper way to verify that the output type is valid. One possible solution is to only do the isinstance check _if_ the flag is on.

 

 

Imported from Jira [BEAM-10782](https://issues.apache.org/jira/browse/BEAM-10782). Original Jira may contain additional context.
Reported by: saavan.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.