[Bug]: PickleCoder/_MemoizingPickleCoder.as_deterministic_coder() raises TypeError
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
`PickleCoder.as_deterministic_coder()` and `_MemoizingPickleCoder.as_deterministic_coder()` raise `TypeError` on every call instead of returning a working deterministic coder.
Both methods do:
```python
def as_deterministic_coder(self, step_label, error_message=None):
return FastPrimitivesCoder(self, requires_deterministic=step_label)
```
`FastPrimitivesCoder.__init__(self, fallback_coder=PickleCoder())` has never accepted a `requires_deterministic` argument, so this raises:
```
TypeError: FastPrimitivesCoder.__init__() got an unexpected keyword argument 'requires_deterministic'
```
Repro on current `master`:
```python
from apache_beam.coders.coders import PickleCoder
PickleCoder().as_deterministic_coder('label')
# TypeError: FastPrimitivesCoder.__init__() got an unexpected keyword argument 'requires_deterministic'
```
Root cause: commit 9be70c9861 (2021-03-08) changed `PickleCoder.as_deterministic_coder` from the working `DeterministicFastPrimitivesCoder(self, step_label)` to the broken `FastPrimitivesCoder(self, requires_deterministic=...)` in the same diff that added a `requires_deterministic` kwarg to a different class, `coder_impl.FastPrimitivesCoderImpl` (the Cython impl, not the `coders.FastPrimitivesCoder` wrapper referenced here). `_MemoizingPickleCoder` (added later) copied the same already-broken line.
Any caller that resolves to one of these coders and asks for a deterministic version hits this, for example `GroupByEncryptedKey.expand()` in `apache_beam/transforms/util.py`, which wraps the call in `except ValueError` and does not catch the `TypeError`, so it crashes instead of logging the intended non-deterministic-key warning.
This is a narrower, separate defect from #28558's own headline claim (which is about `pickle_library` being ignored, and was addressed by @tvalentyn as working as intended for `PickleCoder`). @tvalentyn identified this specific `as_deterministic_coder` bug in a comment on that thread in 2023 and invited a PR for it, but #28558 itself is now closed and its title does not match this fix, so filing it here as its own issue.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- Component: Python SDK
Contributor guide
Assessment
This issue has not been assessed yet.