Sources using Metrics.counter fail with 'Can't pickle ... DelegatingCounter'
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
This fails under Python 2.7 but works fine under Python 3 – looks like an easy fix.
I am reading from a beam.io.utils.CountingSource:
`beam.io.Read(beam_io_utils.CountingSource(NUM_ROWS))`
This fails under python 2.7 with:
Python exception: Can't pickle : attribute lookup apache_beam.metrics.metric.DelegatingCounter failed
Looks like this is because DelegatingCounter is actually an inner class defined inside apache_beam.metrics.metric.Metric. Under Python 3 pickling can resolve it under its qualified name apache_beam.metrics.metric.Metric.DelegatingCounter, but not in Python 2. A fix would be to move it to a top-level class in that module.
This also applies to other sources which use Metrics.counter.
Imported from Jira [BEAM-6389](https://issues.apache.org/jira/browse/BEAM-6389). Original Jira may contain additional context.
Reported by: matthjw.
Contributor guide
Assessment
This issue has not been assessed yet.