gen_protos doesn't retry after failure
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
[~chadrik]
While running using an outdated version of the protobuf package, I got this error:
```
$ python setup.py sdist
...
Traceback (most recent call last):
File "setup.py", line 339, in
'mypy': generate_protos_first(mypy),
File "/usr/local/google/home/ehudm/.pyenv/versions/3.6.9/envs/beam-py3.6.9/lib/python3.6/site-packages/setuptools/__init__.py",
line 145, in setup
return distutils.core.setup(**attrs)
File "/usr/local/google/home/ehudm/.pyenv/versions/3.6.9/lib/python3.6/distutils/core.py",
line 148, in setup
dist.run_commands()
File "/usr/local/google/home/ehudm/.pyenv/versions/3.6.9/lib/python3.6/distutils/dist.py",
line 955, in run_commands
self.run_command(cmd)
File "/usr/local/google/home/ehudm/.pyenv/versions/3.6.9/lib/python3.6/distutils/dist.py",
line 974, in run_command
cmd_obj.run()
File "/usr/local/google/home/ehudm/.pyenv/versions/3.6.9/envs/beam-py3.6.9/lib/python3.6/site-packages/setuptools/command/sdist.py",
line 44, in run
self.run_command('egg_info')
File "/usr/local/google/home/ehudm/.pyenv/versions/3.6.9/lib/python3.6/distutils/cmd.py",
line 313, in run_command
self.distribution.run_command(command)
File "/usr/local/google/home/ehudm/.pyenv/versions/3.6.9/lib/python3.6/distutils/dist.py",
line 974, in run_command
cmd_obj.run()
File "setup.py", line 253, in run
gen_protos.generate_proto_files()
File "/usr/local/google/home/ehudm/src/beam/sdks/python/gen_protos.py", line 345, in generate_proto_files
generate_urn_files(log, out_dir)
File "/usr/local/google/home/ehudm/src/beam/sdks/python/gen_protos.py",
line 188, in generate_urn_files
beam_runner_api_pb2 = _import('beam_runner_api_pb2')
File "/usr/local/google/home/ehudm/src/beam/sdks/python/gen_protos.py",
line 185, in _import
return __import__('api.%s' % m, fromlist=[None])
File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/portability/api/beam_runner_api_pb2.py",
line 15, in
from . import endpoints_pb2 as endpoints__pb2
File "/usr/local/google/home/ehudm/src/beam/sdks/python/apache_beam/portability/api/endpoints_pb2.py",
line 21, in
create_key=_descriptor._internal_create_key,
AttributeError: module 'google.protobuf.descriptor'
has no attribute '_internal_create_key'
```
Rerunning sdist was successful, but the generated files were missing things like TRUNCATE_SIZED_RESTRICTION.
Issue for protobuf version: https://issues.apache.org/jira/browse/BEAM-10552
Imported from Jira [BEAM-10553](https://issues.apache.org/jira/browse/BEAM-10553). Original Jira may contain additional context.
Reported by: udim.
Contributor guide
Research direction
Start with sdks/python/gen_protos.py, especially generate_proto_files and the import path shown in the traceback, then reproduce the failure through setup.py sdist using the outdated protobuf version. Done means a failed generation does not leave incomplete generated files and a subsequent run produces files containing items such as TRUNCATE_SIZED_RESTRICTION.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100