Python: ReadFromDatastore Embedded Entities
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
[Issue 8405](https://issues.apache.org/jira/browse/BEAM-8405) discussed the possibility to support embedded entities when using the conversion methods to/from of the client entity type.
This feature was added in [PR 9805]([https://github.com/apache/beam/pull/9805)], which was shipped in 2.17.
However, there seems to be an issue when Datastore embedded entities do not have a key.
Keys in embedded entities are optional.
Because of this, when using the apache_beam.io.gcp.datastore.v1new.datastoreio.ReadFromDatastore transform, the pipeline fails with the following stacktrace:
```
File "apache_beam/runners/common.py", line 780, in apache_beam.runners.common.DoFnRunner.process
File "apache_beam/runners/common.py", line 440, in apache_beam.runners.common.SimpleInvoker.invoke_process
File "apache_beam/runners/common.py", line 895, in apache_beam.runners.common._OutputProcessor.process_outputs
File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/datastoreio.py",
line 264, in process
yield types.Entity.from_client_entity(client_entity)
File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py",
line 225, in from_client_entity
value = Entity.from_client_entity(value)
File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py",
line 219, in from_client_entity
Key.from_client_key(client_entity.key),
File "/Users/quentin/Work/sensome/dev/senback/dataflow/senback-reporting-db-import-flow/venv/lib/python3.7/site-packages/apache_beam/io/gcp/datastore/v1new/types.py",
line 156, in from_client_key
return Key(client_key.flat_path, project=client_key.project,
AttributeError:
'NoneType' object has no attribute 'flat_path' [while running 'Read from Datastore/Read']
```
Imported from Jira [BEAM-9380](https://issues.apache.org/jira/browse/BEAM-9380). Original Jira may contain additional context.
Reported by: Cavalié.
Contributor guide
Assessment
This issue has not been assessed yet.