Analyze FieldAccessDescriptors and drop fields that are never accessed
Open
gcp
io
java
P3
sub-task
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
We can walk backwards through the graph, analyzing which fields are accessed. When we find paths where many fields are never accessed, we can insert a projection transform to drop those fields preemptively. This can save a lot of resources in the case where many fields in the input are never accessed.
To do this, the FieldAccessDescriptor information must be added to the portability protos.
Imported from Jira [BEAM-4457](https://issues.apache.org/jira/browse/BEAM-4457). Original Jira may contain additional context.
Reported by: reuvenlax.
Subtask of issue #18898
Contributor guide
Assessment
This issue has not been assessed yet.