Update Nexmark Query 10 to use AvroIO
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Nexmark's Query 10 tested writing to sharded files on Google Storage, it used some google specific APIs and it 'manually' ensured sharding. I suppose we can update this to support the other filesystems and withSharding, or we should if not redefine the use case.
Imported from Jira [BEAM-2856](https://issues.apache.org/jira/browse/BEAM-2856). Original Jira may contain additional context.
Reported by: iemejia.
Contributor guide
Research direction
Start with Nexmark Query 10 and inspect how it currently writes sharded files using Google-specific APIs. Read the AvroIO and withSharding usage relevant to the query, then determine whether the goal is filesystem-independent output or a revised use case; done means the agreed behavior is implemented and Query 10 still tests successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100