GoogleCloudPlatform / GoogleCloudPlatform/market-data-transcoder
Add option to interleave messages to an output destination
Open
enhancement
- Dominant language
- Python
- Stars
- 47
- Forks
- 11
- Avg merge
- 19d 10h
- Merged PRs (30d)
- 2
Description
In lieu of sharding output destinations by message type, support interleaved output of all message types to a single destination file, topic, etc.
This has some implications to schema generation and most obviously would not work for BigQuery output types. It would also require that Pub/Sub topic schemas or Avro outputs be handled differently in this mode, perhaps using the Union option in Avro.
Contributor guide
Assessment
This issue has not been assessed yet.