googleapis / googleapis/google-cloud-rust

Refactor stream writers (arrow, proto, json) to generic

Open
#6,762 0 comments 0 reactions 1 assignee Claimed by @haphungw View on GitHub
api: bigquery
Dominant language
Rust
Stars
955
Forks
144
Avg merge
1d 5h
Merged PRs (30d)
279

Description

`write::arrow` and `write::proto` share very similar stream lifecycle (`DefaultWriter`, `BufferedWriter`, `CommittedWriter`, `PendingWriter`).

We consider to unify them under a generic `BaseWriter` ([#6748](https://github.com/googleapis/google-cloud-rust/pull/6748)), but we need to consider a few things to do so with type aliases (`pub type DefaultWriter = common::DefaultWriter`):
1. Rustdoc method invisibility:
- Rustdoc does not document `impl` methods on public type aliases targeting private (`pub(crate)`) types.
- Making `common::BaseWriter` and `DataFormat` `pub` resolves the doc issue, but leaks internal transport plumbing and generic bounds into the crate's public API surface.
2. Premature abstraction:
- `write::proto` is internal (`pub(crate)`) scaffolding to power `JsonStreamWriter` (#6597).
- Adding `trait DataFormat` before dynamic serialization with `prost-reflect` and other implementation details handling risks problems with trait boundaries.

For now, we should defer this until #6597 lands.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.