boostorg / boostorg/beast

Custom Body Type or `response_serializer` for incrementally streaming large amounts of data back to the client

Open
#2,777 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
4.8k
Forks
694
Avg merge
12h 48m
Merged PRs (30d)
1

Description

## Question

I'm using Beast 1.81 with Clang 13.0.0 to present a distributed data management API over HTTP.

I've been reading a lot of the documentation and it's not clear to me what the recommended solution is for streaming large amounts of data back to the client. The pages that look like they are close to what I want are:
- https://www.boost.org/doc/libs/1_81_0/libs/beast/doc/html/beast/using_http/serializer_stream_operations.html
- https://www.boost.org/doc/libs/1_81_0/libs/beast/doc/html/beast/using_http/custom_body_types.html

At the moment, my application presents something similar to POSIX read. That is, it forces the client to make multiple read requests to the HTTP server. This isn't ideal, but is good enough for a first pass _(I'm still getting familiar with Beast)_.

You can view the code in question here:
- https://github.com/irods/irods_client_http_api/blob/38d67aa5ab798952450dc4429c20d82586a375e8/src/endpoints/data_objects/impl.cpp#L417-L420

The application has a background thread pool for long running operations. I want to use that to incrementally stream the data back.

Q. Does a custom body type make sense here? Or would a `response_serializer` be better?
Q. Is `transfer-encoding: chunked` automatically supported by custom body types?
Q. Are there examples which demonstrate how to do this asynchronously?

BTW, thanks for the awesome library.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.