apache / apache/dubbo

Streaming incompatibility with Dubbo

Open
#3,964 6 comments 0 reactions 0 assignees View on GitHub
type/discussion
Dominant language
Java
Stars
41.6k
Forks
26.4k
Avg merge
15h 13m
Merged PRs (30d)
4

Description

- [x] I have searched the [issues](https://github.com/apache/incubator-dubbo/issues) of this repository and believe that this is not a duplicate.
- [x] I have checked the [FAQ](https://github.com/apache/incubator-dubbo/blob/master/FAQ.md) of this repository and believe that this is not a duplicate.

### Environment

* Dubbo version: 2.7.1
* Operating System version: Centos 7.4
* Java version: 1.8.202

### Steps to reproduce this issue

We are having an interface that scan records from HBase and return a stream of records.
`Stream scan(String startRow, String stopRow);`

We have learnt that the Duddo protocol does not support Stream, but it can handle the stream of objects when it is converted into `java.io.InputStream`.

However, when using `ByteArrayInputStream` as the replacement of the `Stream` for our interface, we got an OutOfMemory exception, since the stream grows too large and exceeds the heap size (ByteArrayInputStream keeps the byte[] in memory).

### Our question
Our question is: What would be the best approach for this issue? Do you have an alternative InputStream implementation for this use case? Or do you know of any already-used solution in Dubbo community?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.