google / google/j2objc

HttpURLConnection synchronously downloads complete response in memory

Open
#571 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6k
Forks
999
Avg merge
19h 20m
Merged PRs (30d)
14

Description

I am downloading large files and saving them. What caught me by surprise is that the `HttpURLConnection` fetches the whole response synchronously and in memory. This is problematic:
- You cannot download data from a URL while tracking the progress of the data retrieved thus far, as reading from the input stream blocks until the whole file was fetched
- All of this is handled in memory: when downloading large files memory usage rises significantly.

Is there perhaps another way to achieve this goal (download small chunks on the fly and outputting them) with the current J2Obc, or do we have to resort to native code?

Example code that won't behave as expected:

```
void download(String url, File dest, ProgressTracker tracker) throws IOException {
HttpURLConnection httpConnection = (HttpURLConnection) new URL(url).openConnection();
httpConnection.setRequestMethod("GET");
httpConnection.setDefaultUseCaches(false);
InputStream is = null;
OutputStream os = null;
try {
is = httpConnection.getInputStream();
os = new FileOutputStream(dest);
pipe(is, os, tracker); // reads `is` in blocks, while writing to `os`
} finally {
if (is != null) {
is.close();
}
if (os != null) {
os.close();
}
}
httpConnection.disconnect();
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.