react / react/react-native

Fetching large json data results in OutOfMemoryError

未关闭
#32,134 22 条评论 4 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

🌐Networking Issue: Author Provided Repro
主要语言
C++
星标
127k
派生
25.3k
平均合并
1 天 23 小时
30 天内合并 PR
4

描述

Description

Using fetch to get several Megabytes (currently happens for me at around 80Mb) of JSON data causes Android to panic and throw an OutOfMemoryError. This is due to the fact that the whole response is being read as bytes, quickly filling up the heap.

Stacktrace:

09-01 21:32:21.035  5480  5555 E AndroidRuntime: java.lang.OutOfMemoryError: Failed to allocate a 98836368 byte allocation with 25165824 free bytes and 88MB until OOM, target footprint 133385952, growth limit 201326592
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at okio.Buffer.readByteArray(Buffer.kt:1429)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at okio.Buffer.readByteArray(Buffer.kt:1424)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at okio.RealBufferedSource.readByteArray(RealBufferedSource.kt:238)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at okhttp3.ResponseBody.bytes(ResponseBody.kt:124)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at com.facebook.react.modules.blob.BlobModule$4.toResponseData(BlobModule.java:134)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at com.facebook.react.modules.network.NetworkingModule$2.onResponse(NetworkingModule.java:512)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
09-01 21:32:21.035  5480  5555 E AndroidRuntime: 	at java.lang.Thread.run(Thread.java:923)

React Native version:

System:
    OS: Linux 5.13 Solus 4.3
    CPU: (8) x64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
    Memory: 1.39 GB / 15.52 GB
    Shell: 5.1.8 - /bin/bash
  Binaries:
    Node: 14.17.5 - /usr/bin/node
    Yarn: 1.22.10 - /usr/bin/yarn
    npm: 6.14.14 - /usr/bin/npm
    Watchman: Not Found
  SDKs:
    Android SDK: Not Found
  IDEs:
    Android Studio: Not Found
  Languages:
    Java: 1.8.0_302-solus - /usr/lib64/openjdk-8/bin/javac
  npmPackages:
    @react-native-community/cli: Not Found
    react: 17.0.2 => 17.0.2 
    react-native: 0.65.1 => 0.65.1 
  npmGlobalPackages:
    *react-native*: Not Found

Steps To Reproduce

I created a test repository that shows the aforementioned behaviour. It also includes a 100Mb JSON file for testing, that you can serve locally or access via GitHub directly.
Steps are as follows

  1. Tap on the load data button
  2. Wait for the app to crash

Expected Results

Getting an out of memory error shouldn't really happen with this size of data in my opinion. Sure 100Mb sounds a lot at first, but in enterprise-grade apps this is probably a common scenario. In any case I think there should be a possibility to dynamically switch to streaming the response since okhttp offers bytestream and charstream as well. I'm not that well versed in Java but it would probably help reduce all of the 150Mb landing on the heap at once?

Snack, code example, screenshot, or link to a repository:

https://github.com/curtisy1/ReactNativeFetchRepro

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

先从 BlobModule.java 和 NetworkingModule.java 中展示的 Android 响应处理开始,然后使用其中的 100MB data.json 文件运行链接的 ReactNativeFetchRepro 仓库。将当前的 fetch 路径与 issue 中对 OkHttp ResponseBody byteStream 和 charStream 的引用进行比较;当大型 JSON 响应不再触发报告的 OutOfMemoryError 时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
android, java, react-native
领域
mobile, networking
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
42/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。