lingochamp / lingochamp/FileDownloader

360安全云盘的文件无法下载:FileDownloadGiveUpRetryException

Open
#1,131 2 comments 0 reactions 1 assignee View on GitHub

@rantianhua is already working on this.

Since Oct 19, 2018.

question
Dominant language
Java
Stars
11.1k
Forks
2.2k
PR merge metrics
No merged PRs in 30d

Description

#### Before Issue

1. Please search on the [Issues](https://github.com/lingochamp/FileDownloader/issues)
2. Please search on the [wiki](https://github.com/lingochamp/FileDownloader/wiki)
3. Please set `FileDownloadLog.NEED_LOG=true` and review the Logcat output from main process and `:filedownloader` process ( pay attention to Warn and Error level logcat)

#### Issue

1. What problem do you get?
下载360安全云盘的文件:
[下载资源目录](https://cbabae.link.yunpan.360.cn/lk/surl_ydwseCJfKEA#/-0)
下载【薛之谦-我终于成了别人的女人】下载失败
2. Which version of FileDownloader are you using when you produce such problem?
最新master分支的代码 v1.7.5
3. How to reproduce such problem?
使用多线程下载360安全云盘【薛之谦-我终于成了别人的女人】下载失败
下载链接可以使用QQ浏览器或UC浏览器复制已下载的此文件的URL,有可能过段时间会失效,重新下载复制URL即可。
4. Do you set `FileDownloadLog.NEED_LOG=true`?
YES
5. Could you please reproduce this problem and provide all main process and `:filedownloader` process logcat
##### main process
```
10-12 19:22:11.663 6250-6258/com.liulishuo.filedownloader.demo I/zygote: Do partial code cache collection, code=10KB, data=22KB
10-12 19:22:11.670 6250-6258/com.liulishuo.filedownloader.demo I/zygote: After code cache collection, code=10KB, data=22KB
Increasing code cache capacity to 128KB
10-12 19:22:11.674 6250-6258/com.liulishuo.filedownloader.demo I/zygote: Do partial code cache collection, code=10KB, data=40KB
10-12 19:22:11.675 6250-6258/com.liulishuo.filedownloader.demo I/zygote: After code cache collection, code=10KB, data=40KB
Increasing code cache capacity to 256KB
10-12 19:22:11.678 6250-6258/com.liulishuo.filedownloader.demo I/zygote: Compiler allocated 4MB to compile void android.widget.TextView.(android.content.Context, android.util.AttributeSet, int, int)
10-12 19:22:12.102 6250-6250/com.liulishuo.filedownloader.demo I/Choreographer: Skipped 62 frames! The application may be doing too much work on its main thread.
10-12 19:22:12.582 6250-6258/com.liulishuo.filedownloader.demo I/zygote: Do full code cache collection, code=125KB, data=90KB
After code cache collection, code=125KB, data=65KB
10-12 19:22:13.569 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 8. Thread differ : +1. unknown: +1 [(+)queued-work-looper]
10-12 19:22:13.575 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 8. Thread differ : +1. unknown: +1 [(+)queued-work-looper]
10-12 19:22:13.577 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSize: Thread count = 8. unknown: 8
10-12 19:22:13.944 6250-6258/com.liulishuo.filedownloader.demo I/zygote: Do partial code cache collection, code=126KB, data=72KB
10-12 19:22:13.945 6250-6258/com.liulishuo.filedownloader.demo I/zygote: After code cache collection, code=126KB, data=72KB
Increasing code cache capacity to 512KB
10-12 19:22:15.690 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 12. Thread differ : +4. FlowSingle: +1 [(+)FileDownloader-Flow-01] | Event: +1 [(+)FileDownloader-EventPool1] | LauncherTask: +2 [(+)FileDownloader-LauncherTask1, (+)FileDownloader-LauncherTask2]
10-12 19:22:15.695 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 12. Thread differ : +4. FlowSingle: +1 | Event: +1 | LauncherTask: +2
10-12 19:22:15.698 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSize: Thread count = 12. FlowSingle: 1 | Event: 1 | LauncherTask: 2 | unknown: 8
10-12 19:22:24.469 6250-6264/com.liulishuo.filedownloader.demo I/zygote: Background concurrent copying GC freed 242(299KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 2MB/4MB, paused 783us total 215.971ms
10-12 19:22:24.490 6250-6250/com.liulishuo.filedownloader.demo W/System.err: com.liulishuo.filedownloader.exception.FileDownloadGiveUpRetryException: require range[9534279-19068557) with contentLength(9534279), but the backend response contentLength is 5242880 on downloadId[1947487790]-connectionIndex[1], please ask your backend dev to fix such problem.
10-12 19:22:24.492 6250-6250/com.liulishuo.filedownloader.demo W/System.err: at com.liulishuo.filedownloader.download.FetchDataTask.run(FetchDataTask.java:105)
10-12 19:22:24.494 6250-6250/com.liulishuo.filedownloader.demo W/System.err: at com.liulishuo.filedownloader.download.DownloadRunnable.run(DownloadRunnable.java:121)
10-12 19:22:24.495 6250-6250/com.liulishuo.filedownloader.demo W/System.err: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:457)
10-12 19:22:24.497 6250-6250/com.liulishuo.filedownloader.demo W/System.err: at java.util.concurrent.FutureTask.run(FutureTask.java:266)
10-12 19:22:24.498 6250-6250/com.liulishuo.filedownloader.demo W/System.err: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
10-12 19:22:24.500 6250-6250/com.liulishuo.filedownloader.demo W/System.err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
10-12 19:22:24.501 6250-6250/com.liulishuo.filedownloader.demo W/System.err: at java.lang.Thread.run(Thread.java:764)
10-12 19:22:24.530 6250-6250/com.liulishuo.filedownloader.demo W/StaticLayout: maxLineHeight should not be -1. maxLines:2 lineCount:2
10-12 19:22:24.548 6250-6250/com.liulishuo.filedownloader.demo I/chatty: uid=10162(com.liulishuo.filedownloader.demo) identical 3 lines
10-12 19:22:24.550 6250-6250/com.liulishuo.filedownloader.demo W/StaticLayout: maxLineHeight should not be -1. maxLines:2 lineCount:2
10-12 19:22:30.160 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 9. Thread differ : -3. Event: -1 [(-)FileDownloader-EventPool1] | LauncherTask: -2 [(-)FileDownloader-LauncherTask1, (-)FileDownloader-LauncherTask2]
10-12 19:22:30.165 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 9. Thread differ : -3. Event: -1 | LauncherTask: -2
10-12 19:22:30.167 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSize: Thread count = 9. FlowSingle: 1 | unknown: 8
10-12 19:22:40.450 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 8. Thread differ : -1. FlowSingle: -1 [(-)FileDownloader-Flow-01]
10-12 19:22:40.459 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 8. Thread differ : -1. FlowSingle: -1
10-12 19:22:40.465 6250-6287/com.liulishuo.filedownloader.demo D/FileDownloadApplication: drawUpEachThreadSize: Thread count = 8. unknown: 8
10-12 19:29:52.724 6250-6264/com.liulishuo.filedownloader.demo I/zygote: Background concurrent copying GC freed 64323(2MB) AllocSpace objects, 0(0B) LOS objects, 49% free, 2MB/4MB, paused 142us total 905.575ms
10-12 19:37:30.355 6250-6264/com.liulishuo.filedownloader.demo I/zygote: Background concurrent copying GC freed 10781(600KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 3MB/7MB, paused 250us total 369.721ms
10-12 19:50:39.280 6250-6264/com.liulishuo.filedownloader.demo I/zygote: Background concurrent copying GC freed 0(50KB) AllocSpace objects, 0(0B) LOS objects, 49% free, 6MB/13MB, paused 3.122ms total 757.206ms

```
##### filedownloader process
```
10-12 19:22:14.571 6651-6651/? I/zygote: Late-enabling -Xcheck:jni
10-12 19:22:15.019 6651-6679/com.liulishuo.filedownloader.demo:filedownloader D/NetworkSecurityConfig: No Network Security Config specified, using platform default
10-12 19:22:16.794 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 16. Thread differ : +16. OkHttp: +1 [(+)OkHttp ConnectionPool] | okio: +1 [(+)Okio Watchdog] | Network: +1 [(+)FileDownloader-Network1] | FlowSingle: +1 [(+)FileDownloader-Flow-01] | Connection: +3 [(+)FileDownloader-ConnectionBlock3, (+)FileDownloader-ConnectionBlock2, (+)FileDownloader-ConnectionBlock1] | RemitHandoverToDB: +1 [(+)FileDownloader-RemitHandoverToDB] | unknown: +8 [(+)source-status-callback, (+)FinalizerDaemon, (+)HeapTaskDaemon, (+)ThreadDebugger, (+)main, (+)queued-work-looper, (+)ReferenceQueueDaemon, (+)FinalizerWatchdogDaemon]
10-12 19:22:16.796 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 16. Thread differ : +16. OkHttp: +1 | okio: +1 | Network: +1 | FlowSingle: +1 | Connection: +3 | RemitHandoverToDB: +1 | unknown: +8 [(+)source-status-callback, (+)FinalizerDaemon, (+)HeapTaskDaemon, (+)ThreadDebugger, (+)main, (+)queued-work-looper, (+)ReferenceQueueDaemon, (+)FinalizerWatchdogDaemon]
10-12 19:22:16.798 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSize: Thread count = 16. OkHttp: 1 | okio: 1 | Network: 1 | FlowSingle: 1 | Connection: 3 | RemitHandoverToDB: 1 | unknown: 8
10-12 19:22:24.899 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 15. Thread differ : -1. unknown: -1 [(-)source-status-callback]
10-12 19:22:24.902 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 15. Thread differ : -1. unknown: -1 [(-)source-status-callback]
drawUpEachThreadSize: Thread count = 15. OkHttp: 1 | okio: 1 | Network: 1 | FlowSingle: 1 | Connection: 3 | RemitHandoverToDB: 1 | unknown: 7
10-12 19:22:37.031 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 14. Thread differ : -1. Connection: -1 [(-)FileDownloader-ConnectionBlock2]
10-12 19:22:37.033 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 14. Thread differ : -1. Connection: -1
10-12 19:22:37.034 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSize: Thread count = 14. OkHttp: 1 | okio: 1 | Network: 1 | FlowSingle: 1 | Connection: 2 | RemitHandoverToDB: 1 | unknown: 7
10-12 19:22:39.046 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 10. Thread differ : -4. Network: -1 [(-)FileDownloader-Network1] | FlowSingle: -1 [(-)FileDownloader-Flow-01] | Connection: -2 [(-)FileDownloader-ConnectionBlock3, (-)FileDownloader-ConnectionBlock1]
10-12 19:22:39.051 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 10. Thread differ : -4. Network: -1 | FlowSingle: -1 | Connection: -2
10-12 19:22:39.052 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSize: Thread count = 10. OkHttp: 1 | okio: 1 | RemitHandoverToDB: 1 | unknown: 7
10-12 19:28:16.469 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadInfoDiff: Thread count = 9. Thread differ : -1. OkHttp: -1 [(-)OkHttp ConnectionPool]
10-12 19:28:16.470 6651-6670/com.liulishuo.filedownloader.demo:filedownloader D/FileDownloadApplication: drawUpEachThreadSizeDiff: Thread count = 9. Thread differ : -1. OkHttp: -1
drawUpEachThreadSize: Thread count = 9. okio: 1 | RemitHandoverToDB: 1 | unknown: 7
10-12 19:31:50.957 6651-6656/com.liulishuo.filedownloader.demo:filedownloader I/zygote: Do partial code cache collection, code=30KB, data=21KB
10-12 19:31:50.958 6651-6656/com.liulishuo.filedownloader.demo:filedownloader I/zygote: After code cache collection, code=29KB, data=21KB
Increasing code cache capacity to 128KB

```
6. Can you fix it by yourself and request PR, if not, what's problem do you get when you try to fix it

* debug跟踪了一下代码
![1](https://user-images.githubusercontent.com/9621271/46868713-c1c12e00-ce18-11e8-9293-2d4d1654714d.jpg)
![2 1](https://user-images.githubusercontent.com/9621271/46868720-c4bc1e80-ce18-11e8-9a2c-104340c563de.jpg)
![2](https://user-images.githubusercontent.com/9621271/46868724-c71e7880-ce18-11e8-9c36-762b3f8f8998.jpg)
![3](https://user-images.githubusercontent.com/9621271/46868727-cbe32c80-ce18-11e8-9e94-ec9e2613de6f.jpg)
![4](https://user-images.githubusercontent.com/9621271/46868732-cede1d00-ce18-11e8-9a4b-33abbc27ae43.jpg)
![5](https://user-images.githubusercontent.com/9621271/46868733-d1407700-ce18-11e8-9f28-3110058df0eb.jpg)

有以下发现,在分块下载部分,给服务器传的Range无效,服务器还是返回自己的Range。
这一点我在Postman上测试了一下,确实如此。
![6](https://user-images.githubusercontent.com/9621271/46868930-ad316580-ce19-11e8-9894-a96374c33316.jpg)

然后,我使用UC浏览器和QQ浏览器都能正常下载。我用Fiddler抓包试了下,看了下他们的分块。

* UC浏览器分块:
![8](https://user-images.githubusercontent.com/9621271/46869338-ed451800-ce1a-11e8-91bb-6925197072fd.jpg)

* QQ浏览器分块:
![9](https://user-images.githubusercontent.com/9621271/46869340-f0d89f00-ce1a-11e8-8ae8-4d6482bfbc6f.jpg)

发现他们刚开始都发了一个Range: bytes=0- 的请求。而且都有5242880这个块。
![10](https://user-images.githubusercontent.com/9621271/46870124-4dd55480-ce1d-11e8-87d8-807f182f5cbe.jpg)

我猜测他们自己也提前分好块了,然后请求服务器得到的分块和自己的分块数据不一致,优先又服务器的分块,然后对自己剩下的块再分配,然后再请求服务器验证分块是不是一致,如果不一致,就用服务器的分块。不知道我的猜测正不正确。

所以请教一下怎么能解决这个下载的问题~ 谢谢!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.