07th-mod / 07th-mod/python-patcher

Installer Download / URL checker problems / Strange server behavior with partial requests

未關閉
#165 4 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
JavaScript
星號
214
分支
12
PR 合併指標
30 天內沒有已合併 PR

描述

When investigating why the URL checker didn't work, @tellowkrinkle discovered that the server won't always send the correct response if you only request part of a file.

It looks like every second response will give a different response, alternating between HTTP 200 (the whole file), and HTTP 206 (partial response):

```cmd
C:\Users\drojf>curl --head "https://07th-mod.com/rikachama/graphics/Watanagashi-Graphics.7z" -H "Range: bytes=0-1023"
HTTP/1.1 200 OK
Date: Sun, 19 Sep 2021 01:41:37 GMT
Content-Type: application/x-7z-compressed
Content-Length: 1006748838
Connection: keep-alive
last-modified: Mon, 12 Oct 2020 23:09:17 GMT
etag: "5f84e21d-3c01c4a6"
Cache-Control: max-age=3600
CF-Cache-Status: MISS
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=mzUHRUmntFmMAdOY%2FT9om2cIKrUpjUPKU%2FNdqCofBmZ0ex1wMttuDkfcONIyBCdTUQJ%2FoipAQlY1aaNiDPwqxjJSl7%2BP1213rdqqMwO5PtPJD%2B1KnQeSkVecs2jLbeY%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 690f1cd88c5c16c5-SYD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400

C:\Users\drojf>curl --head "https://07th-mod.com/rikachama/graphics/Watanagashi-Graphics.7z" -H "Range: bytes=0-1023"
HTTP/1.1 206 Partial Content
Date: Sun, 19 Sep 2021 01:41:39 GMT
Content-Type: application/x-7z-compressed
Content-Length: 1024
Connection: keep-alive
last-modified: Mon, 12 Oct 2020 23:09:17 GMT
etag: "5f84e21d-3c01c4a6"
content-range: bytes 0-1023/1006748838
Cache-Control: max-age=3600
CF-Cache-Status: MISS
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=PCimvFfIu0rza0%2FAeyxog9j3bGk2q6r7Tlm7u44XnZcBlKmcL6HOS6lwCstnij1E7isipUtlHQYiQi5rSmoph%2BoTVm7UCu5Bo280PMFUWzlVZr120rQEsCnu1lpm0x8%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 690f1ce28a1a16b9-SYD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
```

I tested doing range requests on another website (downloading the Ubuntu ISO), and that website doesn't have this problem - it always returns 206.

Now this may seem like it would break the installer's "resume downloads" capability, but aria2c always first does a normal request, followed by a range request. When you do the requests this way, the server will always give a correct range response on the second request.

```cmd
09/19 13:16:25 [INFO] CUID#7 - Requesting:
GET /rikachama/graphics/Watanagashi-Graphics.7z HTTP/1.1
User-Agent: aria2/1.35.0
Accept: */*,application/metalink4+xml,application/metalink+xml
Host: 07th-mod.com
Want-Digest: SHA-512;q=1, SHA-256;q=1, SHA;q=0.1

[#9392da 0B/0B CN:1 DL:0B]
09/19 13:16:26 [INFO] CUID#7 - Response received:
HTTP/1.1 200 OK
Date: Sun, 19 Sep 2021 03:16:24 GMT
Content-Type: application/x-7z-compressed
Content-Length: 1006748838
Connection: keep-alive
last-modified: Mon, 12 Oct 2020 23:09:17 GMT
etag: "5f84e21d-3c01c4a6"
Cache-Control: max-age=3600
CF-Cache-Status: MISS
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=e13MFXOE%2FDL0pPprAmHZ%2BvK2cexqyTqMxP5j7X29KHXDlP6CnCWCXGLM25HYUQ57tGbLnhhyYQ64F0pq6%2B0ScohBZJLoufugicKtDdmvSM%2BTw1yOGf6vUXiqQsWzYcc%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 690fa7ad9e1b6a1a-SYD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400

09/19 13:16:26 [INFO] CUID#8 - Requesting:
GET /rikachama/graphics/Watanagashi-Graphics.7z HTTP/1.1
User-Agent: aria2/1.35.0
Accept: */*
Host: 07th-mod.com
Range: bytes=241631232-1006748837
Want-Digest: SHA-512;q=1, SHA-256;q=1, SHA;q=0.1

[#9392da 230MiB/0.9GiB(24%) CN:1 DL:0B]
09/19 13:16:27 [INFO] CUID#8 - Response received:
HTTP/1.1 206 Partial Content
Date: Sun, 19 Sep 2021 03:16:25 GMT
Content-Type: application/x-7z-compressed
Content-Length: 765117606
Connection: keep-alive
last-modified: Mon, 12 Oct 2020 23:09:17 GMT
etag: "5f84e21d-3c01c4a6"
content-range: bytes 241631232-1006748837/1006748838
Cache-Control: max-age=3600
CF-Cache-Status: MISS
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=pse79IiOe%2BsMrwDhjQICTw4v0c8bmSBkby2xudgF7bC9AgpUaT7orsMmTi8n6%2BxfVuuvQ0N0O%2F%2BavyfBTCvIIVDhhHjs8lKr6dtPFHy17iw3fAendzRlQsBs5auBnNQ%3D"}],"group":"cf-nel","max_age":604800}
NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Server: cloudflare
CF-RAY: 690fa7b64fa516e1-SYD
alt-svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400, h3-28=":443"; ma=86400, h3-27=":443"; ma=86400
```

This means that our installer's resume functionality is unaffected by this behavior. Therefore, I don't think we need to investigate/fix this issue.

Tellowkrinkle has put a workaround for the URL checker (along with quite a few other changes) on the `fix_validator_timeout` branch, which should be ready to merge already. I'll close this issue once that is merged.

貢獻指南

這個儲存庫沒有索引到貢獻指南

研究方向

該 issue 描述了一個伺服器行為,其中部分 HTTP 範圍請求在 200 和 206 回應之間交替。解決方法已在 `fix_validator_timeout` 分支上。要理解該問題,請檢查執行範圍請求的 URL 檢查器程式碼,並查看該分支如何修改它。修復可能涉及處理兩種回應代碼,或確保在範圍請求之前先進行完整請求。

由索引模型根據 Issue 內容生成。

評估

領域
backend, cli
Issue 類型
缺陷
難度
4/5
預估耗時
3-5 天
活躍度
停滯
描述清晰度
基本清楚
新手友好度
35/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。