Corrupted/broken .aria2 file can cause install to fail in an unrecoverable manner

Open

#158 opened on Jul 12, 2021 0 comments 0 reactions 0 assignees View on GitHub

bug

Dominant language: JavaScript
Stars: 214
Forks: 12
PR merge metrics: No merged PRs in 30d

Description

(NOTE: unconfirmed - I haven't tried reproducing this myself, but I'm pretty sure this is correct)

We currently have some resume functionality in the installer, provided by aria2. Aria2 uses a `.aria2` file to keep track of the download progress.

If the `.aria2` file becomes corrupted or unreadable, they won't be able to complete the install, even if they retry the whole install process, as aria2 won't automatically clear the `.aria2` file if it is corrupted.

You'll get an error like `errorCode=1 Failed to read from the segment file Umineko Question (Ch. 1-4) Downloads/Umineko-Graphics-1080p-v3.7z.aria2`

If this happens (with the current installer), the only way for the user to currently recover from this situation is to:
- Delete the corrupted `.aria2` file
- Delete the entire downloads folder (like `Umineko Question (Ch. 1-4) Downloads`)
- Start the installer again in another folder to reset all installer state

I'm not sure about how to tackle this (or if to leave it, and just put a note in the wiki). A couple options:

1. It would be nice to have a button which just resets all state related to the installer. We currently have a bit red "force full re-install" button which kind of does this, but this is only available if you already have performed an install - not on the first time you install the mod. So having a button which is always there which is like "clear all installer state and start install" might be a solution.
2. Aria2 may also have a flag which automatically deletes the `.aria2` file if it is corrupted, but I don't know if it exists.
3. We could try to detect this exception from aria2 in the installer itself, then delete the offending file.

Here is an example log file from a user who had this issue:

```log

<<< Status: 4% Downloading: 5.57 GB (total) DL Folder: [Umineko Question (Ch. 1-4) Downloads] URL: [https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4] (Attempt: 1/3) >>>
----- BEGIN EXECUTING COMMAND: [./aria2c --file-allocation=none --continue=true --retry-wait=5 -m 0 -x 8 -s 8 -j 1 --auto-file-renaming=false --allow-overwrite=true --follow-metalink=mem --check-integrity=true --disable-ipv6=true -d Umineko Question (Ch. 1-4) Downloads https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4] -----

07/11 15:27:46 [[1;32mNOTICE[0m]
Downloading 1 item(s)

07/11 15:27:46 [[1;32mNOTICE[0m]
Download complete: [MEMORY]
Umineko-Graphics-1080p-v3.7z.meta4

07/11 15:27:46 [[1;31mERROR[0m]
Exception caught
Exception: [RequestGroup.cc:760]
errorCode=1 Download aborted.
-> [DefaultBtProgressInfoFile.cc:240]
errorCode=1 Failed to read from the segment file Umineko Question (Ch. 1-4) Downloads/Umineko-Graphics-1080p-v3.7z.aria2

07/11 15:27:46 [[1;32mNOTICE[0m]
Download GID#c22be35ae30d97a9 not complete: Umineko Question (Ch. 1-4) Downloads/Umineko-Graphics-1080p-v3.7z

Status Legend:
(OK):download completed.(ERR):error occurred.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.
--------------- EXECUTION FINISHED ---------------

ERROR - failed to download [https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4]. Trying again in 3 seconds...
<<< Status: 4% Downloading: 5.57 GB (total) DL Folder: [Umineko Question (Ch. 1-4) Downloads] URL: [https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4] (Attempt: 2/3) >>>
----- BEGIN EXECUTING COMMAND: [./aria2c --file-allocation=none --continue=true --retry-wait=5 -m 0 -x 8 -s 8 -j 1 --auto-file-renaming=false --allow-overwrite=true --follow-metalink=mem --check-integrity=true --disable-ipv6=true -d Umineko Question (Ch. 1-4) Downloads https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4] -----

07/11 15:27:49 [[1;32mNOTICE[0m]
Downloading 1 item(s)

07/11 15:27:50 [[1;32mNOTICE[0m]
Download complete: [MEMORY]
Umineko-Graphics-1080p-v3.7z.meta4

07/11 15:27:50 [[1;31mERROR[0m]
Exception caught
Exception: [RequestGroup.cc:760]
errorCode=1 Download aborted.
-> [DefaultBtProgressInfoFile.cc:240]
errorCode=1 Failed to read from the segment file Umineko Question (Ch. 1-4) Downloads/Umineko-Graphics-1080p-v3.7z.aria2

07/11 15:27:50 [[1;32mNOTICE[0m]
Download GID#2b2ac9784f531ec6 not complete: Umineko Question (Ch. 1-4) Downloads/Umineko-Graphics-1080p-v3.7z

Status Legend:
(OK):download completed.(ERR):error occurred.

ERROR - failed to download [https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4]. Trying again in 3 seconds...
<<< Status: 4% Downloading: 5.57 GB (total) DL Folder: [Umineko Question (Ch. 1-4) Downloads] URL: [https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4] (Attempt: 3/3) >>>
----- BEGIN EXECUTING COMMAND: [./aria2c --file-allocation=none --continue=true --retry-wait=5 -m 0 -x 8 -s 8 -j 1 --auto-file-renaming=false --allow-overwrite=true --follow-metalink=mem --check-integrity=true --disable-ipv6=true -d Umineko Question (Ch. 1-4) Downloads https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4] -----

07/11 15:27:53 [[1;32mNOTICE[0m]
Downloading 1 item(s)

07/11 15:27:53 [[1;32mNOTICE[0m]
Download complete: [MEMORY]
Umineko-Graphics-1080p-v3.7z.meta4

07/11 15:27:53 [[1;31mERROR[0m]
Exception caught
Exception: [RequestGroup.cc:760]
errorCode=1 Download aborted.
-> [DefaultBtProgressInfoFile.cc:240]
errorCode=1 Failed to read from the segment file Umineko Question (Ch. 1-4) Downloads/Umineko-Graphics-1080p-v3.7z.aria2

07/11 15:27:53 [[1;32mNOTICE[0m]
Download GID#927551f0cd5ded18 not complete: Umineko Question (Ch. 1-4) Downloads/Umineko-Graphics-1080p-v3.7z

Status Legend:
(OK):download completed.(ERR):error occurred.

ERROR - failed to download [https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4]. Trying again in 3 seconds...
Exception in thread Thread-98:
Traceback (most recent call last):
File "D:\obj\windows-release\37win32_Release\msi_python\zip_win32\threading.py", line 926, in _bootstrap_inner
File "D:\obj\windows-release\37win32_Release\msi_python\zip_win32\threading.py", line 870, in run
File "D:\07th-mod_installer\httpGUI.py", line 667, in errorPrintingInstaller
installerFunction(args)
File "D:\07th-mod_installer\uminekoInstaller.py", line 135, in mainUmineko
downloaderAndExtractor.download()
File "D:\07th-mod_installer\common.py", line 940, in download
raise DownloadAndVerifyError("ERROR - Failed to download [{}] after {} attempts. Check aria2/7z in log for details. Installation Stopped".format(url, attempt + 1))
common.DownloadAndVerifyError: ERROR - Failed to download [https://07th-mod.com/Beato/1080p/Umineko-Graphics-1080p-v3.7z.meta4] after 3 attempts. Check aria2/7z in log for details. Installation Stopped
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Look at the installer's download handling in common.py around line 940 and the aria2 command execution. The error log shows the .aria2 file corruption. Need to find where aria2 is called and decide on a fix: either detect the specific error and delete the .aria2 file, or add a user-facing reset button. Check the aria2 documentation for flags that handle corrupted segment files. Testing involves simulating a corrupted .aria2 file and verifying the installer recovers.

Written by the indexing model from the issue text.

Assessment

Tech stack: python
Domain: cli, tooling
Issue type: Bug
Difficulty: 3/5
Estimated time: 1-2 days
Activity status: Stale
Clarity: Mostly clear
Newbie friendliness: 55/100

Corrupted/broken .aria2 file can cause install to fail in an unrecoverable manner

Description

Contributor guide

Research direction

Assessment

Get new issues in your inbox