aboutcode-org / aboutcode-org/scancode-toolkit

extractcode fails to unpack some gzipped files where the only issue is an extra trailing newline

Offen
#2,740 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Python
Sterne
2.6k
Forks
791
Ø Merge
1 T. 12 Std.
Gemergte PRs (30 T.)
5

Beschreibung

### Description

This case was originally proposed in the discussion of #2723.

There are some common conditions where extractcode fails to recursively unpack an archive, and one of those "fairly common" conditions is where a gzipped file happens to contain a trailing newline after the archive. In truth, this makes the gzipped file an invalid archive, so gunzip fails to unpack it and returns an error. However, other tools (such as zless) will display the uncorrupted portions of the archive, and this allows people to view the content. If this content, viewable by a common viewer, contains license or copyright references, it will be missed by the extractcode/scancode combination.

As a proposal subject to discussion, perhaps extractcode could produce a warning in this case, while extracting the uncorrupted portion of the file by using a streaming-style of unpack (zless or equivalent). Whether this extra code complexity is justified, and how to determine whether the streaming unpack produced something useful needs to be discussed. An alternate implementation might be to truncate the original by one byte and try unpack again. If this succeeds, then silently move on.

### How To Reproduce

The target archive is [here](https://github.com/ruby/rake/archive/refs/tags/v0.9.2.2.zip). Within the Ruby Rake zipfile is what appears to be a compressed documentation file, rake-0.9.2.2/doc/rake.1.gz which could be directly downloaded [here](https://github.com/ruby/rake/blob/v0.9.2.2/doc/rake.1.gz) if you wish.

Manually trying to unzip the offending rake.1.gz file, I get:

gunzip rake.1.gz
gzip: rake.1.gz: unexpected end of file

echo $?
1

### System configuration

My test system is: Debian Linux, python 3.7.3, scancode-toolkit 30.1.0, installed from download of released Linux tar archive

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.