aboutcode-org / aboutcode-org/scancode-toolkit

extractcode fails to unpack some gzipped files where the only issue is an extra trailing newline

Aperta
#2,740 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug
Lingua principale
Python
Stelle
2.6k
Fork
791
Merge medio
1g 12h
PR unite (30g)
5

Descrizione

### Description

This case was originally proposed in the discussion of #2723.

There are some common conditions where extractcode fails to recursively unpack an archive, and one of those "fairly common" conditions is where a gzipped file happens to contain a trailing newline after the archive. In truth, this makes the gzipped file an invalid archive, so gunzip fails to unpack it and returns an error. However, other tools (such as zless) will display the uncorrupted portions of the archive, and this allows people to view the content. If this content, viewable by a common viewer, contains license or copyright references, it will be missed by the extractcode/scancode combination.

As a proposal subject to discussion, perhaps extractcode could produce a warning in this case, while extracting the uncorrupted portion of the file by using a streaming-style of unpack (zless or equivalent). Whether this extra code complexity is justified, and how to determine whether the streaming unpack produced something useful needs to be discussed. An alternate implementation might be to truncate the original by one byte and try unpack again. If this succeeds, then silently move on.

### How To Reproduce

The target archive is [here](https://github.com/ruby/rake/archive/refs/tags/v0.9.2.2.zip). Within the Ruby Rake zipfile is what appears to be a compressed documentation file, rake-0.9.2.2/doc/rake.1.gz which could be directly downloaded [here](https://github.com/ruby/rake/blob/v0.9.2.2/doc/rake.1.gz) if you wish.

Manually trying to unzip the offending rake.1.gz file, I get:

gunzip rake.1.gz
gzip: rake.1.gz: unexpected end of file

echo $?
1

### System configuration

My test system is: Debian Linux, python 3.7.3, scancode-toolkit 30.1.0, installed from download of released Linux tar archive

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.