aboutcode-org / aboutcode-org/scancode-toolkit
Failure to process Debian copyright file
- Lenguaje dominante
- Python
- Estrellas
- 2.6k
- Forks
- 791
- Merge medio
- 1 d 12 h
- PR fusionados (30 d)
- 5
Descripción
This fails to scan https://metadata.ftp-master.debian.org/changelogs//main/l/llvm-toolchain-11/llvm-toolchain-11_11.0.1-2_copyright
This was discovered by @JonoYang
The issue is this section with a license stuffed in a copyright paragraph text:
```
Files: clang/lib/Headers/cuda_wrappers/*
clang/lib/Headers/avx512vlvp2intersectintrin.h
clang/lib/Headers/avx512vp2intersectintrin.h
Copyright: 2016-2020, Google Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
.
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
```
We should not fail there and also support licenses stored in a copyright section.
```
mkdir -p tmp/debian
wget -O tmp/debian/copyright https://metadata.ftp-master.debian.org/changelogs//main/l/llvm-toolchain-11/llvm-toolchain-11_11.0.1-2_copyright
SCANCODE_DEBUG_PACKAGE_API=yes scancode --system-package --yaml - tmp/debian/
Setup plugins...
Collect file inventory...
Scan files for: packages with 1 process(es)...
[--------------------] 0_parse:.is_datafile: /tmp/scancode-toolkit-31.0.0b5/tmp/debian/copyright
[####################] 2
headers:
- tool_name: scancode-toolkit
tool_version: 31.0.0b5
options:
input:
- tmp/debian/
--system-package: yes
--yaml: '-'
notice: |
Generated with ScanCode and provided on an "AS IS" BASIS, WITHOUT WARRANTIES
OR CONDITIONS OF ANY KIND, either express or implied. No content created from
ScanCode should be considered or used as legal advice. Consult an Attorney
for any legal advice.
ScanCode is a free software code scanning tool from nexB Inc. and others.
Visit https://github.com/nexB/scancode-toolkit/ for support and download.
start_timestamp: '2022-05-26T093850.661119'
end_timestamp: '2022-05-26T093852.560430'
output_format_version: 2.0.0
duration: '1.8993210792541504'
message:
errors:
- 'Path: debian/copyright'
warnings: []
extra_data:
system_environment:
operating_system: linux
cpu_architecture: 64
platform: Linux-4.15.0-177-generic-x86_64-with-glibc2.17
platform_version: '#186~16.04.1-Ubuntu SMP Wed Apr 20 09:41:17 UTC 2022'
python_version: "3.8.12 (default, Jan 29 2022, 10:00:28) \n[GCC 5.4.0 20160609]"
spdx_license_list_version: '3.16'
files_count: 1
dependencies: []
packages: []
files:
- path: debian
type: directory
package_data: []
for_packages: []
scan_errors: []
- path: debian/copyright
type: file
package_data: []
for_packages: []
scan_errors:
- |
ERROR: for scanner: packages:
ERROR: Unknown error:
Traceback (most recent call last):
File "/tmp/scancode-toolkit-31.0.0b5/src/scancode/interrupt.py", line 91, in interruptible
return NO_ERROR, func(*(args or ()), **(kwargs or {}))
File "/tmp/scancode-toolkit-31.0.0b5/src/scancode/api.py", line 350, in get_package_data
package_datas = _get_package_data(
File "/tmp/scancode-toolkit-31.0.0b5/src/scancode/api.py", line 308, in _get_package_data
return recognize_package_data(
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/recognize.py", line 66, in recognize_package_data
return list(_parse(location, datafile_handlers=datafile_handlers))
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/recognize.py", line 88, in _parse
for parsed in handler.parse(location):
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 100, in parse
dc = parse_copyright_file(location)
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 198, in parse_copyright_file
dc = StructuredCopyrightProcessor.from_file(
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 453, in from_file
dc.detect_license()
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 693, in detect_license
files_license_detections = self.get_license_detections(
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 721, in get_license_detections
get_license_detection_from_nameless_paragraph(paragraph=paragraph)
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 1681, in get_license_detection_from_nameless_paragraph
start_line, _ = paragraph.get_field_line_numbers('license')
File "/tmp/scancode-toolkit-31.0.0b5/venv/lib/python3.8/site-packages/debian_inspector/copyright.py", line 182, in get_field_line_numbers
return self.line_numbers_by_field[field_name]
KeyError: 'license'
Scanning done.
Some files failed to scan properly:
Path: debian/copyright
Summary: packages with 1 process(es)
Errors count: 1
Scan Speed: 16.31 files/sec.
Initial counts: 2 resource(s): 1 file(s) and 1 directorie(s)
Final counts: 2 resource(s): 1 file(s) and 1 directorie(s)
Timings:
scan_start: 2022-05-26T093850.661119
scan_end: 2022-05-26T093852.560430
setup_scan:licenses: 1.79s
setup: 1.79s
total: 1.91s
Removing temporary files...done.
```
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.