aboutcode-org / aboutcode-org/scancode-toolkit

Failure to process Debian copyright file

未關閉
#2,977 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
bug
主要語言
Python
星號
2.6k
分支
791
平均合併
1 天 12 小時
30 天內合併 PR
5

描述

This fails to scan https://metadata.ftp-master.debian.org/changelogs//main/l/llvm-toolchain-11/llvm-toolchain-11_11.0.1-2_copyright

This was discovered by @JonoYang

The issue is this section with a license stuffed in a copyright paragraph text:
```
Files: clang/lib/Headers/cuda_wrappers/*
clang/lib/Headers/avx512vlvp2intersectintrin.h
clang/lib/Headers/avx512vp2intersectintrin.h
Copyright: 2016-2020, Google Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
.
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
```

We should not fail there and also support licenses stored in a copyright section.

```
mkdir -p tmp/debian
wget -O tmp/debian/copyright https://metadata.ftp-master.debian.org/changelogs//main/l/llvm-toolchain-11/llvm-toolchain-11_11.0.1-2_copyright
SCANCODE_DEBUG_PACKAGE_API=yes scancode --system-package --yaml - tmp/debian/
Setup plugins...
Collect file inventory...
Scan files for: packages with 1 process(es)...
[--------------------] 0_parse:.is_datafile: /tmp/scancode-toolkit-31.0.0b5/tmp/debian/copyright
[####################] 2
headers:
- tool_name: scancode-toolkit
tool_version: 31.0.0b5
options:
input:
- tmp/debian/
--system-package: yes
--yaml: '-'
notice: |
Generated with ScanCode and provided on an "AS IS" BASIS, WITHOUT WARRANTIES
OR CONDITIONS OF ANY KIND, either express or implied. No content created from
ScanCode should be considered or used as legal advice. Consult an Attorney
for any legal advice.
ScanCode is a free software code scanning tool from nexB Inc. and others.
Visit https://github.com/nexB/scancode-toolkit/ for support and download.
start_timestamp: '2022-05-26T093850.661119'
end_timestamp: '2022-05-26T093852.560430'
output_format_version: 2.0.0
duration: '1.8993210792541504'
message:
errors:
- 'Path: debian/copyright'
warnings: []
extra_data:
system_environment:
operating_system: linux
cpu_architecture: 64
platform: Linux-4.15.0-177-generic-x86_64-with-glibc2.17
platform_version: '#186~16.04.1-Ubuntu SMP Wed Apr 20 09:41:17 UTC 2022'
python_version: "3.8.12 (default, Jan 29 2022, 10:00:28) \n[GCC 5.4.0 20160609]"
spdx_license_list_version: '3.16'
files_count: 1
dependencies: []
packages: []
files:
- path: debian
type: directory
package_data: []
for_packages: []
scan_errors: []
- path: debian/copyright
type: file
package_data: []
for_packages: []
scan_errors:
- |
ERROR: for scanner: packages:
ERROR: Unknown error:
Traceback (most recent call last):
File "/tmp/scancode-toolkit-31.0.0b5/src/scancode/interrupt.py", line 91, in interruptible
return NO_ERROR, func(*(args or ()), **(kwargs or {}))
File "/tmp/scancode-toolkit-31.0.0b5/src/scancode/api.py", line 350, in get_package_data
package_datas = _get_package_data(
File "/tmp/scancode-toolkit-31.0.0b5/src/scancode/api.py", line 308, in _get_package_data
return recognize_package_data(
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/recognize.py", line 66, in recognize_package_data
return list(_parse(location, datafile_handlers=datafile_handlers))
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/recognize.py", line 88, in _parse
for parsed in handler.parse(location):
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 100, in parse
dc = parse_copyright_file(location)
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 198, in parse_copyright_file
dc = StructuredCopyrightProcessor.from_file(
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 453, in from_file
dc.detect_license()
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 693, in detect_license
files_license_detections = self.get_license_detections(
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 721, in get_license_detections
get_license_detection_from_nameless_paragraph(paragraph=paragraph)
File "/tmp/scancode-toolkit-31.0.0b5/src/packagedcode/debian_copyright.py", line 1681, in get_license_detection_from_nameless_paragraph
start_line, _ = paragraph.get_field_line_numbers('license')
File "/tmp/scancode-toolkit-31.0.0b5/venv/lib/python3.8/site-packages/debian_inspector/copyright.py", line 182, in get_field_line_numbers
return self.line_numbers_by_field[field_name]
KeyError: 'license'

Scanning done.
Some files failed to scan properly:
Path: debian/copyright
Summary: packages with 1 process(es)
Errors count: 1
Scan Speed: 16.31 files/sec.
Initial counts: 2 resource(s): 1 file(s) and 1 directorie(s)
Final counts: 2 resource(s): 1 file(s) and 1 directorie(s)
Timings:
scan_start: 2022-05-26T093850.661119
scan_end: 2022-05-26T093852.560430
setup_scan:licenses: 1.79s
setup: 1.79s
total: 1.91s
Removing temporary files...done.
```

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。