aboutcode-org / aboutcode-org/scancode-toolkit

Detecting npm dependencies licenses, fetching remote data from the registry

Open
#2,591 13 comments 0 reactions 0 assignees View on GitHub
dependencies live-online-scan new feature
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

### Description

Given the below dependencies and devdependencies, I'm expecting scancode to tell me the license types of these dependencies, but it's not reporting them.

```
"dependencies": {
"@azure/cosmos": "^3.11.0",
"dotenv": "^9.0.0",
"fastify": "^3.15.1",
"fastify-swagger": "^4.7.0",
"nodemon": "^2.0.7",
"uuid": "^8.3.2"
},
"devDependencies": {
"@types/jest": "^26.0.23",
"@types/node": "^15.0.2",
"@typescript-eslint/eslint-plugin": "^4.22.1",
"@typescript-eslint/parser": "^4.22.1",
"eslint": "^7.25.0",
"eslint-config-airbnb-base": "^14.2.1",
"eslint-config-prettier": "^6.15.0",
"eslint-import-resolver-typescript": "^2.4.0",
"eslint-plugin-import": "^2.22.1",
"eslint-plugin-node": "^11.1.0",
"eslint-plugin-prettier": "^3.4.0",
"husky": "^6.0.0",
"jest": "^26.6.3",
"prettier": "^2.2.1",
"supertest": "^6.1.3",
"ts-jest": "^26.5.6",
"ts-node": "^9.1.1",
"typescript": "^4.2.4"
},
```

In the example above, these are the actual licenses for the dependencies:

"@azure/cosmos": MIT,
"dotenv": BSD-2-Clause,
"fastify": MIT,
"fastify-swagger": MIT,
"nodemon": MIT,
"uuid": MIT

the package.json scan (or the scan of the project) when shown in scancode-workbench doesn't report these licenses.

In the json results itself, packages.dependencies.fastify-swagger as an example, has the below output:

```
{
"type": "npm",
"namespace": null,
"name": "fastify-swagger",
"version": "4.7.0",
"qualifiers": {},
"subpath": null,
"primary_language": "JavaScript",
"description": null,
"release_date": null,
"parties": [],
"keywords": [],
"homepage_url": null,
"download_url": "https://registry.npmjs.org/fastify-swagger/-/fastify-swagger-4.7.0.tgz",
"size": null,
"sha1": null,
"md5": null,
"sha256": null,
"sha512": null,
"bug_tracking_url": null,
"code_view_url": null,
"vcs_url": null,
"copyright": null,
"license_expression": null,
"declared_license": null,
"notice_text": null,
"root_path": "project",
"dependencies": [
{
"purl": "pkg:npm/fastify-plugin@%5E3.0.0",
"requirement": "^3.0.0",
"scope": "requires",
"is_runtime": true,
"is_optional": false,
"is_resolved": true
}
],
"contains_source_code": null,
"source_packages": [],
"extra_data": {},
"purl": "pkg:npm/fastify-swagger@4.7.0",
"repository_homepage_url": "https://www.npmjs.com/package/fastify-swagger",
"repository_download_url": "https://registry.npmjs.org/fastify-swagger/-/fastify-swagger-4.7.0.tgz",
"api_data_url": "https://registry.npmjs.org/fastify-swagger/4.7.0"
},
```

But, the npm page for fastify-swagger reports the license type as MIT:
https://www.npmjs.com/package/fastify-swagger
Additionally, the source also does:
https://github.com/fastify/fastify-swagger/blob/master/LICENSE

### How To Reproduce

> Tell us how to reproduce the issue.

```
git clone https://github.com/nexB/scancode-toolkit.git
cd scancode-toolkit
docker build -t scancode-toolkit .
docker run -v $PWD/:/project scancode-toolkit -clpeui --json-pp /project/result.json /project
```

```
Setup plugins...
Collect file inventory...
Scan files for: info, licenses, copyrights, packages, emails, urls with 1 process(es)...
Scanning done.
Summary: info, licenses, copyrights, packages, emails, urls with 1 process(es)
Errors count: 0
Scan Speed: 3.69 files/sec. 38.46 KB/sec.
Initial counts: 92 resource(s): 76 file(s) and 16 directorie(s)
Final counts: 92 resource(s): 76 file(s) and 16 directorie(s) for 792.56 KB
Timings:
scan_start: 2021-07-12T172409.314478
scan_end: 2021-07-12T172431.737338
setup_scan:licenses: 1.65s
setup: 1.65s
inventory: 0.13s
scan: 20.61s
output:json-pp: 0.57s
output: 0.57s
total: 23.00s
Removing temporary files...done.
```

### System configuration

> For bug reports, it really helps us to know:

* What OS are you running on? (MacOS)
* What version of scancode-toolkit was used to generate the scan file? 21.3.31
* What installation method was used to install/run scancode? (pip/source download/other) docker

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.