aboutcode-org / aboutcode-org/scancode-toolkit

Missing infromation about GIT and PATH dependencies from Ruby package scan

Open
#2,781 10 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

### Description

I need to parse Ruby dependencies from `Gemfile.lock`. For regular GEM dependencies, everything's ok, but for GIT and PATH dependencies, important information is missing from the generated output, such as a URL address to the git repository or information about the type of the dependency. Also, I'd expect that in the example below, purl for the Github dependency would start with this: `pkg:github`.

When I do the dependency parsing from Python interpreter by importing [gemfile_lock.py](https://github.com/nexB/scancode-toolkit/blob/develop/src/packagedcode/gemfile_lock.py) from this repo, I get all the necessary information that would be needed, so it seems to me that while the parsing module gathering data works properly, not all the information is shown to the user.

### How To Reproduce

Inside directory `testing/`, there is file Gemfile.lock with contents (I took two dependencies from [porta/Gemfile.lock](https://github.com/3scale/porta/blob/master/Gemfile.lock)):

```
GIT
remote: https://github.com/mikz/httpclient.git
revision: fec23fb32fb899b87a8b2c94e2d2069b6b4c633c
branch: ssl-env-cert
specs:
httpclient (2.8.3)

PATH
remote: lib/developer_portal
specs:
developer_portal (1.0.0)
liquid (~> 3.0.6)
railties (>= 3.2)
```

Running `scancode --package --json-pp out.json testing/` from the `testing/`s parent directory produces `out.json` file.

out.json excerpt:

```
{
"type": "gem",
"namespace": null,
"name": "httpclient",
"version": "fec23fb32fb899b87a8b2c94e2d2069b6b4c633c",
"qualifiers": {},
"subpath": null,
"primary_language": "Ruby",
"description": null,
"release_date": null,
"parties": [],
"keywords": [],
"homepage_url": null,
"download_url": null,
"size": null,
"sha1": null,
"md5": null,
"sha256": null,
"sha512": null,
"bug_tracking_url": null,
"code_view_url": null,
"vcs_url": null,
"copyright": null,
"license_expression": null,
"declared_license": null,
"notice_text": null,
"root_path": "testing",
"dependencies": [],
"contains_source_code": null,
"source_packages": [],
"extra_data": {},
"purl": "pkg:gem/httpclient@fec23fb32fb899b87a8b2c94e2d2069b6b4c633c",
"repository_homepage_url": "https://rubygems.org/gems/httpclient/versions/fec23fb32fb899b87a8b2c94e2d2069b6b4c633c",
"repository_download_url": "https://rubygems.org/downloads/httpclient-fec23fb32fb899b87a8b2c94e2d2069b6b4c633c.gem",
"api_data_url": "https://rubygems.org/api/v2/rubygems/httpclient/versions/fec23fb32fb899b87a8b2c94e2d2069b6b4c633c.json"
}
```

Notice that `"vcs_url": null,`, `repository_homepage` address is getting generated and purl format isn't what I'd expect.

### System configuration

> For bug reports, it really helps us to know:

* What OS are you running on? RHEL 8.5
* What version of scancode-toolkit was used to generate the scan file? 30.1.0
* What installation method was used to install/run scancode? pip

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.