aboutcode-org / aboutcode-org/scancode-toolkit

Improve the consolidated component identifier returned by the consolidate option

Open
#1,686 3 comments 0 reactions 2 assignees Claimed by @JonoYang View on GitHub
bug enhancement
Dominant language
Python
Stars
2.6k
Forks
791
Avg merge
1d 12h
Merged PRs (30d)
5

Description

I recently scanned package cayley-0.7.5.tar.gz (https://github.com/cayleygraph/cayley/archive/v0.7.5.tar.gz) using scancode-toolkit-3.1.0 and I specified the `--consolidate` option. The scan results were initially very mysterious. Eighteen `consolidated_components` were created with the identifier `the_cayley_authors_n` where n was an integer from 1 to 18. Each one of these consolidated components had exactly the same `consolidated_copyright` and `consolidated_license_expression`. In each case the `type` was `license-holders`. While the results returned are not, strictly speaking, invalid, the results are not useful or valuable, since there is no indication at all why there are 18 of these, and how each one might actually refer to a meaningful set of files in the package. You have to look at the file details to figure that out.

I did some analysis of the scan results details, and noticed that each of these 18 consolidated_components was consolidated from a group of files under a unique common directory. The pattern is clear and repetitive: an "assembly" directory followed by one or more lower-level detailed files or directories within that "assembly". I have attached the results of my analysis to this issue. The .zip file contains:
* a screenshot of one example in the raw json file.
* the json-formatted scan results
* an Excel spreadsheet with my analysis.

Please note in the Excel spreadsheet that I have provided a column called "Suggested Identifier" that proposes a meaningful conolidated component identifier. For example, instead of `the_cayley_authors_4` I suggest `graph-gaedatastore` since that consolidated component actually comprises 5 objects in the common Path `cayley-0.7.5/graph/gaedatastore` . There are multiple examples in the spreadsheet, including two consolidated components that do not fit the pattern of 18 described here; however, even for those exceptions, I have suggested better consolidated component identifiers.

[consolidated-component-analysis.zip](https://github.com/nexB/scancode-toolkit/files/3520554/consolidated-component-analysis.zip)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.