clearlydefined / clearlydefined/crawler

Scancode if failing on a git component

Open
#442 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
60
Forks
38
Avg merge
1h 49m
Merged PRs (30d)
1

Description

When attempting to harvest:

`git/github/AndroidSDKSources/android-sdk-sources-for-api-level-31/2c56b25f619575bea12f9c5520ed2259620084ac`

ClearlyDefined and Licensee run just fine, but Scancode errors out with:

```

crawler_1 | [I] Analyzing scancode@cd:/git/github/androidsdksources/android-sdk-sources-for-api-level-31/2c56b25f619575bea12f9c5520ed2259620084ac using ScanCode. input: /tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31 output: /tmp/cd-VMvcP7 {"crawlerId":"18d884e4-fd2d-491a-bd8f-fede171cc5e9","buildNumber":"0"}
crawler_1 | trackException:
crawler_1 | Error: Error: Command failed: /opt/scancode-toolkit-3.0.2/scancode --copyright --license --info --license-text --is-license-text --package --license-diag --strip-root --email --url --license-clarity-score --classify --generated --summary --summary-key-files --timeout 1000 -n 2 --json-pp /tmp/cd-VMvcP7 /tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31
crawler_1 | Setup plugins...
crawler_1 | Collect file inventory...
crawler_1 | ERROR: failed to collect codebase at: '/tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31'
crawler_1 | Traceback (most recent call last):
crawler_1 | File "/opt/scancode-toolkit-3.0.2/src/scancode/cli.py", line 707, in scancode
crawler_1 | max_in_memory=max_in_memory
crawler_1 | File "/opt/scancode-toolkit-3.0.2/src/scancode/resource.py", line 280, in __init__
crawler_1 | self._populate()
crawler_1 | File "/opt/scancode-toolkit-3.0.2/src/scancode/resource.py", line 439, in _populate
crawler_1 | parent = parent_by_loc.pop(top)
crawler_1 | KeyError: '/tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31/com/android/ims/rcs/uce'
crawler_1 |
crawler_1 | Removing temporary files...done.
crawler_1 |
crawler_1 | at ChildProcess.exithandler (child_process.js:308:12)
crawler_1 | at ChildProcess.emit (events.js:314:20)
crawler_1 | at ChildProcess.EventEmitter.emit (domain.js:483:12)
crawler_1 | at maybeClose (internal/child_process.js:1022:16)
crawler_1 | at Socket. (internal/child_process.js:444:11)
crawler_1 | at Socket.emit (events.js:314:20)
crawler_1 | at Socket.EventEmitter.emit (domain.js:483:12)
crawler_1 | at Pipe. (net.js:675:12) {
crawler_1 | killed: false,
crawler_1 | code: 2,
crawler_1 | signal: null,
crawler_1 | cmd: '/opt/scancode-toolkit-3.0.2/scancode --copyright --license --info --license-text --is-license-text --package --license-diag --strip-root --email --url --license-clarity-score --classify --generated --summary --summary-key-files --timeout 1000 -n 2 --json-pp /tmp/cd-VMvcP7 /tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31',
crawler_1 | stdout: '',
crawler_1 | stderr: 'Setup plugins...\n' +
crawler_1 | 'Collect file inventory...\n' +
crawler_1 | "ERROR: failed to collect codebase at: '/tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31'\n" +
crawler_1 | 'Traceback (most recent call last):\n' +
crawler_1 | ' File "/opt/scancode-toolkit-3.0.2/src/scancode/cli.py", line 707, in scancode\n' +
crawler_1 | ' max_in_memory=max_in_memory\n' +
crawler_1 | ' File "/opt/scancode-toolkit-3.0.2/src/scancode/resource.py", line 280, in __init__\n' +
crawler_1 | ' self._populate()\n' +
crawler_1 | ' File "/opt/scancode-toolkit-3.0.2/src/scancode/resource.py", line 439, in _populate\n' +
crawler_1 | ' parent = parent_by_loc.pop(top)\n' +
crawler_1 | "KeyError: '/tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31/com/android/ims/rcs/uce'\n" +
crawler_1 | '\n' +
crawler_1 | 'Removing temporary files...done.\n'
crawler_1 | }
crawler_1 | at AzureApplicationInsightsLogger.log (/opt/service/node_modules/winston-azure-application-insights/lib/winston-azure-application-insights.js:159:12)
crawler_1 | at transportLog (/opt/service/node_modules/winston/lib/winston/logger.js:234:15)
crawler_1 | at /opt/service/node_modules/winston/node_modules/async/lib/async.js:157:13
crawler_1 | at _each (/opt/service/node_modules/winston/node_modules/async/lib/async.js:57:9)
crawler_1 | at Object.async.each (/opt/service/node_modules/winston/node_modules/async/lib/async.js:156:9)
crawler_1 | at exports.Logger.Logger.log (/opt/service/node_modules/winston/lib/winston/logger.js:246:9)
crawler_1 | at exports.Logger.log (/opt/service/node_modules/diagnostic-channel-publishers/dist/src/winston.pub.js:53:28)
crawler_1 | at exports.Logger.target. [as error] (/opt/service/node_modules/winston/lib/winston/common.js:54:18)
crawler_1 | at Crawler._errorHandler (/opt/service/ghcrawler/lib/crawler.js:132:19)
crawler_1 | at async Crawler._run (/opt/service/ghcrawler/lib/crawler.js:73:25)
crawler_1 | [I] Error scancode@cd:/git/github/androidsdksources/android-sdk-sources-for-api-level-31/2c56b25f619575bea12f9c5520ed2259620084ac Command failed: /opt/scancode-toolkit-3.0.2/scancode --copyright --license --info --license-text --is-license-text --package --license-diag --strip-root --email --url --license-clarity-score --classify --generated --summary --summary-key-files --timeout 1000 -n 2 --json-pp /tmp/cd-VMvcP7 /tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31
crawler_1 | Setup plugins...
crawler_1 | Collect file inventory...
crawler_1 | ERROR: failed to collect codebase at: '/tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31'
crawler_1 | Traceback (most recent call last):
crawler_1 | File "/opt/scancode-toolkit-3.0.2/src/scancode/cli.py", line 707, in scancode
crawler_1 | max_in_memory=max_in_memory
crawler_1 | File "/opt/scancode-toolkit-3.0.2/src/scancode/resource.py", line 280, in __init__
crawler_1 | self._populate()
crawler_1 | File "/opt/scancode-toolkit-3.0.2/src/scancode/resource.py", line 439, in _populate
crawler_1 | parent = parent_by_loc.pop(top)
crawler_1 | KeyError: '/tmp/cd-l2Zcrv/android-sdk-sources-for-api-level-31/com/android/ims/rcs/uce'
crawler_1 |
crawler_1 | Removing temporary files...done.

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.