aboutcode-org / aboutcode-org/scancode.io

Improve pipeline creator UX to navigate CodebaseResource

Open
#369 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
215
Forks
203
Avg merge
4d 8h
Merged PRs (30d)
6

Description

In https://github.com/nexB/scancode.io/tree/app-package-files I am working toward adding details of installed files for a given package type. This is designed to run either as a secondary pipeline after another scan, or to later integrate as a step.

I found it difficult to navigate CodebaseResource for doing these operations:

- given a resource navigate up to its parent
- given a pathlib.Path get the corresponding resource
- OR given an absolute path string get the corresponding resource
- walk the filesystem project/codebase directory and be able to easily get to a corresponding CodebaseResource

Some ugly thing I did include this https://github.com/nexB/scancode.io/blob/app-package-files/scanpipe/pipelines/add_files_to_packages.py#L58

Basically the operations I am doing for PyPI packages consist in:
- iterate all DiscoveredPackage of type "pypi"
- given a METADATA file present in the codebase for an installed "pypi" package
- go to the parent dir (typically a .dist-info) and create a native distribution object that can parse all installed wheel metafiles
- for each installed file of this distribution, find its CodebaseResource. a distribution file will be a pathlib.Path
- Add the package to the CodebaseResource discovered_packages

We have a decent way for rootfs resources which are rooted by construction. But for other resources this is more involved.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.