aboutcode-org / aboutcode-org/purldb

debian visitor: do not unnecessarily process ls-lR files

Open
#5 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
67
Forks
69
Avg merge
8d 8h
Merged PRs (30d)
1

Description

Currently the Debian visitor seems to download and process the `ls-lR` files. Although these change on an almost daily basis it might be wise to add one extra step to calculate a checksum of the `ls-lR` file that was downloaded and verify if there is a checksum from a previous invocation. If there is none, continue processing. If there is one, and it doesn't match, continue processing. If there is one and it matches, exit.

After processing the checksum should be stored in a persistent (configurable?) location.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.