llrs / llrs/repo.data

Process all & subset vs appending new packages?

Open
#5 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

CRAN
Dominant language
R
Stars
3
Forks
1
Avg merge
2h 33m
Merged PRs (30d)
1

Description

CRAN functions are usually in a list format CRAN_archive_db(), CRAN_aliases_db(), CRAN_rdxrefs_db().
Currently the packages process it all and then retrieves the info for a specific package requested.
This is great if many packages are requested (packages = NULL).

If few packages are requested this is slow (even with cache). It might be better to simply subset the list to the requested packages and add the resulting polished data.frame to the cache dataset.

Perhaps some benchmark would be required. Currently my impression is that the download is the slowest part.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the CRAN_archive_db(), CRAN_aliases_db(), and CRAN_rdxrefs_db() entry points, comparing requests with packages = NULL against requests for a small package set. Benchmark processing, download, and cache behavior, then define done as a measured improvement for selective requests without regressing all-package processing.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.