openSUSE / openSUSE/MirrorCache

Per-package daily download history via the REST API

Open
#661 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Perl
Stars
50
Forks
23
PR merge metrics
No merged PRs in 30d

Description

stat_download returns rolling windows (cnt_1d/7d/30d/total) but not a series, so to build a daily history each consumer has to poll once a day and store the result. I do that for https://github.com/r-observatory/autoobs-downloads.

The daily data is already collected, though. agg_download_pkg holds one row per (period, dt, metapkg_id, folder_id, country):

https://github.com/openSUSE/MirrorCache/blob/fb90e38583b3d702a48b9c828972d3d3f2ee9cb3/lib/MirrorCache/resources/migrations/Pg.sql#L450-L458

StatAggPkg already writes period='day' rows on a schedule:

https://github.com/openSUSE/MirrorCache/blob/fb90e38583b3d702a48b9c828972d3d3f2ee9cb3/lib/MirrorCache/Task/StatAggPkg.pm#L38-L39

Cleanup keeps them for 6 months:

https://github.com/openSUSE/MirrorCache/blob/fb90e38583b3d702a48b9c828972d3d3f2ee9cb3/lib/MirrorCache/Task/Cleanup.pm#L178-L183

And stat_download already reads period='day' (last 30 days) before collapsing it into windows:

https://github.com/openSUSE/MirrorCache/blob/fb90e38583b3d702a48b9c828972d3d3f2ee9cb3/lib/MirrorCache/WebAPI/Controller/Rest/Metapkg.pm#L176-L199

Could you expose those rows directly, e.g. GET /rest/package/<id>/stat_download_daily returning [{date, count}, ...]? It would be a near-copy of stat_download:

select to_char(dt,'YYYY-MM-DD') as date, sum(cnt) as count
from agg_download_pkg
where metapkg_id = ? and period = 'day'
group by dt order by dt;

plus one route line next to the existing stat_download route:

https://github.com/openSUSE/MirrorCache/blob/fb90e38583b3d702a48b9c828972d3d3f2ee9cb3/lib/MirrorCache/WebAPI.pm#L194

That saves every downstream from running its own snapshotter, and gives up to 6 months of backfill.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with lib/MirrorCache/WebAPI/Controller/Rest/Metapkg.pm, where stat_download reads daily rows, and lib/MirrorCache/WebAPI.pm for the adjacent route. Check the agg_download_pkg schema and existing day aggregation, then expose the requested date/count response ordered by date; verify the endpoint returns the retained daily history for a package.

Written by the indexing model from the issue text.

Assessment

Tech stack
perl, postgresql
Domain
api, backend, databases
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.