Condition publication on mirroring status
Open
@benoit74 is already working on this.
Since Sep 8, 2026.
enhancement
question
- Dominant language
- Python
- Stars
- 7
- Forks
- 1
- Avg merge
- 15h 20m
- Merged PRs (30d)
- 18
Description
Blocked by https://github.com/kiwix/minibrain/issues/2
With the current setup, the publication of new ZIM files happen in this order:
- File is uploaded from worker to master mirror (as a tmp file that is then renamed).
At this moment, anyone with the path can download it from mirrorbrain or master mirror URL (MB forwards to master if not known). - At random library-gen scans files and rebuilds library.xml.
At this moment file is in the catalog for everybody. - At random mirrorbrain scans folder. Discovering new files, it needs to compute a lot of hashes which can summed to minutes for large file on busy server.
Once complete,.md5,.torrentand other endpoints are available. - At random each mirror discovers via rsync the new file and syncs it.
Once complete, the file is downloadable using the mirror URL - At random mirrobrain scans the mirrors and discovers that the file is present on the mirror.
Only then can it forward clients to that mirror for that file.
Keep in mind mirror can have geo-restriction so maybe this one only serves some country•ies.
That's the journey of a ZIM file. Problems can occur for instance:
- After step 2, it's visible and thus users can attempt to download. It's only on master mirror so every user is served by master mirror.
- After step 2, other tools like imager-service sees the content and use it (it's most likely an update of some existing content). Because hashes dont exist yet, building image fails.
We want I think a condition to only publish after step 3.
We should also be able to configure mirroring threshold for worldwide availability.
This obviously mandates that the load-balancer shares its state and config in a machine readable format.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.