Create a ZIM with all versions of a given project
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Currently, it is only possible to create one ZIM per slug. E.g. one for one for lua5.4, one for lua5.3, one for lua~5.2 and so on.
Analyzing a bit devdocs, it showed this means we will create 716 ZIMs, with new ZIMs coming at every release of projects (lua, ...).
This is way too much, and this is not really convenient for the end-user who typically know he needs documentation of a given programming language, but usually need documentation for multiple versions.
We would like the scraper to be able to embed all versions of a given project (e.g. lua) in a single ZIM. This obviously means significant changes to the codebase, both in terms of ZIM structure (to accommodate to multiple slugs in one ZIM) but also in terms of UI (to be able to switch from one slug to the next one).
It probably also means that something needs to be done to provide custom search capabilities directly in the UI: currently suggestions and full-text search are already hard to use, but it will become even harder if we have multiple slugs in one ZIM. Or maybe this means we need to enrich the search capabilities of the libzim (e.g. ask the libzim to search only in a subset of articles ...).
To make the feature as generic as possible, I think that we should continue to use the CSV --slug parameter to specify all slugs that we want to embed, and add a --oneSingleZim (or something like this, I'm bad at choosing names) parameter to indicate we want all slugs in a single ZIM file. With this approach, it means we have lot's of flexibilities, and we can for instance embed all Vue (Vue, VueRouter, Vuex, ...) in a single ZIM file even if they are distinct project son DevDocs side. Or we can even create a "all DevDocs" ZIM with all documentation.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the scraper entry point that handles the CSV --slug parameter and ZIM generation, then trace the UI and search paths described in the issue. Done would require a --oneSingleZim-style option, support for multiple slugs in one ZIM, version switching in the UI, and search behavior that works across the embedded projects.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, frontend, search
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100