openzim / openzim/libzim

Writer function inefficient mimetype dirent entry rewriting

Open
#73 3 comments 0 reactions 1 assignee View on GitHub

@mgautierfr is already working on this.

Since Nov 5, 2017.

enhancement
Dominant language
C++
Stars
256
Forks
75
Avg merge
8d 51m
Merged PRs (30d)
4

Description

During the development of zimdiff/zimpatch we had the problem that two ZIM files were almost equal, except that the mimetypes were not sorted in the same way, so all dirent entry mimetype values were different.

This was an issue because zimpatches files were not equal to the original files. To avoid this, the zimlib forces currently the order of the mimetypes in the list in the header. They are sorted alphabeticaly.

Unfortunately, I see two problems with this:

  • This changes the specification of the format (we still don't have changed anything in the specifications)
  • The sorting of the mimetypes is done after all articles are inserted and this needs to rewrite all the dirent entries. Something which is really not efficient/elegant.

I think an alternative approach would be to allow to force the mime-type list before inserting the articles. This would shortcut the dynamic creation of this mime-type list and consequently avoid the two problems listed above.

First reported at https://phabricator.wikimedia.org/T57363

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.