internetarchive / internetarchive/epub

Fix header/footer removal for texts

Open
#32 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
41
Forks
3
PR merge metrics
No merged PRs in 30d

Description

The current header/footer identification scheme looks at the first/last logical paragraph on a page and attempts to match it against a regex to determine if it's a header/footer to be removed.

This has a number of failure modes:
- header & page number can be split into two separate blocks by the page layout analysis causing them to be in separate paragraphs
- page layout analysis can fail in ways that cause the header to not be the first logical paragraph (e.g. marginalia, noise, etc)
- scannos can introduce noise that causes the regex to fail

A more robust header/footer removal strategy would greatly improve the quality of epubs, particularly for text-to-speech where they're very distracting. Some scoring attributes might be things like physical page position, all caps, leading/trailing number patching pageno from scandata, edit distance from inferred header template, etc

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.