Page-length estimation and the interaction between breakable footnotes and page-shrinking / balancing

Breaking a footnote onto a following page, particularly when the note is on one of the last lines of the page can be a very useful tool to avoid large chunks of white space. But while breaking notes is thus a useful possibility it should probably not be the default.

Currently, there are 2 processes that may result in broken notes:

1. TeX's internal page-breaking algorithm. This only(?) applies at the rare \fp marker, or in non-paragraphed footnotes, and depends on the page-length calculation (estimation) being correct.
2. The 'balancing' cycle, where the page filling / balancing mechanism attempts a given chunk of body text. If the body text and associated inserts don't fit, then notes are checked to see if one or more lines getting shaved off the note would allows that chunk to fit. Only then is the possibility of a smaller chunk of text considered.

If TeX's page-length estimate is too long, TeX will try to break the page early, it will all fit, but the page will have too much whitespace. People will moan at us. Thus, historically, we have deliberately chosen settings underestimate page length. The current settings account for notes at 40% (single column) or 80% (double column) of their estimated size.

Thus when long / multiple notes are in use on a page, TeX will try to fit more lines than can fit onto the page, and this may result in many failed fit-attempt cycles per page, wasting time. When (eventually) a balancing body text is found, the result is multiple 'excess' lines of body text being included on the page and the (last) note getting split between pages, rather than those lines being pushed onto the next page and the note remaining whole.

solutions/suggestions

• TeX's processing speed would be helped by accurate page-length estimates, and this would also reduce needless breaking of notes.
• There should be some interaction between the page balancer and the footnote shaver, so that the balancer first tries solutions without broken notes and only then ones with broken notes. Or some evaluation of penalties (or similar scores) for different solutions.

Start by tracing TeX's page-length estimation, internal page-breaking algorithm, balancing cycle, and footnote-shaving behavior described in the issue. Measure how estimated note sizes affect fit attempts and determine how balancing should prioritize unbroken notes; done should include a defined behavior and tests or reproducible cases for long and multiple footnotes.