Performance problems with long infinite scrolling list

How is it possible to achieve a DOM recycling like it is described here with packery?
Especially with more than one image column.
https://developers.google.com/web/updates/2016/07/infinite-scroller

Packery doesn't work the same way in both directions. The layout is calculated in one direction. And normally from top to bottom. This means that you can add any new items to the bottom, which are then simply placed at the bottom. But if you turn the (scroll-)direction around and add items above the viewport, the layout will be recalculated (from top to bottom) and all items will be rearranged, i.e. everything will move. If you could change the direction of the layout calculation from bottom to top in the moment when the scrolling direction is reversed and the items have to stay fixed in the viewport, the problem would be solved. Which, as far as I can see, doesn't work.

What possibilities are there to remove images below and above the viewport or to recycle the DOM, so that you can achieve a performance gain and only the images are stored (images in the scope of the viewport, once above and once below) that you do not unnecessarily overload the browser, especially with a large number of images in a very long list.

@desandro

Start by reading the issue alongside the linked Google infinite-scroller article and reviewing Packery's existing layout behavior in both scroll directions. No repository file, test, or concrete implementation boundary is identified; the work would need a defined DOM-recycling approach and measurable performance criteria before completion can be verified.

Written by the indexing model from the issue text.