Looking for an efficient way to monitor for download folder changes

Motivation

I reported https://progress.opensuse.org/issues/123797 about a problem that a pipeline that was monitoring the URL http://download.opensuse.org/repositories/GNOME:/Medias/images/iso/?P=GNOME_Next* is now always reporting changes even though no new files where showing up in that folder.

the content always changes as the generated HTML page shows a "csrf-token" changing on each call. I found that by calling

diff <(curl -sS "http://download.opensuse.org/repositories/GNOME:/Medias/images/iso/?P=GNOME_Next*") <(curl -sS "http://download.opensuse.org/repositories/GNOME:/Medias/images/iso/?P=GNOME_Next*")

which yields:

<       <meta name="csrf-token" content="3a72819665c9adb750ad4d5e8054961c5b1c3efc" />
---
>       <meta name="csrf-token" content="0b17cbf73c393a1baa4daee989158038caeafc7c" />

As there is also no "last-modified" served in HEAD of those documents and also not when looking into files themselves like

curl --head "http://download.opensuse.org/repositories/GNOME:/Medias/images/iso/GNOME_Next.x86_64.iso.sha256"

I wonder what is the best approach to look for changes in a folder and trigger external services accordingly. Right now the best approach I found is to download the checksum file http://download.opensuse.org/repositories/GNOME:/Medias/images/iso/GNOME_Next.x86_64.iso.sha256 recurringly and check for changes in content

Suggestions

• Maybe the always changing "csrf-token" can be removed from those generated pages. I don't think this needs to be delivered to users
• Would it be possible to add a "last-modified" to the page header?

Start by reproducing the reported changes with the download URL and the two curl requests, including the checksum file and HEAD request. Compare the generated directory response and headers to determine which stable change signal can support monitoring; done means avoiding false changes while still exposing actual folder updates.