Option to prevent the scraping of shared links

Context

When sharing a link to a document with a friend over a communication tool such as Facebook Messenger, the document is downloaded by the tool provider and a preview of the document is published in the tool.

Problem

Tool providers, especially spying corporations, scrape the content of the documents users link to. This often goes against the point of using Nextcloud to not share the documents with them.

Desired solution

User: When sharing a link, users have an option to prevent access from scraping/preview bots to access the document linked. Users and/or administrators have an option to enable this by default.

Technically: Nextcloud analyses HTTP headers/ User-Agents of incoming requests and returns an 403 Forbidden error if a bot is detected.

Alternatives considered

Link Password protection: Achieves the expected goal, but has a poor usability when the confidentiality of the documents is low. Users will have to generate a non-trivial password and add it in their message next to the link, else the content will be scraped. Users may also often forget to password-protect the link.

Filter on the reverse proxy: When using a reverse-proxy in front of Nextcloud, system administrators can add filters on the HTTP headers/IP ranges to block the crawlers. However all links are then considered the same, and these rules have to be implemented by every system administrator, for every reverse-proxy (Nginx, Haproxy, Caddy, ...)

No files, tests, or entry points are named in the issue. Start by locating the shared-link request handling and configuration paths, then determine how bot detection and user or administrator defaults fit there; done means configurable shared-link protection that blocks detected preview bots without preventing intended access.