[ Proposal ] Downloader changes

pinging a couple of stakeholders just for their thoughts: @m4xw @jdgleaver @twinaphex

Description

The downloader is currently very limited. One of the key limitations is a lack of any sort of federation (e.g., downloading cores from other repos), along with not being able to download things to specific locations. (see: https://github.com/libretro/RetroArch/issues/10182 and https://github.com/libretro/RetroArch/issues/11208 among others).

Expected behavior

I propose allowing multiple buildbot URLs to be specified, possibly separated with a pipe in the config, and they would be parsed in order (that is, if someone appended stenzek's hypothetical duckstation repo, it would list their core[s] over ours in the case of conflicts).

The next proposal (doesn't depend on the previous) is to use a standardized layout of directories across supported platforms (could be a problem for some console platforms that are pretty invested in nonstandard paths already ; a related issue for linux is here: https://github.com/libretro/RetroArch/issues/11621 ) and always download things to the top level, so we can include the desired directory structure with the download and everything just goes where it needs to live. For example, a dolphin core download would include a 'cores' directory with the core inside it, an 'info' subdirectory inside that containing the info, and a system/dolphin-emu subdir that contains any legally distributable files. In the case of Flycast, the empty system/dc directory could be provided so users don't need to create it (after looking up what its called). We could also consider including a README in any of these empty directories that lists BIOS files, etc. (yes, I know this is already in the info files, but it's not a very friendly format to parse).

This second proposal could cause problems for people who change the location of directories, though, which includes me (I use a network location for my 'system' dir), so it's not something to change lightly. However, I think it would provide a significant improvement to usability.

Actual behavior

The downloader can only pull from a single server at a time, and users have to move ancillary files around manually.

The issue names no source files, tests, or entry points. Start by locating the downloader and its buildbot URL configuration, then review linked issues 10182, 11208, and 11621. Done would require an agreed design and implementation plan for multiple repositories, standardized download layouts, and configurable destination paths.