Provide a better abstraction for customizing downloader behavior

Author: @dralley (dalley)

Redmine Issue: 7352, https://pulp.plan.io/issues/7352

Customization of downloader behavior is currently achieved by subclassing one of the downloader types from the Plugin API and then modifying the get_downloader() method and downloader_factory property on the Remote (sub)class.

A limitation of this abstraction is that while the get_downloader() method accepts **kwargs to pass down to the (possibly customized) downloader, it will pass all kwargs to any downloader without discriminating by the type of downloader. Therefore it's difficult to send a kwarg to be used on a customized HTTP downloader without it being also sent to other types of downloaders such as FileDownloader which are not expecting it.

This could be worked around by additionally modifying the DownloaderFactory, but it's currently suboptimal because:

• The methods you'd want to override are underscore-prefixed (private)
• It would create extra coupling and interdependence between the classes; not the end of the world, but kind of messy

We could probably benefit from revisiting the design and incorporate some of our experience with customizing downloader behavior in various plugins.

Start by reading the Plugin API downloader types and the Remote methods named in the issue: get_downloader() and downloader_factory. Compare how customized HTTP downloaders and other downloader types receive kwargs, then define a cleaner customization boundary where downloader-specific options are handled safely; the work is done when the design avoids private-method coupling and does not pass unsupported kwargs to unrelated downloaders.