Define Satpy extension/plugin package interfaces
@pnuu is already working on this.
Since May 20, 2019.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 335
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 10
Description
Feature Request
Is your feature request related to a problem? Please describe.
This was first brought up by @mraspaud, @jsolbrig, and me at the Spring PCW in Madison, WI (@pnuu and @sjoro were also brought in to the discussion at times iirc). The main real world use case for this feature request is a user wanting to add composite recipes/algorithms that have more than the usual requirements (ancillary data, complex build process like fortran algorithms, etc). It would be difficult to maintain these types of recipes in Satpy core especially considering these types of cases are not only for composites but also readers, writers, and possibly enhancements.
Describe the solution you'd like
After much discussion one solution seemed to be the most flexible and should work for all use cases. Alternative solutions are described in the additional context below.
The best solution seems to be extension packages that define entry points that can be used by satpy core. Satpy will see these entry points and look in the installed extension packages for configuration files.
We could define cookiecutter repositories for reader/writer/composite/enhancement extension packages to simplify the process of creating new ones. You can see an example of this type of repository with the jupyter widget repository.
Describe any changes to existing user workflow
Besides installing additional packages/dependencies, normal users should not see any difference.
Additional context
Originally the best solution seemed to be to use data_files in setup.py to put new configuration files in <prefix>/etc/satpy/. Extension packages could also put configs in this directory so they would be automatically discovered by satpy core. However, if you install a package in development mode (pip install -e . or python setup.py develop) or python setup.py install then the package is installed as an egg and data_files are kept inside the egg package. Given this problem, this doesn't seem like a good idea.
Another option would be to have extension packages call special satpy registration functions in their __init__.py modules. However, this would mean users would have to import any extension packages to use the features provided by them. With the main solution defined here with entry points, import is not necessary.
In a future issue (tomorrow maybe), I'll describe other similar changes that would simplify extension packages being defined. Mainly, converting the current sensor-based composite YAML files to be directories with a series of composite YAML configs for each sensor.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.