danhper / danhper/python-i18n

Support loading translation files with importlib.resources files() API

Open
#30 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
246
Forks
46
PR merge metrics
No merged PRs in 30d

Description

Right now the library lets me pass in a string path to a directory containing translation files & internally the files are loaded using standard python file system (`os.path`) and file loading (`open()`) code.

I'd also like to be able to pass in a python style package and then the library would go away and use importlib.resources to load all the translation files in that package.

The main reason for my use case is that `importlib.resources` should work more reliably with various freezing programs such as PyOxidiser & PyInstaller in the future, which is useful for distributing games. Right now when including data files in my GUI library package (in this case the translations for default text on GUI elements) I have to fall back on using the `__file__` hack to get a path without `importlib.resources` . There may be other additional benefits to `importlib.resources` as it is a fairly recent addition to the python standard library.

Desired user API:
```
i18n.load_package_path.append("data.translations")
```

Example of some potential `importlib.resources` usage internally:
```
from importlib.resources import files

def load_translation_file_from_package(file_name, package, locale=config.get('locale')):
skip_locale_root_data = config.get('skip_locale_root_data')
root_data = None if skip_locale_root_data else locale
translations_dic = load_resource(package.joinpath(file_name), root_data)

def load_package(package_name, locale=config.get('locale')):
package = files(package_name)
for file in package.iterdir():
if file.is_file() and file.name.endswith(config.get('file_format')):
if '{locale}' in config.get('filename_format') and not locale in file.name:
continue
load_translation_file_from_package(file.name, package, locale)
```

I've skipped a bunch of the required code there, but from a swift glance over it - it should be possible to adapt what's here already to load using `importlib.resources` methods like `.read_text()` which creates a `StringIO` that can be parsed by the various `.json` and `.yaml` parsers.

There are also some shenanigans with configuration needed to support this as:

a) the new files() API to `importlib.resources` was only added in Python 3.9 replacing an older API.
b) the older API itself only existed in the standard library since Python 3.7
c) However, there is a backport called `importlib_resources` for pre 3.9 versions of Python - but that has the downside of introducing an additional dependency if you are leery of those (though at least this one is in the standard library past 3.7/3.9).

I'm hoping this is the last time the standard resource loading APIs for python are going to change.

**Extra note in case you try testing this**: data packages, just like regular python packages, need an `__init__.py` file in their directory. However since the 3.9 files() API you don't need an `__init__.py` file in parent directories of your data directory any more so you can do:

```
main-package
|___code_file.py
|___another_code_file.py
|___data
|___translations
|___ __init__.py
|___ main-package.en.json
|___ main-pacakge.fr.json

```

Here is also a helpful video on why importlib.resources exists if you want more information than I have provided here:
https://pyvideo.org/pycon-us-2018/get-your-resources-faster-with-importlibresources.html

Useful details on the config required here:
https://discuss.python.org/t/deprecating-importlib-resources-legacy-api/11386

Docs for importlib.resources from the backport:
https://importlib-resources.readthedocs.io/en/latest/index.html

Official documentation from Python 3.9:
https://docs.python.org/3.9/library/importlib.html#module-importlib.resources

P.S. I'm enjoying using the library, making light work of a task I've been wary of, so thanks for creating it!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.