Dynamically import configuration module from specific path
- Dominant language
- Python
- Stars
- 69
- Forks
- 36
- Avg merge
- 3h 30m
- Merged PRs (30d)
- 1
Description
### Describe the problem
First of all, thank you for your job and for making this repo available for everyone!
I was trying to understand how your codec generator works. While using the repository as a submodule of another repo, I noticed that the function `get_config_module` from `tools_config.py` allways attempts to dynamically import a module based on the file name in `CONFIG_ARGS['config_file']`. However, it exclusively uses `importlib.import_module`, which relies on the module being discoverable in Python's `sys.path`. This introduces a minor limitation: if the specified file is not located in a directory already included in sys.path, the function will fail with a ModuleNotFoundError.
Here are two specific cases I encountered:
- If I generate a configuration file called `config.py` in a directory different from the current working directory (as shown in the attached image), cbexigen imports the `config.py` file located in the current working directory instead of the one in `config_dir` directory

- If I generate a configuration file named `config_default` located in a directory other than the current working directory, I get a ModuleNotFoundError:

I think that it is a path dependency issue: The code does not use the full path of the configuration file; it extracts only the file name. This assumes the file resides in a directory that is already accessible via sys.path, which may not always be the case.
The function we are talking about is:
```python
def get_config_module():
global __CONFIG_MODULE
if __CONFIG_MODULE is None:
config_module_name = Path(CONFIG_ARGS['config_file']).name
if config_module_name.endswith('.py'):
config_module_name = config_module_name[:-3]
__CONFIG_MODULE = importlib.import_module(config_module_name)
return __CONFIG_MODULE
```
### Describe your solution
I tried implementing a solution using `importlib.util`, which provides a more robust approach by loading the module directly from its specified file path. This method does not rely on the file being located in `sys.path` or having a unique name.
Here’s how the proposed solution addresses the issues:
- Direct Path Import: By using `importlib.util.spec_from_file_location` and `module_from_spec`, the module is loaded explicitly from the provided path. This bypasses the sys.path mechanism, ensuring that the correct file is always imported.
- Name Handling: The module name is derived programmatically from the file's base name using `stem` property (e.g., config.py becomes config), without manually checking if the string `.py` is contained in the name.
Here’s the updated implementation, which also ensures backward compatibility:
```python
def get_config_module():
global __CONFIG_MODULE
if __CONFIG_MODULE is None:
config_module = Path(CONFIG_ARGS['program_dir'], CONFIG_ARGS['config_file']).resolve()
config_module_name = config_module.stem
spec = importlib.util.spec_from_file_location(config_module_name, config_module)
__CONFIG_MODULE = importlib.util.module_from_spec(spec)
spec.loader.exec_module(__CONFIG_MODULE)
return __CONFIG_MODULE
```
### Additional context
My project is formed with the following structure:
- src: source code, where I have configuration files and some utilities to craft V2G messages via Python
- cbexigen: repo as submodule
- README
- etc
My working environment was formed by:
- Pycharm IDE
- Python virtual environment based on Python3.11
- I have also created a `pyproject.toml` in order to make your library installable and allow Pycharm to automatically resolve all dependencies without the need of manually adding the cbexigen/src to sys.path. It also allow autocompletion when the instruction `import cbexigen` is used
The pyproject.toml I am using is quite similar to:
```toml
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"
[project]
name = "my_project"
version = "0.0.1rc0"
authors = [
]
readme = "README.md"
requires-python = ">=3.9"
dependencies = [
"Jinja2",
"xmlschema",
]
[project.optional-dependencies]
dev = [
]
[tool.setuptools.packages.find]
where = ["src", "cbexigen/src"]
```
I hope something of this information could be useful, thank you in advance!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.