EVerest / EVerest/cbexigen

Dynamically import configuration module from specific path

Open
#90 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
69
Forks
36
Avg merge
3h 30m
Merged PRs (30d)
1

Description

### Describe the problem

First of all, thank you for your job and for making this repo available for everyone!

I was trying to understand how your codec generator works. While using the repository as a submodule of another repo, I noticed that the function `get_config_module` from `tools_config.py` allways attempts to dynamically import a module based on the file name in `CONFIG_ARGS['config_file']`. However, it exclusively uses `importlib.import_module`, which relies on the module being discoverable in Python's `sys.path`. This introduces a minor limitation: if the specified file is not located in a directory already included in sys.path, the function will fail with a ModuleNotFoundError.

Here are two specific cases I encountered:

- If I generate a configuration file called `config.py` in a directory different from the current working directory (as shown in the attached image), cbexigen imports the `config.py` file located in the current working directory instead of the one in `config_dir` directory
![config1](https://github.com/user-attachments/assets/fcebc9fa-b191-42f6-a064-3b27b255d53c)
- If I generate a configuration file named `config_default` located in a directory other than the current working directory, I get a ModuleNotFoundError:
![config2](https://github.com/user-attachments/assets/6a744394-01ce-451e-9193-5dca1d4ac9ca)

I think that it is a path dependency issue: The code does not use the full path of the configuration file; it extracts only the file name. This assumes the file resides in a directory that is already accessible via sys.path, which may not always be the case.

The function we are talking about is:

```python
def get_config_module():
global __CONFIG_MODULE

if __CONFIG_MODULE is None:
config_module_name = Path(CONFIG_ARGS['config_file']).name
if config_module_name.endswith('.py'):
config_module_name = config_module_name[:-3]
__CONFIG_MODULE = importlib.import_module(config_module_name)

return __CONFIG_MODULE
```

### Describe your solution

I tried implementing a solution using `importlib.util`, which provides a more robust approach by loading the module directly from its specified file path. This method does not rely on the file being located in `sys.path` or having a unique name.

Here’s how the proposed solution addresses the issues:

- Direct Path Import: By using `importlib.util.spec_from_file_location` and `module_from_spec`, the module is loaded explicitly from the provided path. This bypasses the sys.path mechanism, ensuring that the correct file is always imported.
- Name Handling: The module name is derived programmatically from the file's base name using `stem` property (e.g., config.py becomes config), without manually checking if the string `.py` is contained in the name.

Here’s the updated implementation, which also ensures backward compatibility:

```python
def get_config_module():
global __CONFIG_MODULE

if __CONFIG_MODULE is None:

config_module = Path(CONFIG_ARGS['program_dir'], CONFIG_ARGS['config_file']).resolve()
config_module_name = config_module.stem

spec = importlib.util.spec_from_file_location(config_module_name, config_module)
__CONFIG_MODULE = importlib.util.module_from_spec(spec)
spec.loader.exec_module(__CONFIG_MODULE)

return __CONFIG_MODULE
```

### Additional context

My project is formed with the following structure:

- src: source code, where I have configuration files and some utilities to craft V2G messages via Python
- cbexigen: repo as submodule
- README
- etc

My working environment was formed by:

- Pycharm IDE
- Python virtual environment based on Python3.11
- I have also created a `pyproject.toml` in order to make your library installable and allow Pycharm to automatically resolve all dependencies without the need of manually adding the cbexigen/src to sys.path. It also allow autocompletion when the instruction `import cbexigen` is used

The pyproject.toml I am using is quite similar to:

```toml
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "my_project"
version = "0.0.1rc0"
authors = [
]
readme = "README.md"
requires-python = ">=3.9"
dependencies = [
"Jinja2",
"xmlschema",
]

[project.optional-dependencies]
dev = [
]

[tool.setuptools.packages.find]
where = ["src", "cbexigen/src"]

```
I hope something of this information could be useful, thank you in advance!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.