ImperialCollegeLondon / ImperialCollegeLondon/virtual_ecosystem

Tackling relative paths in config files.

Open
#273 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
20
Forks
5
Avg merge
2d 1h
Merged PRs (30d)
34

Description

Each config file can (in theory - limited to the `data` config and `core` at present) contain file paths, which _could_ be relative. If so, those are assumed to be relative, to the referring config file. The problem is that the referring file path is lost when the configs are merged.

@alexdewar has tackled this for existing cases in #269, but it leaves a hanging issue which _might_ affect later configs that contain paths. An example might be a path to a plant PFT file or something similar.

One way this could be tackled is to introduce an `isPath = true` property on appropriate JSONSchema entries. That way, the code can unambiguously identify paths that need to be made absolute using an existing canonical source of validation information.
However, I think that then requires two passes and a new Validator:

1. Merge the original config content to identify the modules and hence build the schema.
2. Run a PathResolverValidator using that schema over the individual files in `self.toml_contents`: this validates contents but also look for entries with `isPath=true` and updates the value with the absolute paths of the `toml_contents` keys. It doesn't add defaults though, because has to happen on the last validation pass to avoid creating clashes.
3. Now merge the updated `toml_contents` again and then run this through the current default filling validation.

This is a lot more complex but is also explicit about which inputs are paths.

_Originally posted by @davidorme in https://github.com/ImperialCollegeLondon/virtual_rainforest/pull/269#pullrequestreview-1557180149_

Contributor guide

Open the contributing guide

Research direction

Read PR #269 and trace the config-validation flow that merges self.toml_contents, focusing on where the referring config path is lost. The work is complete when relative paths in later config entries remain resolvable while defaults are still applied only during the final validation pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.