google / google/ml_collections
Saving ConfigDict to File -- Using ABSL
- Dominant language
- Python
- Stars
- 1k
- Forks
- 49
- PR merge metrics
- No merged PRs in 30d
Description
## The Problem:
I'm currently using the following pattern to load a config from dictionary
```python
from absl import app, flags
from ml_collections import config_flags
from .config import MY_CONFIG_DICT
FLAGS = flags.FLAGS
_CONFIG = config_flags.DEFINE_config_from_dict('config', MY_CONFIG_DICT)
# ...
```
`absl` allows me to alter the config from the command line - this is great for running experiments with different parameters. However, if I want to re-use the same configuration for another run, there doesn't seem to be a de-facto way to do this.
For example, I want to be able to run:
```bash
python -m src.experiment --config.value1 10 --config.value2 20
```
and then be able to re-use this config without having to remember to flags.
**How can we save and re-load a `ConfigDict` instance when using `absl` flags?**
## My Approach:
My current approach to this is:
**In the experiment script:**
1. Load config into flags as above.
2. Save the updated `ConfigDict` to a `.yml` file.
**In a postprocessing script:**
1. Load the original config using the same method.
2. Provide an additional flag which takes a path to the saved `.yml` file.
3. Update the original config as follows:
```python
# postprocessing.py
from absl import flags
from ml_collections import config_dict, config_flags
from .config import MY_CONFIG_DICT
FLAGS = flags.FLAGS
_CONFIG = config_flags.DEFINE_config_from_dict('config', MY_CONFIG_DICT)
_LOAD_CONFIG = flags.DEFINE_string('load_config', None, '.yml file from which to update config.')
def update_config():
def _update_config_dict(a, b) -> config_dict.ConfigDict:
da = a.to_dict(preserve_field_references=True)
db = b.to_dict(preserve_field_references=True)
return config_dict.ConfigDict(da | db)
with open(FLAGS.load_config, 'r') as f:
config_from_yaml = yaml.load(f, Loader=yaml.UnsafeLoader)
FLAGS.config = _update_config_dict(FLAGS.config, config_from_yaml)
def main(_):
update_config()
# ...
````
While this works it doesn't seem a great approach.
Is there any standard way to accomplish this - or should I be dealing with updated configs in a different manner?
Contributor guide
Assessment
This issue has not been assessed yet.