pyinfra-dev / pyinfra-dev/pyinfra
Merging of data structures in data hierarchy
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6k
- Forks
- 548
- Avg merge
- 7d 17h
- Merged PRs (30d)
- 13
Description
Describe the solution you'd like
In the existing implementation of the data hierarchy, there is a good way for handling overriding of flat values, but for the case where you might want to use nested structures (e.g. dictionaries, lists, sets) it might be useful to allow for merging of those structures as you traverse the hierarchy. This could be made configurable using the config functionality so that it can be set on a per-module basis.
For instance, if I have a deploy where I would like to install and configure a MySQL database, I can set the default set of configuration options in all.py as a dictionary of key/value pairs. In a staging.py file that targets hosts in a pre-production environment I can then specify a dictionary that overrides a subset of those values.
This can currently be achieved by flattening the structure and specifying all of the keys at the top level and prefixing them with a namespace for the deploy, but using a more rich data structure it provides implicit namespacing. It also provides the advantage of being able to render out configuration files that are intended to be JSON or YAML by simply serializing the merged result of the data object and writing that as the contents of the file.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the existing data hierarchy and config functionality to understand how flat values are currently overridden as modules are traversed. Define the merge behavior for nested dictionaries, lists, and sets, including how it is configured per module, then verify that merged results can be serialized for JSON or YAML output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100