Enforce eager validation of configuration parameters
- Dominant language
- Python
- Stars
- 132
- Forks
- 69
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 47
Description
Right now config parameters (read from `ceph-nvmeof.conf` or from whatever indicated via the `-c ` with the [Python `configparser` standard library](https://docs.python.org/3/library/configparser.html)) are validated whenever accessed. The potential validation issues are:
1. **The parameter is not defined**. This is partially solved by 2 approaches:
* Define defaults in the `ceph-nvmeof.conf` template. However, this complicates code maintenance, since configuration is partially detached from the code.
* Define defaults at access time. This leads to potential inconsistencies between config file defaults.
1. **The parameter type is not correct** (e.g.: `rpc_socket` is a number). `configparser` doesn't enforce any type of schema or static typing, hence all type checking is dynamic.
1. **The parameter is semantically correct** (e.g.: `rpc_socket` is an invalid file).
This leads to a lazy validation approach, which is problematic, as once the gateway is initialized there's no way to fix configuration problems (the config file is read once at initialization phase).
This should be generally aligned with a 'fail-fast' approach refactoring. That is, fail ASAP if:
* config parameters are invalid,
* Ceph auth permissions are incorrect (right now there are situations where the gateway starts without proper permissions and only fails when it tries to write something to the RADOS state object).
Suggestions to fix this:
* Enforce eager validation:
* This has already been tried with home-brew approach: https://github.com/ceph/ceph-nvmeof/pull/126.
* Using existing libraries, like [`configargparse`](https://github.com/bw2/ConfigArgParse), which leverages `argparse` syntax to enforce preemptive validation.
* Define a schema for the config file:
* Given `configparse` INI files can be easily converted into dicts, and dicts into JSON, we could easily use JSON Schema to perform a validation. However that only covers validation issues [\#1](#param_not_defined) and [\#2](#param_not_correct), and JSON Schema is not the friendliest thing to work with (and doesn't enforce configuration-as-code pattern either).
### Related
* https://github.com/ceph/ceph-nvmeof/issues/49
* https://github.com/ceph/ceph-nvmeof/issues/96
* https://github.com/ceph/ceph-nvmeof/pull/126
Contributor guide
Research direction
Start by tracing how configuration is read with Python's configparser and how parameters are validated during gateway initialization. Review the prior approach in pull request #126 and the related issues #49 and #96 before choosing between eager validation, configargparse, or schema validation. Done means invalid configuration and incorrect Ceph auth permissions fail before the gateway starts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100