finos / finos/symphony-bdk-python
BdkConfigLoader.load_from_file decodes the config with the platform codec, silently corrupting non-ASCII values
- Dominant language
- Python
- Stars
- 37
- Forks
- 44
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 11
Description
## Describe the bug
`BdkConfigLoader.load_from_file` reads the config with no encoding, so Python decodes it with `locale.getpreferredencoding(False)` — the ANSI codepage on Windows:
```python
# symphony/bdk/core/config/loader.py:27
config_content = config_path.read_text()
```
YAML and JSON are both specified as UTF-8, so any non-ASCII value in a config is decoded with the wrong codec on a non-UTF-8 host.
**The failure is silent.** On a codepage that happens to map the bytes there is no exception at all: `load_from_file` returns a `BdkConfig` whose values are mojibake, the bot starts, and it fails later against the proxy or pod with an error that points nowhere near the config.
## To Reproduce
A config carrying a non-ASCII proxy credential, written to disk as UTF-8 bytes, loaded on Windows with a cp950 console:
```
on disk : password: "sésame-café" (b'password: "s\xc3\xa9same-caf\xc3\xa9"')
loaded : password='s矇same-caf矇' ← wrong, and nothing reported it
username: 'caf矇-user'
```
The sibling entry point on the same content is correct, which isolates it to the read:
```
load_from_content(...) → password='sésame-café' ✓
```
Non-ASCII in a proxy password is not exotic, and `defaultHeaders` and the various name fields are all free text.
## Expected behavior
`load_from_file` and `load_from_content` agree for the same config, on every host.
## Environment
Windows 10, Python 3.11, `locale.getpreferredencoding(False)` = `cp950`, repository at `21a0370` (tip of `main`).
Not specific to CJK locales: cp1252 is the default on en-US Windows and mangles the same bytes just as quietly. A `LC_ALL=C` host gets ASCII and raises instead.
## Additional context
The repository already has the right pattern elsewhere — `symphony/bdk/core/service/user/user_service.py:798` opens with `encoding="utf-8"` — so the fix is one argument on line 27 for consistency with it.
Two other reads have the same implicit-encoding shape but are not affected in practice, and I have deliberately left them out of scope rather than bundling them:
- `bdk_rsa_key_config.py:59` — a PEM private key, which is base64 and therefore ASCII
- `on_disk_datafeed_id_repository.py:56` — a datafeed id this library writes itself
Happy to raise them separately if you would like them tightened too.
I have a fix and a regression test ready. The test drives the loader in a child interpreter with `PYTHONUTF8=0 LC_ALL=C` so it reproduces on Linux CI as well, and compares `load_from_file` against `load_from_content` as the oracle. Glad to open a PR against this issue — our FINOS CLA is already on file.
Contributor guide
Research direction
Start in symphony/bdk/core/config/loader.py at line 27 and compare load_from_file with load_from_content and the UTF-8 pattern in symphony/bdk/core/service/user/user_service.py:798. Run or add the described regression test under a forced non-UTF-8 locale, and confirm both entry points return identical values for UTF-8 config content.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 90/100