finos / finos/symphony-bdk-python

BdkConfigLoader.load_from_file decodes the config with the platform codec, silently corrupting non-ASCII values

Open Beginner friendly
#395 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
37
Forks
44
Avg merge
1d 2h
Merged PRs (30d)
11

Description

## Describe the bug

`BdkConfigLoader.load_from_file` reads the config with no encoding, so Python decodes it with `locale.getpreferredencoding(False)` — the ANSI codepage on Windows:

```python
# symphony/bdk/core/config/loader.py:27
config_content = config_path.read_text()
```

YAML and JSON are both specified as UTF-8, so any non-ASCII value in a config is decoded with the wrong codec on a non-UTF-8 host.

**The failure is silent.** On a codepage that happens to map the bytes there is no exception at all: `load_from_file` returns a `BdkConfig` whose values are mojibake, the bot starts, and it fails later against the proxy or pod with an error that points nowhere near the config.

## To Reproduce

A config carrying a non-ASCII proxy credential, written to disk as UTF-8 bytes, loaded on Windows with a cp950 console:

```
on disk : password: "sésame-café" (b'password: "s\xc3\xa9same-caf\xc3\xa9"')
loaded : password='s矇same-caf矇' ← wrong, and nothing reported it
username: 'caf矇-user'
```

The sibling entry point on the same content is correct, which isolates it to the read:

```
load_from_content(...) → password='sésame-café' ✓
```

Non-ASCII in a proxy password is not exotic, and `defaultHeaders` and the various name fields are all free text.

## Expected behavior

`load_from_file` and `load_from_content` agree for the same config, on every host.

## Environment

Windows 10, Python 3.11, `locale.getpreferredencoding(False)` = `cp950`, repository at `21a0370` (tip of `main`).

Not specific to CJK locales: cp1252 is the default on en-US Windows and mangles the same bytes just as quietly. A `LC_ALL=C` host gets ASCII and raises instead.

## Additional context

The repository already has the right pattern elsewhere — `symphony/bdk/core/service/user/user_service.py:798` opens with `encoding="utf-8"` — so the fix is one argument on line 27 for consistency with it.

Two other reads have the same implicit-encoding shape but are not affected in practice, and I have deliberately left them out of scope rather than bundling them:

- `bdk_rsa_key_config.py:59` — a PEM private key, which is base64 and therefore ASCII
- `on_disk_datafeed_id_repository.py:56` — a datafeed id this library writes itself

Happy to raise them separately if you would like them tightened too.

I have a fix and a regression test ready. The test drives the loader in a child interpreter with `PYTHONUTF8=0 LC_ALL=C` so it reproduces on Linux CI as well, and compares `load_from_file` against `load_from_content` as the oracle. Glad to open a PR against this issue — our FINOS CLA is already on file.

Contributor guide

Open the contributing guide

Research direction

Start in symphony/bdk/core/config/loader.py at line 27 and compare load_from_file with load_from_content and the UTF-8 pattern in symphony/bdk/core/service/user/user_service.py:798. Run or add the described regression test under a forced non-UTF-8 locale, and confirm both entry points return identical values for UTF-8 config content.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
90/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.