adafruit / adafruit/circup

UnicodeDecodeError when reading ANSI files

Closed
#222 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
172
Forks
41
Avg merge
1d 20h
Merged PRs (30d)
1

Description

If circup tries to find the version information in a `.py` file that has ANSI encoded characters like `‘’`, it throws [here](https://github.com/adafruit/circup/blob/main/circup/shared.py#L120C1-L121C41):

```
Found device at D:\, running CircuitPython 9.1.0-beta.3.
Searching for dependencies for: ['Adafruit_CircuitPython_esp32spi']
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "C:\dev\adafruit\envs\circup\Scripts\circup.exe\__main__.py", line 7, in
File "C:\dev\adafruit\envs\circup\Lib\site-packages\click\core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\click\decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\circup\commands.py", line 347, in install
device_modules = ctx.obj["backend"].get_device_versions()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\circup\backends.py", line 73, in get_device_versions
return self.get_modules(os.path.join(self.device_location, self.LIB_DIR_PATH))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\circup\backends.py", line 63, in get_modules
return self._get_modules(device_url)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\circup\backends.py", line 806, in _get_modules
return _get_modules_file(device_lib_path, self.logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\circup\shared.py", line 87, in _get_modules_file
metadata = extract_metadata(source, logger)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\adafruit\envs\circup\Lib\site-packages\circup\shared.py", line 121, in extract_metadata
content = source_file.read()
^^^^^^^^^^^^^^^^^^
File "", line 322, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x91 in position 5007: invalid start byte
```

This could be changed from:
```py
with open(path, "r", encoding="utf-8") as source_file:
content = source_file.read()
```

To:
```py
try:
with open(path, "r", encoding="utf-8") as source_file:
content = source_file.read()
except UnicodeDecodeError:
with open("a.txt", "r") as source_file:
content = source_file.read()
```

Contributor guide

Open the contributing guide

Research direction

Start in circup/shared.py at extract_metadata and reproduce the failure with a Python file containing ANSI-encoded characters. Check how version metadata is extracted and verify that reading such a file no longer raises UnicodeDecodeError while the existing metadata lookup still completes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.