bndr / bndr/pipreqs

UnicodeDecodeError: 'gbk' codec can't decode byte 0x8c in position 358: illegal multibyte sequence

Open
#241 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.5k
Forks
424
PR merge metrics
No merged PRs in 30d

Description

# What happened
I used pipreqs to freeze package requirements for one of my projects by running 'pipreqs' in the project directory, but it caused this error: UnicodeDecodeError: 'gbk' codec can't decode byte 0x8c in position 358: illegal multibyte sequence.

# What I expected
It should work without causing any problem.

# Full traceback
Traceback (most recent call last):
File "c:\users\myAccount\anaconda3\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\myAccount\anaconda3\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\myAccount\anaconda3\Scripts\pipreqs.exe\__main__.py", line 7, in
File "c:\users\myAccount\anaconda3\lib\site-packages\pipreqs\pipreqs.py", line 470, in main
init(args)
File "c:\users\myAccount\anaconda3\lib\site-packages\pipreqs\pipreqs.py", line 406, in init
candidates = get_all_imports(input_path,
File "c:\users\myAccount\anaconda3\lib\site-packages\pipreqs\pipreqs.py", line 122, in get_all_imports
contents = f.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0x8c in position 358: illegal multibyte sequence

# What I did then
I went to c:\users\myAccount\anaconda3\lib\site-packages\pipreqs\pipreqs.py and modified lines near line 122(contents = f.read()), namely, the lines concerning file encoding:
original:
```python
with open_func(file_name, "r", encoding=encoding) as f:
contents = f.read()

```
after adding try/except:
```python
try:
with open_func(file_name, "r", encoding=encoding) as f:
contents = f.read()
except:
with open_func(file_name, "r", encoding='utf-8') as f:
contents = f.read()
```
Then it worked without problems. Albeit a temp fix.

# What need to be done
The assumption about the encoding of the source code file need to be revised?

Contributor guide

Open the contributing guide

Research direction

Start in pipreqs.py, especially get_all_imports around line 122, and reproduce the pipreqs command on a project containing a source file that triggers the reported GBK decoding error. Trace how the encoding argument reaches open_func and define behavior for files that cannot be decoded with the current assumption. Done means pipreqs completes without this UnicodeDecodeError and generates the requirements output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.