bndr / bndr/pipreqs

pipreqs --force fails with UnicodeDecodeError when scanning non-UTF-8 Python files

Open Beginner friendly
#551 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.5k
Forks
424
PR merge metrics
No merged PRs in 30d

Description

(.venv) sys-317@DESKTOP-64SE9IT:/mnt/e/mamba_trial$ pipreqs
INFO: Not scanning for jupyter notebooks.
Traceback (most recent call last):
File "/mnt/e/mamba_trial/.venv/bin/pipreqs", line 7, in
sys.exit(main())
File "/mnt/e/mamba_trial/.venv/lib/python3.10/site-packages/pipreqs/pipreqs.py", line 609, in main
init(args)
File "/mnt/e/mamba_trial/.venv/lib/python3.10/site-packages/pipreqs/pipreqs.py", line 533, in init
candidates = get_all_imports(
File "/mnt/e/mamba_trial/.venv/lib/python3.10/site-packages/pipreqs/pipreqs.py", line 136, in get_all_imports
contents = read_file_content(file_name, encoding)
File "/mnt/e/mamba_trial/.venv/lib/python3.10/site-packages/pipreqs/pipreqs.py", line 181, in read_file_content
contents = f.read()
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb1 in position 81: invalid start byte

Contributor guide

Open the contributing guide

Research direction

The error occurs in pipreqs/pipreqs.py at line 181 in read_file_content. Start by examining that function to see how it reads files and handles encoding. The issue is likely that the file is being read with a default UTF-8 encoding when it contains non-UTF-8 characters. Look for where encoding is set or passed. Check if there's a way to detect file encoding or fallback to a different encoding. Run pipreqs on a test directory with a non-UTF-8 Python file to reproduce the error. The fix should ensure the function can handle various encodings gracefully.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, tooling
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.