tensorflow / tensorflow/models

UnicodeDecodeError:'utf-8' codec can't decode byte....

Open
#8,655 1 comment 0 reactions 3 assignees View on GitHub

@pkulzc is already working on this.

Since Jun 10, 2020.

models:research:odapi type:bug
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

Prerequisites

Please answer the following questions for yourself before submitting an issue.

  • I am using the latest TensorFlow Model Garden release and TensorFlow 2.
  • I am reporting the issue to the correct repository. (Model Garden official or research directory)
  • I checked to make sure that this issue has not already been filed.

1. The entire URL of the file you are using

https://github.com/tensorflow/models/tree/master/research/...

2. Describe the bug

When I tried to create pet TFRecord according the instruction, I met the following error:
Traceback (most recent call last):
File "object_detection\dataset_tools\create_pet_tf_record.py", line 319, in

tf.app.run()
File "C:\Users\HXN\AppData\Local\Programs\Python\Python37\lib\site-packages\te
nsorflow\python\platform\app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "C:\Users\HXN\AppData\Local\Programs\Python\Python37\lib\site-packages\ab
sl\app.py", line 299, in run
_run_main(main, args)
File "C:\Users\HXN\AppData\Local\Programs\Python\Python37\lib\site-packages\ab
sl\app.py", line 250, in _run_main
sys.exit(main(argv))
File "object_detection\dataset_tools\create_pet_tf_record.py", line 278, in
main
examples_list = dataset_util.read_examples_list(examples_path)
File "e:\tensorflow\models\research\object_detection\utils\dataset_util.py", l
ine 63, in read_examples_list
lines = fid.readlines()
File "C:\Users\HXN\AppData\Local\Programs\Python\Python37\lib\site-packages\te
nsorflow\python\lib\io\file_io.py", line 174, in readlines
self._preread_check()
File "C:\Users\HXN\AppData\Local\Programs\Python\Python37\lib\site-packages\te
nsorflow\python\lib\io\file_io.py", line 79, in _preread_check
self.__name, 1024 * 512)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 85: invalid
continuation byte

And someone said it must be a version bug of Python and suggested to revise below (in dataset_util.py):
with tf.gfile.GFile(path) as fid:
lines = fid.readlines()
to:
with tf.gfile.GFile(path,'rb') as fid:
lines = fid.readlines()

BUT it can't work and report the same error.
Who can help me ?

3. Steps to reproduce

Steps to reproduce the behavior.

4. Expected behavior

A clear and concise description of what you expected to happen.

5. Additional context

Include any logs that would be helpful to diagnose the problem.

6. System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04):Win 7
  • Mobile device name if the issue happens on a mobile device:
  • TensorFlow installed from (source or binary):binary
  • TensorFlow version (use command below):2.2.0
  • Python version:3.7.6
  • Bazel version (if compiling from source):
  • GCC/Compiler version (if compiling from source):
  • CUDA/cuDNN version:
  • GPU model and memory:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.