tensorflow / tensorflow/datasets
Invalid UTF8 bytes in default TAGS.txt
@pierrot0 is already working on this.
Since May 13, 2024.
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 1.6k
- Avg merge
- 3h 54m
- Merged PRs (30d)
- 1
Description
Short description
Some of the language tags in the default TAGS.txt cause a UnicodeDecodeError.
Environment information
-
Operating System: Windows 11
-
Python version: 3.10.13
-
tensorflow-datasets/tfds-nightlyversion:tfds-nightly4.9.4.dev202405100044 -
tensorflow/tf-nightlyversion:tensorflow2.10.0 -
Does the issue still exists with the last
tfds-nightlypackage (pip install --upgrade tfds-nightly) ? Yes
Reproduction instructions
Make a toy dataset with tfds new test. Then try to instantiate the builder.
from test.test_dataset_builder import *
b = Builder()
Link to logs
Stack trace here
Expected behavior
The builder to instantiate without error.
Additional context
Deleting lines 73, 79, 126, 128, 156, and 173 in TAGS.txt fixes the problem. These are all language tags.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.