allenai / allenai/bilm-tf

UnicodeError when run example: python -m unittest discover tests/

Open
#225 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.6k
Forks
447
PR merge metrics
No merged PRs in 30d

Description

eEncodeError: 'ascii' codec can't encode character '\u0100' in position 6: ordinal not in range(128)

======================================================================
ERROR: test_weighted_layers (test_elmo.TestWeightedLayers)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_elmo.py", line 118, in test_weighted_layers
self._check_weighted_layer(1.0, do_layer_norm=True, use_top_only=False)
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_elmo.py", line 28, in _check_weighted_layer
batcher = Batcher(vocab_file, 50)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 204, in __init__
lm_vocab_file, max_token_length
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 117, in __init__
super(UnicodeCharsVocabulary, self).__init__(filename, **kwargs)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 29, in __init__
for line in f:
File "/research/d2/hrwang/pythonlib/anaconda3/envs/tensorflow-gpu/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 820: ordinal not in range(128)

======================================================================
ERROR: test_weighted_layers_no_norm (test_elmo.TestWeightedLayers)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_elmo.py", line 121, in test_weighted_layers_no_norm
self._check_weighted_layer(1.0, do_layer_norm=False, use_top_only=False)
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_elmo.py", line 28, in _check_weighted_layer
batcher = Batcher(vocab_file, 50)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 204, in __init__
lm_vocab_file, max_token_length
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 117, in __init__
super(UnicodeCharsVocabulary, self).__init__(filename, **kwargs)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 29, in __init__
for line in f:
File "/research/d2/hrwang/pythonlib/anaconda3/envs/tensorflow-gpu/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 820: ordinal not in range(128)

======================================================================
ERROR: test_weighted_layers_top_only (test_elmo.TestWeightedLayers)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_elmo.py", line 124, in test_weighted_layers_top_only
self._check_weighted_layer(None, do_layer_norm=False, use_top_only=True)
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_elmo.py", line 28, in _check_weighted_layer
batcher = Batcher(vocab_file, 50)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 204, in __init__
lm_vocab_file, max_token_length
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 117, in __init__
super(UnicodeCharsVocabulary, self).__init__(filename, **kwargs)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 29, in __init__
for line in f:
File "/research/d2/hrwang/pythonlib/anaconda3/envs/tensorflow-gpu/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 820: ordinal not in range(128)

======================================================================
ERROR: test_bilm (test_model.TestBidirectionalLanguageModel)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_model.py", line 56, in test_bilm
batcher = Batcher(vocab_file, 50)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 204, in __init__
lm_vocab_file, max_token_length
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 117, in __init__
super(UnicodeCharsVocabulary, self).__init__(filename, **kwargs)
File "/research/d2/hrwang/biLM/bilm-tf/bilm/data.py", line 29, in __init__
for line in f:
File "/research/d2/hrwang/pythonlib/anaconda3/envs/tensorflow-gpu/lib/python3.5/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 820: ordinal not in range(128)

======================================================================
ERROR: test_bilm_token (test_model.TestBidirectionalLanguageModelTokenInput)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/research/d2/hrwang/biLM/bilm-tf/tests/test_model.py", line 161, in test_bilm_token
fout.write('\n'.join(all_tokens))
UnicodeEncodeError: 'ascii' codec can't encode character '\u2022' in position 488: ordinal not in range(128)

----------------------------------------------------------------------
Ran 24 tests in 200.365s

FAILED (errors=14)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.