Possible ByteVocabulary Bug
- Dominant language
- Python
- Stars
- 596
- Forks
- 60
- Avg merge
- 19h 23m
- Merged PRs (30d)
- 1
Description
```
464 def _decode_tf(self, ids):
465 """Decode in TensorFlow.
466
467 Args:
468 ids: a 1d tf.Tensor with dtype tf.int32
469 Returns:
470 a tf Scalar with dtype tf.string
471 """
472 return tf.py_function(func=self.decode, inp=[ids], Tout=tf.string)
```
The param 'ids' passed to _decode_tf above is a 1d tf.Tensor, and on line 472 it is wrapped into a list, but when self.decode is called with the param [ids], it throws the error in the list comprehension on line 100 (shown below):
File "/home/ubuntu/anaconda3/envs/google_t5/lib/python3.7/site-packages/seqio/vocabularies.py", line 100, in
decode
for i in clean_ids
File "/home/ubuntu/anaconda3/envs/google_t5/lib/python3.7/site-packages/seqio/vocabularies.py", line 100, in
for i in clean_ids
File "/home/ubuntu/anaconda3/envs/google_t5/lib/python3.7/site-packages/tensorflow/python/framework/ops.py",
line 1007, in __bool__
return bool(self._numpy())
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```
92 def decode(self, ids: Iterable[int]):
93 """Detokenizes int32 iterable to a string, up through first EOS."""
94 clean_ids = list(ids)
95
96 if self.unk_id is not None:
97 vocab_size = self._base_vocab_size
98 clean_ids = [
99 self.unk_id if i >= vocab_size else i
100 for i in clean_ids
101 ]
102
103 if self.eos_id is not None and self.eos_id in clean_ids:
104 clean_ids = clean_ids[:clean_ids.index(self.eos_id) + 1]
105
106 return self._decode(clean_ids)
```
Contributor guide
Assessment
This issue has not been assessed yet.