GothenburgBitFactory / GothenburgBitFactory/tasklib

Unicode error

Open
#82 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
159
Forks
29
PR merge metrics
No merged PRs in 30d

Description

I have a task generated by bugwarrior from a github issue. The `githubbody` UDA contains this line:

```
UDA githubbody: blah blah blah blhín blah blah blah
```

Also, another github issue, that contains this: 👏🏽, is being transformed into:

```
Annotation: 2020-07-24 14:00:31 -- @pontusj - Well done! That was not at all obvious! ꃭ辱ꃭ붿
```

While using the taskwiki plugin in vim, I got an error:

```
Error detected while processing function provider#python3#Call:
line 18:
Error invoking 'python_execute_file' on channel 3 (python3-script-host):
Traceback (most recent call last):
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/main.py", line 666, in
WholeBuffer.update_from_tw()
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/errors.py", line 26, in wrapped_function
original_function(*args, **kwargs)
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/decorators.py", line 10, in wrapped_function
original_function(*args, **kwargs)
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/main.py", line 44, in update_from_tw
c.evaluate_viewports()
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/cache.py", line 249, in evaluate_viewports
port.sync_with_taskwarrior()
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/viewport.py", line 343, in sync_with_taskwarrior
to_add, to_del = self.get_tasks_to_add_and_del()
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/viewport.py", line 317, in get_tasks_to_add_and_del
matching_tasks = self.matching_tasks
File "/home/nikos/.local/share/nvim/plugged/taskwiki/taskwiki/viewport.py", line 294, in matching_tasks
return set(
File "/usr/lib/python3.8/site-packages/tasklib/task.py", line 476, in __iter__
self._result_cache = self._execute()
File "/usr/lib/python3.8/site-packages/tasklib/task.py", line 508, in _execute
return self.backend.filter_tasks(self.filter_obj)
File "/usr/lib/python3.8/site-packages/tasklib/backends.py", line 334, in filter_tasks
for line in self.execute_command(args):
File "/usr/lib/python3.8/site-packages/tasklib/backends.py", line 290, in execute_command
stdout, stderr = [x.decode('utf-8') for x in p.communicate()]
File "/usr/lib/python3.8/site-packages/tasklib/backends.py", line 290, in
stdout, stderr = [x.decode('utf-8') for x in p.communicate()]
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 78938: invalid continuation byte
```

Using `task edit` I managed to pinpoint the problem to these particular taskwarrior tasks. I used vim with `task edit` and searched for `\%xed`. The character `í` was highlighted. That's how I assumed the first task was to blame. Removing the strange character did not solve the issue. To find the second task, I manually loaded each task in taskwiki. Removing the strange characters at the end, solved the issue. So perhaps that was the issue all along.

Perhaps, it is the responsibility of bugwarrior to filter such characters, but given the broad range of sources that can go into taskwarrior, perhaps it would be simpler to fix such an issue in tasklib.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.