allenai / allenai/nlpstack

Tokenizer fails on input

Open
#127 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
43
Forks
8
PR merge metrics
No merged PRs in 30d

Description

Kevin says:

> Turns out that line has a weird ^F character in it:
> Animals with webbed feet are better able to move through water .

This kills the tokenizer.

(But the ^F character might have gotten obliterated by copy and paste, so to report, put it back in manually.)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.