Printing Word & Chunk objects that contain Unicode
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.9k
- Forks
- 1.6k
- PR merge metrics
- No merged PRs in 30d
Description
Hi! I'm working with texts containing Unicode in Python 2.7.9.
I noticed that using "print" on Word objects and Chunk objects seem to behave differently. In short, printing a Word object containing an Unicode character works, but printing a Chunk containing an Unicode character. For example:
Printing: sentence.string
The French for cake is gâteau .
Printing: sentence.words
Word(u'The/DT')
Word(u'French/NNP')
Word(u'for/IN')
Word(u'cake/NN')
Word(u'is/VBZ')
Word(u'g\xe2teau/NN')
Word(u'./.')
Printing: sentence.chunks
Chunk('The French/NP')
Chunk('for/PP')
Chunk('cake/NP-SBJ-1')
Chunk('is/VP-1')
---------------------------------------------------------------------------
UnicodeEncodeError Traceback (most recent call last)
<ipython-input-21-68762238b21e> in <module>()
15 print "\nPrinting: sentence.chunks"
16 for chunk in sentence.chunks:
---> 17 print chunk
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe2' in position 8: ordinal not in range(128)
Maybe using "print" isn't the best way to output Words and Chunks, but it's convenient when experimenting. Is there a reason why Word and Chunk don't behave consistently regarding Unicode?
Thanks :)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the Word and Chunk printing examples in Python 2.7.9 and inspect how each object is converted for output. Done means printing a Chunk containing a Unicode character no longer raises UnicodeEncodeError and behaves consistently with Word.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- developer-experience
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100