clips / clips/pattern

Issue with storing a parsetree

Open
#150 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.9k
Forks
1.6k
PR merge metrics
No merged PRs in 30d

Description

Hello !

Painful issue right here.

I have built a parse tree quite simply on a large volume of texts like :

s = parsetree(string, relations=True, lemmata=True)

with s being of the type : <class 'pattern.text.tree.Text'>

If I do a pprint(s) I get a very clean data structure like :


             WORD   TAG    CHUNK   ROLE   ID     PNP    LEMMA               

@MAP_Information   NN     NP      -      -      -      @map_information   
               et   CC     -       -      -      -      et                  
          pendant   IN     PP      -      -      PNP    pendant             
               ce   PRP    NP      SBJ    1      PNP    ce                  
            temps   NN     NP ^    SBJ    1      PNP    temps               

Which is want I want ! So I would like to store the exact same data structure to any file like a DataFrame, a CSV, plain text... for better readability and user-friendliness.

However this is not possible since it all the outputs belong to <class 'pattern.text.tree.Text'> or <class 'pattern.text.tree.Sentence'> etc... classes, which make them painful to use.

For example I cannot :

encode my object to utf-8 for exporting :

s = s.encode('utf8')

AttributeError: 'Text' object has no attribute 'encode'

Export my object as a text file :

with open("D:\\Testpprint4.txt", 'w') as export :
    for sentence in s :
        export.write(sentence)

TypeError: expected a string or other character buffer object

Put my object to a dataframe :

`pd = pandas.DataFrame(sentence)

PandasError : DataFrame constructor not properly called!`

Or use pprint.pformat to store it as a csv since it does not deal with utf-8.

Would you have a solution ?

Thanks,

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no repository files or tests. Start with the parsetree(...) call and the Text and Sentence classes, then inspect how their contents are represented and whether an existing export interface is documented. Done would require a maintainer-defined output format and corresponding verification, which the issue does not specify.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.