Can't import the tokenizer
- Dominant language
- Jupyter Notebook
- Stars
- 113
- Forks
- 22
- PR merge metrics
- No merged PRs in 30d
Description
when I tried to launch the code
```python
import tkseem as tk
import csv
# initialize the tokenizer
tokenizer = tk.WordTokenizer()
tokenizer.train('data.txt')
# open the csv file and read the text data
with open('../uploads/tweets-ar.csv', mode='r', encoding='utf-8') as csv_file:
csv_reader = csv.reader(csv_file)
for row in csv_reader:
text = row[0]
# tokenize the text using tkseem
tokens = tokenizer.tokenize(text)
print(tokens)
```
i got this error
```shell
File "pathtomycode\tokenize.py", line 5, in
tokenizer = tk.WordTokenizer()
AttributeError: partially initialized module 'tkseem' has no attribute 'WordTokenizer' (most likely due to a circular import)
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the import from the reported tokenize.py example and inspect how tkseem exposes WordTokenizer. Check the import path and package entry point first, then verify that the example can initialize the tokenizer, train on data.txt, and reach tokenization of the CSV input without the reported AttributeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100