rasbt / rasbt/python-machine-learning-book
Chapter 8: Shuffling the DataFrame in newer versions of pandas
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 12.6k
- Forks
- 4.4k
- PR merge metrics
- No merged PRs in 30d
Description
Just a note in case it's helpful to anyone else - I seemed to be getting 100% accuracy with the on-line sentiment analysis classifier (pages 246-246), but it turned out to be because the code used to shuffle the dataset before exporting it to CSV on page 235 hadn't worked.
In the version of pandas I'm using (0.23.4), it looks like df.index.values is needed in order to get the indexes of a DataFrame as a list. So, this:
df = df.reindex(np.random.permutation(df.index))
now needs to be this:
df = df.reindex(np.random.permutation(df.index.values))
Hope that helps someone!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the DataFrame shuffle code on page 235 and check how it behaves with pandas 0.23.4 before the dataset is exported to CSV. Verify that the resulting CSV is actually shuffled and that the sentiment-analysis example on pages 246-246 no longer reports misleading accuracy from ordered data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100