Sort sentences by topic: using LDA/PCA to remix novels/twitter.

Topic modeling is easy with Python and Gensim: http://radimrehurek.com/gensim/index.html

For example, I could take a novel and convert its sentences to bag-of-words vectors.
[See this tutorial http://radimrehurek.com/gensim/tut1.html]

Then I could perform PCA on all the vectors, extract the principle component, and sort by it. This would arrange sentences along a topic path. Sentences talking about the same stuff clump together, and as you read it gradually moves onto talking about different stuff.

I think it'd be interesting for the twitter novels to sort their tweets this way.

Or I could group sentences together by topic, and name each chapter by the most prominent phrases/words in each topic.

Or I could leverage WordNet or ConceptNet to do a more clever topic-modeler. (This gives me knowledge, for example, that "dog" and "cat" are both "animals", and so sentences about dogs and cats may be grouped together in a topic). One approach: for each word in each bag-of-words vector, add its hypernym or/and its hypernym's hypernym to the bag.

Topic modeling with LDA: http://en.wikipedia.org/wiki/Latent_Dirichlet_allocation
Dimensionality reduction with PCA: http://en.wikipedia.org/wiki/Principal_component_analysis

``` python
# PCA is easy in python with sklearn:
from sklearn import decomposition
data = [[1,2,4,5],[7,2,1,2],[5,7,8,2]]
pca = decomposition.PCA(n_components=1) # reduce to one dimension
pca.fit(data)
data_reduced_to_one_dimension = pca.transform(data)
```

The issue names no project files, tests, or entry points; start by reviewing the proposed Gensim bag-of-words/LDA workflow and the scikit-learn PCA example. Before implementation, define whether the goal is sentence sorting, topic grouping and chapter naming, or WordNet/ConceptNet enrichment, along with a testable completion criterion.

Written by the indexing model from the issue text.