Using feature vectors of just 1 feature
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.1k
- Forks
- 724
- PR merge metrics
- No merged PRs in 30d
Description
Hello to everyone!
I'm implementing a LightFM model on a TV series dataset. I want to include user features on Binge-watcher users and item features on binge-worthy series: basically I have two matrices [number of users x 1] and [number of items x 1], with 1 if a user is a binge-watcher and 0 if it is not (same holds for bingeworthy series).
If I try to convert these arrays into csr matrix and feed them into the model with
` self.lightFM_model = self.lightFM_model.fit(self.URM_train,
epochs=self.epochs,
num_threads=self.num_threads,
user_features=self.users_features,
item_features=self.items_features)
`
I'd get "ValueError: Not all estimated parameters are finite, your model may have diverged. Try decreasing the learning rate or normalising feature values and sample weights". I read in a [stack overflow question](https://stackoverflow.com/questions/55125253/lightfm-error-not-all-estimated-parameters-are-finite-your-model-may-have-dive) that the problem might be related to user or items having no feature set to 1. In my case, since 3K users over 42K total users are binge-watcher, most of them (39K) have a feature value that's 0, so, before converting those two arrays to CSR, I stacked them with an identity matrix so that I have no row with all zeros. For example, if I had just 3 users (and just the second one is a binge-watcher), I'd get a feature matrix like this one before converting it into CSR:
`
[[0],
[1],
[0]]
=>
[[0 1 0 0],
[1 0 1 0],
[0 0 0 1]]
`
Problem's that by using this approach my recommender performance worsen by a thousand times wrt the same LightFM model without the binge-watching / binge-worthy feature vectors, becoming worse than a random recommender. I'm assuming there's something wrong with stacking that identity matrix to my feature vector, making it basically garbage.
Can I train a lightFM model with just 1 feature for user and 1 feature for items or is it impossible?
Thanks in advance for your help!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The report names no repository files or tests to inspect and describes a LightFM training question rather than a defined code change. Start by reproducing the reported ValueError with the one-feature and identity-augmented matrices, then determine whether the issue requires a documented usage answer or a reproducible project fix.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100