lyst / lyst/lightfm

Using feature vectors of just 1 feature

Open
#593 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
5.1k
Forks
724
PR merge metrics
No merged PRs in 30d

Description

Hello to everyone!
I'm implementing a LightFM model on a TV series dataset. I want to include user features on Binge-watcher users and item features on binge-worthy series: basically I have two matrices [number of users x 1] and [number of items x 1], with 1 if a user is a binge-watcher and 0 if it is not (same holds for bingeworthy series).
If I try to convert these arrays into csr matrix and feed them into the model with
` self.lightFM_model = self.lightFM_model.fit(self.URM_train,
epochs=self.epochs,
num_threads=self.num_threads,
user_features=self.users_features,
item_features=self.items_features)
`
I'd get "ValueError: Not all estimated parameters are finite, your model may have diverged. Try decreasing the learning rate or normalising feature values and sample weights". I read in a [stack overflow question](https://stackoverflow.com/questions/55125253/lightfm-error-not-all-estimated-parameters-are-finite-your-model-may-have-dive) that the problem might be related to user or items having no feature set to 1. In my case, since 3K users over 42K total users are binge-watcher, most of them (39K) have a feature value that's 0, so, before converting those two arrays to CSR, I stacked them with an identity matrix so that I have no row with all zeros. For example, if I had just 3 users (and just the second one is a binge-watcher), I'd get a feature matrix like this one before converting it into CSR:
`
[[0],
[1],
[0]]
=>
[[0 1 0 0],
[1 0 1 0],
[0 0 0 1]]
`
Problem's that by using this approach my recommender performance worsen by a thousand times wrt the same LightFM model without the binge-watching / binge-worthy feature vectors, becoming worse than a random recommender. I'm assuming there's something wrong with stacking that identity matrix to my feature vector, making it basically garbage.
Can I train a lightFM model with just 1 feature for user and 1 feature for items or is it impossible?
Thanks in advance for your help!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The report names no repository files or tests to inspect and describes a LightFM training question rather than a defined code change. Start by reproducing the reported ValueError with the one-feature and identity-augmented matrices, then determine whether the issue requires a documented usage answer or a reproducible project fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.