lyst / lyst/lightfm

Code silently fails after running model.fit()

Open
#720 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.1k
Forks
724
PR merge metrics
No merged PRs in 30d

Description

I am building a recommender system with LightFM. When the line of code model.fit() is executed, the code is immediately stopped. Sometimes, I run the code again and then it works and the model is trained (without having changed any parameters !)

I tried to change some parameters :
- How sparse my matrix is (from 50% to 99.99%)
- The number of epochs
- The number of latent factors
- The loss that I use

I've set the random.seed(42) and the random_state=42.

My interaction list is [(user_id, item_id, score)]
I build my dataset properly, and I don't have any Nan or Inf.

But when i run the code, it sometimes work, and it sometimes don't. Without any error message! I've set the verbose=True :
- When it works i see the Epoch bar going to 100%
- When it does not work, I see the Epoch bar at 0% and the code is exited / stopped.

Do you have any idea why it is happening ?

I am on Windows and using Visual Studio Code with LightFM 1.7

Here is my code :

````
def matrix_factorization(self, interaction_matrix: pd.DataFrame):

dataset = LightfmMatrixFactorizer.build_dataset(interaction_matrix=interaction_matrix)

interaction_list = [
(user_id, item_id, score)
for user_id in interaction_matrix.index
for item_id in interaction_matrix.columns
if (score := interaction_matrix.loc[user_id, item_id]) > self.threshold
]

interactions, weights = dataset.build_interactions(interaction_list)

sparsity = 1 - (interactions.nnz / (interactions.shape[0] * interactions.shape[1]))
print(f"Sparsity: {sparsity:.2%}")

# Initialize LightFM model (e.g., warp method for implicit feedback)
model = LightFM(no_components=self.n_factors, loss=self.loss, random_state=42)

# Train the LightFM model
try:
model.fit(interactions=interactions, sample_weight=weights, epochs=self.epochs, num_threads=self.num_threads, verbose = True)
except Exception as e:
print(f"Error occurred: {e}")
````
Main :
````
lightfm3 = LightfmMatrixFactorizer(score_percentile=70, n_factors=5, loss='bpr', epochs=10, num_threads=1)

im = pd.read_csv("data/interaction_matrix/global_interaction_matrix.csv", index_col=0)

print(im.head(3))

user_factors, item_factors, approx_matrix = lightfm3.matrix_factorization(interaction_matrix=im)
````

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue identifies model.fit() as the failing entry point and provides a Windows reproduction context, but no repository file or test is named. Start by reproducing the call with the supplied parameters and inspect how the failure exits; done means the intermittent stop has a verified cause and a documented or tested resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.