Code silently fails after running model.fit()
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.1k
- Forks
- 724
- PR merge metrics
- No merged PRs in 30d
Description
I am building a recommender system with LightFM. When the line of code model.fit() is executed, the code is immediately stopped. Sometimes, I run the code again and then it works and the model is trained (without having changed any parameters !)
I tried to change some parameters :
- How sparse my matrix is (from 50% to 99.99%)
- The number of epochs
- The number of latent factors
- The loss that I use
I've set the random.seed(42) and the random_state=42.
My interaction list is [(user_id, item_id, score)]
I build my dataset properly, and I don't have any Nan or Inf.
But when i run the code, it sometimes work, and it sometimes don't. Without any error message! I've set the verbose=True :
- When it works i see the Epoch bar going to 100%
- When it does not work, I see the Epoch bar at 0% and the code is exited / stopped.
Do you have any idea why it is happening ?
I am on Windows and using Visual Studio Code with LightFM 1.7
Here is my code :
````
def matrix_factorization(self, interaction_matrix: pd.DataFrame):
dataset = LightfmMatrixFactorizer.build_dataset(interaction_matrix=interaction_matrix)
interaction_list = [
(user_id, item_id, score)
for user_id in interaction_matrix.index
for item_id in interaction_matrix.columns
if (score := interaction_matrix.loc[user_id, item_id]) > self.threshold
]
interactions, weights = dataset.build_interactions(interaction_list)
sparsity = 1 - (interactions.nnz / (interactions.shape[0] * interactions.shape[1]))
print(f"Sparsity: {sparsity:.2%}")
# Initialize LightFM model (e.g., warp method for implicit feedback)
model = LightFM(no_components=self.n_factors, loss=self.loss, random_state=42)
# Train the LightFM model
try:
model.fit(interactions=interactions, sample_weight=weights, epochs=self.epochs, num_threads=self.num_threads, verbose = True)
except Exception as e:
print(f"Error occurred: {e}")
````
Main :
````
lightfm3 = LightfmMatrixFactorizer(score_percentile=70, n_factors=5, loss='bpr', epochs=10, num_threads=1)
im = pd.read_csv("data/interaction_matrix/global_interaction_matrix.csv", index_col=0)
print(im.head(3))
user_factors, item_factors, approx_matrix = lightfm3.matrix_factorization(interaction_matrix=im)
````
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue identifies model.fit() as the failing entry point and provides a Windows reproduction context, but no repository file or test is named. Start by reproducing the call with the supplied parameters and inspect how the failure exits; done means the intermittent stop has a verified cause and a documented or tested resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100