lyst / lyst/lightfm

ValueError: The user feature matrix specifies more features than there are estimated feature embeddings: 19400 vs 81728.

Open
#639 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.1k
Forks
724
PR merge metrics
No merged PRs in 30d

Description

I have the following datasets:

Users:
10.000 Rows.
Features: User-Id, name, age, los, ou, gender, skills, language, grade, career interests

Trainings:
Training-Id, training name, main skill

Trainings Taken
User-Id, Training-Id, TrainingTaken
TrainingTaken will be a 10 when the user took the training, otherwise it wont appear in the dataset

The idea is to make a recommneder for trainings :)

I used this helper class for the matrices. https://github.com/Med-ELOMARI/LightFM-Dataset-Helper

``from lightfm_dataset_helper.lightfm_dataset_helper import DatasetHelper`

I defined the feature columns for user and trainings.

items_column = "Training-Id"
user_column = "User-Id"
ratings_column = "TrainingTaken"

items_feature_columns = [
    "training name",
    "main skill"
]

user_features_columns = ["name","age","los","ou", "gender", "skills", "language", "grade", "career interests"]

Then I build the matrices
dataset_helper_instance = DatasetHelper(
    users_dataframe=usersdf,
    items_dataframe=trainingsdf,
    interactions_dataframe=trainingstakendf,
    item_id_column=items_column,
    items_feature_columns=items_feature_columns,
    user_id_column=user_column,
    user_features_columns=user_features_columns,
    interaction_column=ratings_column,
    clean_unknown_interactions=True,
)
dataset_helper_instance.routine()

Then I train:

from lightfm import LightFM
from lightfm.cross_validation import random_train_test_split
(train, test) = random_train_test_split(interactions=dataset_helper_instance.interactions, test_percentage=0.2)

model = LightFM(loss='warp')

model.fit(
    interactions=dataset_helper_instance.interactions,
    sample_weight=dataset_helper_instance.weights,
    item_features=dataset_helper_instance.item_features_list,
    user_features=dataset_helper_instance.user_features_list,
    verbose=True,
    epochs=20,
    num_threads=20,
)

then I try to use the predict

import numpy as np
from lightfm.data import Dataset
#predict existing users
scores = model.predict(user_ids=81727, item_ids=[1])
print(scores)

However I am getting this error:
ValueError: The user feature matrix specifies more features than there are estimated feature embeddings: 19400 vs 81728.

what could be wrong?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting the DatasetHelper-generated interactions, weights, item_features_list, and user_features_list alongside the LightFM model dimensions, then trace the model.predict call for existing users. A useful outcome would be a minimal reproduction that identifies whether the mismatch is in the helper-generated matrices or the prediction inputs, with the relevant documentation or validation updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.