ValueError: The user feature matrix specifies more features than there are estimated feature embeddings: 19400 vs 81728.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.1k
- Forks
- 724
- PR merge metrics
- No merged PRs in 30d
Description
I have the following datasets:
Users:
10.000 Rows.
Features: User-Id, name, age, los, ou, gender, skills, language, grade, career interests
Trainings:
Training-Id, training name, main skill
Trainings Taken
User-Id, Training-Id, TrainingTaken
TrainingTaken will be a 10 when the user took the training, otherwise it wont appear in the dataset
The idea is to make a recommneder for trainings :)
I used this helper class for the matrices. https://github.com/Med-ELOMARI/LightFM-Dataset-Helper
``from lightfm_dataset_helper.lightfm_dataset_helper import DatasetHelper`
I defined the feature columns for user and trainings.
items_column = "Training-Id"
user_column = "User-Id"
ratings_column = "TrainingTaken"
items_feature_columns = [
"training name",
"main skill"
]
user_features_columns = ["name","age","los","ou", "gender", "skills", "language", "grade", "career interests"]
Then I build the matrices
dataset_helper_instance = DatasetHelper(
users_dataframe=usersdf,
items_dataframe=trainingsdf,
interactions_dataframe=trainingstakendf,
item_id_column=items_column,
items_feature_columns=items_feature_columns,
user_id_column=user_column,
user_features_columns=user_features_columns,
interaction_column=ratings_column,
clean_unknown_interactions=True,
)
dataset_helper_instance.routine()
Then I train:
from lightfm import LightFM
from lightfm.cross_validation import random_train_test_split
(train, test) = random_train_test_split(interactions=dataset_helper_instance.interactions, test_percentage=0.2)
model = LightFM(loss='warp')
model.fit(
interactions=dataset_helper_instance.interactions,
sample_weight=dataset_helper_instance.weights,
item_features=dataset_helper_instance.item_features_list,
user_features=dataset_helper_instance.user_features_list,
verbose=True,
epochs=20,
num_threads=20,
)
then I try to use the predict
import numpy as np
from lightfm.data import Dataset
#predict existing users
scores = model.predict(user_ids=81727, item_ids=[1])
print(scores)
However I am getting this error:
ValueError: The user feature matrix specifies more features than there are estimated feature embeddings: 19400 vs 81728.
what could be wrong?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the DatasetHelper-generated interactions, weights, item_features_list, and user_features_list alongside the LightFM model dimensions, then trace the model.predict call for existing users. A useful outcome would be a minimal reproduction that identifies whether the mismatch is in the helper-generated matrices or the prediction inputs, with the relevant documentation or validation updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100