Plans for running in production / near real-time

Nobody has claimed this yet.

So I've read over the many great "issues" here and the comments seem very helpful.

I'm interested in running LightFM in production / near real-time. From what I've gathered, the best way to do this is:

- Prep/fit my initial data using Dataset()
- Subsequent additional data will call Dataset.fit_partial()
- Fit the initial model using LightFM.fit()
- Subsequent additional data will be fit with LightFM.fit_partial()

What I am planning is to have a Python script that continuously runs and looks for new data in a queue (probably using Redis), fits new data, and produces/caches recommendations. From what I gather reading comments here, I can save the model using pickle so I don't have to re-train in entirety later in the event my script fails, etc; I could just open the pickle file and resume.

My questions:

1. When I resume training using the pickled LightFM model: Should I recreate the entire Dataset(), or just start where I left off with data that wasn't processed, and call fit_partial?

2. Should I pickle the LightFM model every so often, as a "save state"? For example, every hour (arbitrary time value)

3. I'm using user/item features. Should I call build_user|item_features() after dataset.fit_partial()? If yes, should it contain all user/item features, or only for the new user(s)/item(s)?

4. When using user/item features for prediction, do I pass ALL user/item features (entire dataset) or only the features for the user(s)/item(s) I am predicting for?

5. Are user/item interactions cumulative, or do they overwrite each other? For example, if a user interacts with an item multiple times with differing weights, is this handled in the model?

Thanks so much, everyone!

Start by reading the Dataset, fit_partial, feature-building, prediction, and model-serialization APIs mentioned in the issue. Verify each question against current behavior and examples; done means a concise production and near-real-time guide answers all five points. No repository file or test is named in the payload.

Written by the indexing model from the issue text.