NVIDIA-Merlin / NVIDIA-Merlin/Merlin
[RMP] Define Merlin Models high level API and align wrapper classes for traditional models with it
@EvenOldridge is already working on this.
Since Aug 14, 2022.
- Dominant language
- Python
- Stars
- 907
- Forks
- 129
- PR merge metrics
- No merged PRs in 30d
Description
Problem:
At the moment, different models have different APIs. This is unintuitive for the user. For instance, the XGBoost wrapper accepts a schema and target_columns, where implicit and LightFM do not. This makes it challenging for a user to create a unified workflow for experimentation and has other undesirable effects. For instance, it is not obvious that schema on a Merlin Dataset can be assigned to as follow train.schema = train.schema.remove_col('rating') which is a workaround to not being able to pass the schema into the constructor separately, and without this piece of information, one might be led down the path of reconstructing their dataset, which is very unwieldy.
Additionally, some functionality is missing across the wrapper classes that would be very helpful to working on an ML model (in particular, the save and load functionality)
Goal:
To make the high-level Merlin Models API more approachable to the users, create a better experience, and foster adoption.
Constraints:
Some preliminary discussion on what the API might look like has been had, but it would be useful to consider the suggestions with feedback (or go ahead) from the entire team, to capture the collective experience.
Starting Point:
Discussions on this have been had on Slack
A doc that captures some of the suggestions and feedback can be found here
Here is what the API might look like:
class MerlinModel():
def __init__(schema)
def fit(train)
def evaluate(dataset)
def predict(dataset)
def save(path)
def load(path)
The overarching idea is to use schema for everything. The workflow would be as follows. The user creates a Merlin Dataset and then tells the model what columns to use for training, etc, by passing in a modified schema as necessary. Columns can be removed from the schema with the without method, etc.
This requires a bit of functionality on the Schema itself to be added -- ease of tagging columns (and removing tags from columns).
It is not clear to me where the load method should reside. It might create a better experience where the model could be loaded without having to instantiate an object, via potentially making the load method a class method.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.