NVIDIA-Merlin / NVIDIA-Merlin/Merlin

[RMP] Establish a metadata standard for serializing information about Merlin components

Open
#489 2 comments 0 reactions 1 assignee View on GitHub

@EvenOldridge is already working on this.

Since Jul 29, 2022.

roadmap
Dominant language
Python
Stars
907
Forks
129
PR merge metrics
No merged PRs in 30d

Description

Problem:

In Systems, we need to know information about the various ops that go into an Ensemble. The primary one is the input/output schemas for the data and models that make up the ensemble.

We are able to infer some of this from eg the Tensorflow model, but are not able to do so from more flexible frameworks like pytorch or xgboost.

Saving an NVTabular workflow produces a small amount of info in metadata.json, and I propose we expand that concept to record any expected metadata about models, nvt workflows, and other components that we expect to load in a Systems ensemble.

Goal:

  • Systems can load any model produced by Models and know the input/output schema information and any additional critical metadata TBD.
  • Systems can load any workflow produced by NVTabular and know the input/output schema information and any additional critical metadata TBD.

Constraints:

  • This information should be serialized to disk along with the artifact (model, workflow). We should not rely on any kind of service for keeping track of this information. (yet??)
  • The metadata format should be consistent with some standard fields (library versions, input_schema, output_schema)
  • Input/output schemas should ser/deser into the Schema python class
  • It should also be extensible in order to provide optional information about the artifact.

Starting Point:

Core repo:

  • Define a metadata format and schema that will be shared among all of the Merlin libraries (live in core).

Models repo:

  • Override the Model.save method in Models to include a metadata file including the required fields.

Systems repo:

  • Begin using the

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.