[FEA] POC integration with TF Serving

Open
#186 1 comment 0 reactions 1 assignee View on GitHub

@oyilmaz-nvidia is already working on this.

Since Sep 18, 2020.

Assessment

This issue has not been assessed yet.

Description

Inference P1 TensorFlow

Users who leverage TF Serving for model inference and TF Transform for preprocessing will have a hard time leveraging NVTabular preprocessing because they need a way to execute the preprocessing graph on raw data at inference time (which essentially means building it in to their TensorFlow graph at model export time).

We can hack around this functionality by adding a TF equivalent to individual Op logic (including the necessary stats as constants) as some sort of method on the Op (or for conceptual cleanliness, some parallel tf_ops.py or something which mirrors all Ops with a child class with the appropriate method), then build a function which calls the ops in the appropriate order and compiles and exports the graph as a SavedModel (basically just a Graph protobuf and its assets).

Definitely easier said than done, I'm sure figuring out some of the plumbing of which ops individual features flow between will be at least somewhat non-trivial, but that's at least the idea in principle. I've written up a doc comparing the functionality of NVTabular and TFTransform here that will hopefully illuminate both how to make this functionality work intuitively in the short term, as well as how to build equivalent functionality in pure NVT + Triton in the longer term.

Dominant language
Python
Stars
1.2k
Forks
149
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from NVIDIA-Merlin/NVTabular

All issues in NVIDIA-Merlin/NVTabular

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.