dotnet / dotnet/machinelearning

Slot Reconciliation Transform

Open
#407 0 comments 0 reactions 1 assignee Claimed by @codemzs View on GitHub
area-Transforms enhancement help wanted Priority:2
Dominant language
C#
Stars
9.4k
Forks
2k
Avg merge
2d 20h
Merged PRs (30d)
11

Description

In ML.NET, a predictor that is trained on N features, can be applied to any column with N features. This is fine, and normally works well. However, imagine that you have a model trained over "old" data. Then you have a new pipeline, or new data-file, perhaps with new features, or some other such thing. In order to apply the "old" model to the "new" data, a user would need to figure out how exactly to map that data to the new model. Sometimes this can be easy, but it at least involves some in-depth dissection of the old and new schemas, and the user would need to do this themselves by specifying indices. Even when this information is readily available it can be annoying to do.

We can imagine a more convenient way: if both the old and new feature vectors had slot names metadata (which is of course entirely possible), it would be a simple enough matter to figure out how to map the new feature vectors so that they looked like the old schema, and so that application of the old model would be then straight-forward.

# Definition

We can describe this in the form of a slot-mapping. Let `m` be the function mapping input slot indices to the set of output slot indices and null. (Null, because some input slots could be dropped.) Let `I` and `O` be the arrays of input metadata (usually `SlotNames`) that we are using to construct this mapping.

1. `m(i) == j` if and only if `I[i] == O[j]`.
2. `m(i) == null` if `I[i]` occurs nowhere in `O`.
3. Any uncovered output slot index will be handled either by allowing that value to have the default value for the output type (i.e., 0.0 for floating point values), or by throwing an error.

For any value `X` in *both* `I` and `O`, that value must occur exactly once. Otherwise, obviously, the mapping would be ambiguous.

We could imagine the following examples of reconciliations, where I provide slot names as an array like [A,B,C] to indicate slots named "A", "B", and "C", for example:

1. If we reconcile [A,B,C] to [A,C], we'd map slots 0,2 to slots 0,1, respectively, and drop slot 1.
2. If we reconcile [A,B,C] to [A,C,D], we'd map slots 0,2 to slots 0,1, respectively, drop slot 1, and either throw with the error that we could not find a slot named D, or else, have the output map to the default value for that column. (This might be a configurable `bool` option.)
3. If we reconcile [A,A,B,C] to [A,B,C,C], we'd have to throw with the error that the mapping is ambiguous, both on the grounds that we have no idea which of the two A to take, or which of the C to map to.
4. If we reconcile [A,A,B,C] to [B,C,D,D], we'd map slots 2,3 to slots 0,1, and assign the default value for that column to output slots 2,3. (Whether we'd want to output a warning that there's something fishy going on with the names, is another matter to consider.)

# Implementation

The interface to this transform would take:

* The normal one-to-one column transform inputs (`IDataView` to transform, name of output column, name of source column),
* The source schema and name of the column to reconcile that input to... if that name is left unspecified it can default to the source column name,
* The name of the metadata kind along which reconciliation will happen, by default `SlotNames`. (For the sake of the discussion we will treat this as the standard scenario.)

The resulting output's new column will contain a mapping of slots from the input column. (Normally reconciled along `SlotNames` metadata, though this should be configurable.)

An open question is whether the deserialize model merely holds the learnt reconciliation (that is, the index mapping), or whether the reconciliation should happen each time it is deserialized. The latter is the more principled design, but is considerably less efficient since it implies the serialization of the resulting metadata

## Related Work

Note that this task bears some resemblance to what is done in the code here:

https://github.com/dotnet/machinelearning/blob/ecc6857410f56cdc67c666de4e08844df3a1e288/src/Microsoft.ML.Data/Evaluators/EvaluatorUtils.cs#L452

There are some substantial differences though.

* That involves a reconciliation among multiple data-views, whereas this issue describes a reconciliation between two data-views.
* Also in the existing case the output is to get the union of all slots, whereas here we just want the input data view's column to resemble the input schema's column.
* We want this to be a transform, that is, it can be saved as part of a data model. That code above is primarily for reconciling different cross-validation folds for per-instance output, and so did not need to be saved as part of a data model.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.