combust / combust/mleap

Serialization filename conflicts when transformers have the same UID

Open
#663 2 comments 0 reactions 0 assignees View on GitHub
waiting for input
Dominant language
Scala
Stars
1.5k
Forks
315
PR merge metrics
No merged PRs in 30d

Description

I'm attempting to rebuild our ML pipeline using MLeap to serve our models. Our transformers are written in Java and mostly extend Spark's `UnaryTransformer` class. They all also start with something like
```java
private static String uid = Identifiable$.MODULE$.randomUID("PatternNormalizerTransformer");
```
Because `uid` is static, the variable gets created the first time a `PatternNormalizerTransformer` (or whatever transformer) is instantiated, then all subsequent instances use the same value. This means if a pipeline has more than one `PatternNormalizerTransformer` they’ll actually both have the same `uid`. This hasn’t proven to be a problem in the past because when Spark serializes its stages it prepends the stage number to the beginning of the uid. For example, in the `stages` directory of one of our serialized Spark models we might have stage subdirectories like
```
00_PatternNormalizerTransformer_011f91059577
01_PatternNormalizerTransformer_011f91059577
02_DowncaseTransformer_0f24dea8e78f
03_NumberNormalizerTransformer_30ce0fc90f82
etc
```
Despite both `PatternNormalizerTransformer` s having the same uid – “PatternNormalizerTransformer_011f91059577” – there’s no conflict because of the stage number. MLeap does not seem to prepend this number when serializing though, so I’m running into “file already exists: PatternNormalizerTransformer_011f91059577" type errors when trying to serialize a model that has more than one of a particular transformer type.*

I tried just changing the `uid` field to not be static, so that each instance gets its own `uid` (as it should be anyway), but this causes problems when trying to set `Param`s using `transformer.set(myParam, value)`:
```
Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Param null__inputCol does not belong to PatternNormalizerTransformer_a2b7256a4abb.
```
The param name “null__inputCol” hints as to what is going on. Usually, the param is named “{uid}__inputCol”. The fact that here it’s null suggests that the inputCol (and presumably outputCol) `Param`s are created as part of the `UnaryTransformer` parent class before `uid` has been set. At that point the `Param` is thought to belong to a transformer with uid “null”. Then once the `PatternNormalizerTransformer` is instantiated and given an actual `uid`, that `uid` no longer matches the “null” value associated with the `Param` and throws an error.

So I’m currently stuck. My hope is there is a way to define how MLeap names its stage directories during serialization, i.e. prepending stage index numbers to the uid. Is this possible? Alternatively, are there any other solutions to this issue?

---
\* I thought of creating a transformer that has multiple input/output columns, to avoid having to have more than one instance of a particular transformer in a pipeline, but as these transformers often take different sets of parameters (for example, the `PatternNormalizerTransformer` takes a regex pattern parameter and a case sensitive bool parameter), they would require their own transformers

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.