microsoft / microsoft/SynapseML

TabularSHAP hangs indefinitely

Open
#1,945 11 comments 0 reactions 2 assignees View on GitHub

@memoryz is already working on this.

Since May 1, 2023.

area/rai
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

Hi SynapseML team,

I am using TabularSHAP to explain predictions of the LightGBM classification model using SynapseML version `com.microsoft.azure:synapseml_2.12:0.9.5-35-e962330b-SNAPSHOT`

Based on the [tutorial](https://microsoft.github.io/SynapseML/docs/next/features/responsible_ai/Interpretability%20-%20Tabular%20SHAP%20explainer), I have prepared my code below. I am using only one executor with five cores to ensure the code doesn't fail due to concurrency, and I repartition all data frames to use a single partition.

```
from pyspark.ml import Pipeline
from pyspark.ml.feature import VectorAssembler
from synapse.ml.lightgbm import JavaMLReadable, LightGBMClassifier, LightGBMClassificationModel
from synapse.ml.explainers import TabularSHAP

model = LightGBMClassifier.load('')
train_data = spark.read.parquet('').repartition(1).cache() # type: about 15K records
features = [...list of features...]

vector_assembler = VectorAssembler(inputCols = features, outputCol = 'features').setHandleInvalid("skip")
pipeline = Pipeline(stages=[vector_assembler, model])
model = pipeline.fit(train_data)

explain_instances = model.transform(train_data.limit(5).repartition(1).cache()).cache()
train_sample = train_data.orderBy(F.rand()).limit(100).repartition(1).cache()

shap = TabularSHAP(
inputCols=features,
outputCol="shapValues",
numSamples=10,
model=model,
targetCol="probability",
targetClasses=[1],
backgroundData=train_sample
)

explained = shap.transform(explain_instances).cache()
```

On the last line, when I call `shap.transform()`, the notebook shows me the progress bar but waits forever. I think it gets into some lock, but I am not sure. But I am sure it doesn't wait for the data frames to be calculated, because, while I have removed these lines from the code, I call `.show()` to force caching...

What am I missing?

Thank you.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.