Getting Prediction Values Change for individual instances

I have trained a model for a binary classification. I am interested in finding the Prediction Value Change for individual instances. I know that using

```
model.get_feature_importance(Pool(X_test, label=y_test.,cat_features=categorical_features_indices),
type='PredictionValuesChange')
```
Gives me the values over the whole test set. I am trying to do this for individual instance such as:

```
model.get_feature_importance(Pool(X_test.iloc[0:1], label=y_test.iloc[0:1],cat_features=categorical_features_indices),
type='PredictionValuesChange')
```
Despite the fact that `Pool()` returns as object, it seems get_feature_importance() method is not happy with individual rows. This is somehow would be similar to `shap.force_plot()` method where one can see the SHAP values on each instances. I wonder if making `PredictionValuesChange` makes sense for individual cases at all?