Providing explanations for recommendations

Apologies if this has been covered somewhere already (I did search the issues for the term "explanation") but I was wondering about support for providing some explanation for why a recommendation has been made.

I've just been experimenting in the context of recommending similar users as opposed to recommending movies, but I think we can talk about explaining a recommendation based on the NN's internal representations in both cases no?

Given that in, say, the movie context, that we have two networks of equal dimensionality (e.g. 16) one representing users, and the other representing movies, can we work as follows provide an explanation for a recommendation?

Explain the recommendation of the user with id 496 to user with id 1 by adding their two embedding vectors and then looping through all the movie embeddings and doing a dot product to assess the extent to which each movie was involved in the "recommendation":

unique_movies = set()
         
for m in movies.take(100000):
  unique_movies.add(m.numpy())

user1 = model.user_model(np.array(["1"]))
user2 = model.user_model(np.array(["496"]))

df = pd.DataFrame()
for movie in movies:
    movie_vector = model.movie_model(np.array([movie]))
    score = np.dot((user1 + user2),np.transpose(movie_vector))
    df = df.append({'interest': [movie], 'score':[score]}, ignore_index=True)
pd.set_option('display.max_rows', None)
tf.print(df.sort_values(by='score', ascending=False))

this gives a list of the different movies that are influencing why these two users are close together in the user model (maybe?)

                                               interest             score
                                      interest             score
1512      [b'Reluctant Debutante, The (1958)']    [[[8.097834]]]
1128          [b'Little Princess, The (1939)']    [[[6.621847]]]
473               [b'Madame Butterfly (1995)']   [[[6.3796196]]]
1656  [b'Faster Pussycat! Kill! Kill! (1965)']   [[[6.2495246]]]
76                   [b'Poison Ivy II (1995)']   [[[6.1312265]]]
...                                        ...               ...
418          [b'Very Natural Thing, A (1974)']  [[[-5.3535113]]]
977                   [b'Men of Means (1998)']   [[[-5.416351]]]
868               [b'Truman Show, The (1998)']   [[[-5.512992]]]
26              [b'Further Gesture, A (1996)']  [[[-5.5256224]]]
1447                 [b'Critical Care (1997)']   [[[-5.964835]]]

I'm not sure if my logic is entirely sound here. In a test with my own data set the highest ranked elements were ones shared between the two users. In this movies example the two users share ratings for these two films:

734     [b'Star Trek III: The Search for Spock (1984)']      [[[1.5130659]]]
1116                           [b'Pulp Fiction (1994)']     [[[0.11822829]]]

which are not scored particularly highly according to my "explanation" metric. The main difference between my data set and the one in the movielens one is that mine has an order of magnitude more users, but an order of magnitude fewer "movies". I'll do more tests on my data to confirm what I'm seeing there ...

I thought I'd post what I have so far to see if anyone else is thinking about recommendation "explanations" and if my approach sort of makes sense in the large, i.e. use the two NNs internal representations as equivalent in terms of mapping the recommendation space ... or am I mis-understanding how these embeddings are related ...?

No repository files, tests, or entry points are named. Start by evaluating the proposed user_model and movie_model embedding calculation and whether it explains recommendation scores; done would require an agreed explanation design, implementation scope, and validation criteria.