tensorflow / tensorflow/recommenders

Ambiguous DCN layer

Open
#554 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
Python
Stars
2k
Forks
300
PR merge metrics
No merged PRs in 30d

Description

I would like to ask for some help regarding stacked Cross layers.

From this tutorial, the calling of a simple DCN layer looks like this:

if use_cross_layer:
      self._cross_layer = tfrs.layers.dcn.Cross(
          projection_dim=projection_dim,
          kernel_initializer="glorot_uniform")

......

x = tf.concat(embeddings, axis=1)

# Build Cross Network
if self._cross_layer is not None:
    x = self._cross_layer(x)

From the documentation of the DCN Cross layer:

# after embedding layer in a functional model:
input = tf.keras.Input(shape=(None,), name='index', dtype=tf.int64)
x0 = tf.keras.layers.Embedding(input_dim=32, output_dim=6)
x1 = Cross()(x0, x0)
x2 = Cross()(x0, x1)
logits = tf.keras.layers.Dense(units=10)(x2)
model = tf.keras.Model(input, logits)

I really don't get how it is possible to properly call the Cross layer first with one variable then with two. The second looks more reasonable for me according to the DCN paper and the tutorial.

What I would like to do is to stack multiple Cross layers and properly feed them with the data from the embedding layers.

self.cross_layers = []

for i in range(n_cross_layers):
    self.cross_layers.append(tfrs.layers.dcn.Cross(
        projection_dim=projection_dim,
        #kernel_regularizer='glorot_uniform'
    ))


def call(self, inputs: tf.Tensor, training: bool = False) -> tf.Tensor:

    embedding = tf.concat(
        [self.embeddings[key](inputs[key])
         for key in self.embedding_features],
        axis=1
    )
    embedding = tf.concat([
        embedding,
        tf.stack([inputs[feature] for feature in self.features], axis=1),
    ], axis=1)

    if self.cross_layers:
        x1 = self.cross_layers[0](embedding, embedding)

        for cross_layer in self.cross_layers[1:]:
            x1 = cross_layer(embedding, x1)

        embedding = x1

    embedding = self.dense_layers(embedding, training)

    return embedding

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the TensorFlow Recommenders Cross layer API documentation and the linked DCN tutorial, focusing on the differing call signatures and the stacked-layer example in the issue. Clarify how the first and subsequent Cross layers should be called, and document a complete, unambiguous stacking pattern so users can tell when the guidance is complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.