clab / clab/dynet

Use a Dynet Tensor to lookup parameters

Open
#719 4 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

Is it possible to lookup parameters using an object of type ``?

I have `LookupParameters` and usually I can call it like this:

`emb = my_parameters[tok_idx] # tok_idx is a Python integer`

However, I'd like to replace `tok_idx` with a 1D Dynet Tensor.

I'm trying to incorporate [scheduled sampling](https://arxiv.org/pdf/1506.03099.pdf) (Bengio et al 2015) into my sequence-to-sequence model while retaining the ability to use autobatching. In this approach, during training, I sometimes pass to the LSTM the most likely token from the previous timestep instead of the gold token (a simple model would use the gold token only). I can compute the most likely token using Dynet Tensors and argmax, but I can't then compute that tensor's embedding using `LookupParameters` (as far as I know).

Additionally, I'm wondering if the following calls a forward pass (similar to calling `.npvalue()`) on the vector expression `output_scores`:

`most_likely_token = output_scores.tensor_value().argmax()`

I know that `.as_numpy()` probably calls a forward pass on a tensor, but not sure about this function. I want to make sure that autobatching doesn't break when I do this.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.