Use a Dynet Tensor to lookup parameters
- Dominant language
- C++
- Stars
- 3.4k
- Forks
- 701
- PR merge metrics
- No merged PRs in 30d
Description
Is it possible to lookup parameters using an object of type ``?
I have `LookupParameters` and usually I can call it like this:
`emb = my_parameters[tok_idx] # tok_idx is a Python integer`
However, I'd like to replace `tok_idx` with a 1D Dynet Tensor.
I'm trying to incorporate [scheduled sampling](https://arxiv.org/pdf/1506.03099.pdf) (Bengio et al 2015) into my sequence-to-sequence model while retaining the ability to use autobatching. In this approach, during training, I sometimes pass to the LSTM the most likely token from the previous timestep instead of the gold token (a simple model would use the gold token only). I can compute the most likely token using Dynet Tensors and argmax, but I can't then compute that tensor's embedding using `LookupParameters` (as far as I know).
Additionally, I'm wondering if the following calls a forward pass (similar to calling `.npvalue()`) on the vector expression `output_scores`:
`most_likely_token = output_scores.tensor_value().argmax()`
I know that `.as_numpy()` probably calls a forward pass on a tensor, but not sure about this function. I want to make sure that autobatching doesn't break when I do this.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.