clab / clab/dynet

TopK for Tensor?

Open
#1,263 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

Faster TopK functions for Tensor can be useful for processes like beam search. Correct me if I'm wrong, currently I have not found efficient way to do this directly on GPU in dynet.

It seems that tensorflow and pytorch adopt two different methods for this (but seemingly not with Eigen), which can be good references or resources:
Tensorflow (heap-based):
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/core/kernels/topk_op_gpu.cu.cc
PyTorch (radix-selection):
https://github.com/pytorch/pytorch/blob/master/aten/src/THC/THCTensorTopK.cuh

Will it be ok to adapt them into dynet? (Putting the relevant codes in `third_party` with corresponding comments about lisence and sources.)
Maybe something like:
`pair TensorTools::topk(const Tensor& v, unsigned dim = 0, unsigned k = 1);`
If ok, I would like to work on this.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the proposed TensorTools::topk API and the linked TensorFlow and PyTorch GPU implementations. Determine how the relevant code could fit under third_party with license and source comments, and define the requested dimension and k behavior. Done means DyNet exposes an efficient GPU TopK operation for Tensor.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, pytorch, tensorflow
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.