tensorflow / tensorflow/models

DELG implementation does not have the auto-encoder as cited in paper.

Open
#9,189 12 comments 1 reaction 1 assignee View on GitHub

@andrefaraujo is already working on this.

Since Sep 2, 2020.

models:research type:feature
Dominant language
Python
Stars
77.7k
Forks
44.8k
PR merge metrics
No merged PRs in 30d

Description

The DELG paper

According to the latest version DELG paper, the local feature part of DELG is slightly different from DELF. It apply an auto-encoder on features and then use the attention weights on reconstructed features. losses of auto-encoder reconstruction and attention classifier are used together to train the local features. Just like the two screenshot below:

截屏2020-09-02 下午7 23 51 截屏2020-09-02 下午7 21 35

The implementation of DELG in DELF repo

However, the code implementation of DELG here does not include the auto-encoder.

  1. In delg_model.py, The class DELG is a subclass of DELF and use the exact local feature part as DELF.

  2. in train.py, we can look at the train_step function (line 242 -322). Only SparseCategoricalCrossentropy losses of two classifier (descriptor and attention) are taken into account.

Question about the auto-encoder feature

Does the implementation of DELG here miss the auto-encoder? or the paper has a wrong description about its model structure?

Is there any plan to add the auto-encoder inside?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.