pytorch / pytorch/vision

Improve features extraction user experience

Open
#7,392 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.9k
Forks
7.3k
Avg merge
1d 15h
Merged PRs (30d)
13

Description

🚀 The feature

Each model implements a function which returns the names of it's features.

Motivation, pitch

Currently, there is no nice way to extract features of multiple models. This is because the models are not implemented in a consistent manner, Each model has their own naming of the layers which makes the end user experience very nasty.

For example, I want to take in the model name as argument and extract its features

import random
import torchvision
from torchvision.models.feature_extraction import create_feature_extractor

model_names = torchvision.models.list_models(module=torchvision.models)

model_name = random.choice(model_names)
model = getattr(torchvision.models, model_name)()

feature_pyramid = create_feature_extractor(model, return_nodes={
                              'layer1': 'layer1',
                              'layer2': 'layer2',
                              'layer3': 'layer3',
                              'layer4': 'layer4',
                          })

But, this is not possible, because not models do not have consistent layer naming. So, now I need to figure out what the naming convention of each model is.

I understand that now the layer names can't be changed since it will not allow backward compatibility. Therefore, the proposal.

Alternatives
  • torchvision.models.feature_extraction.get_graph_node_names returns the names of all the parameters and not just the layers which outputs a feature.

  • timm has a very nice user experience for the same, which is what I would like torchvision to replicate

  • One could do this for each model but would take effort to generalize to all models.

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with torchvision.models.feature_extraction.create_feature_extractor and get_graph_node_names, then compare the feature-layer naming across the models referenced by the issue. The issue does not name implementation files or tests; done would require a consistent way to request model features without model-specific layer names, along with coverage for the supported models.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.