automl / automl/NASLib

Different taskonomy n. classes

Open
#190 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
595
Forks
133
PR merge metrics
No merged PRs in 30d

Description

Hi, I am using the code, and I noticed that in the graph creation of TransBench101 both Micro and Macro the number of classes is different from those reported on the original work

See line 44 https://github.com/yawen-d/TransNASBench/blob/training_code_with_config/configs/train_from_scratch/class_scene/config.py, and all the config files, and the original paper https://arxiv.org/pdf/2105.11871 Table 1.

While in this work in naslib>search_spaces>transbench101>graph.py the classes are so defined

if dataset == "jigsaw":
self.num_classes = 1000
elif dataset == "class_object":
self.num_classes = 100 # why not 75?
elif dataset == "class_scene":
self.num_classes = 63 # why not 47??


Is ther a reason for doing so? How does this affect the results?

Contributor guide

Open the contributing guide

Research direction

Compare naslib/search_spaces/transbench101/graph.py with the referenced TransNASBench config files, especially config.py line 44, and Table 1 of the linked paper. Determine whether the class-count mismatch is intentional and document or correct the behavior after checking how these values affect graph creation and reported results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.