facebookresearch / facebookresearch/detectron2

How can I train a subset of a custom data set

Open
#4,847 1 comment 0 reactions 0 assignees View on GitHub
documentation
Dominant language
Python
Stars
34.7k
Forks
7.9k
PR merge metrics
No merged PRs in 30d

Description

I successfully registered a custom dataset with 8 classes of objects. And it can be trained well.

Now, 4 of the 8 classes in my custom dataset is no need to be detected anymore。

I don't want to change the json file, so how do I re-register, so that the model can only focuses on the four useful objects when it trained?

**in detrex I tried to use the “MetadataCatalog.get” but it can not help**
**the error is**
AssertionError: Attribute 'thing_classes' in the metadata of 'my_eldataset_train' cannot be set to a different value!
['duanshan', 'heiban', 'xianzhuangquexian', 'xuhan'] != ['duanshan', 'heiban', 'xianzhuangquexian', 'xuhan', 'yinlie', 'crush', 'finger', 'star']

**codes**
import itertools
from omegaconf import OmegaConf

import detectron2.data.transforms as T
from detectron2.config import LazyCall as L
from detectron2.data import (
build_detection_test_loader,
build_detection_train_loader,
get_detection_dataset_dicts,
MetadataCatalog,
)
from detectron2.data.datasets import register_coco_instances
from detectron2.evaluation import COCOEvaluator

from detrex.data import DetrDatasetMapper

dataloader = OmegaConf.create()

register_coco_instances("my_eldataset_train", {}, '/data1/wzydatasets/yuanle/coco/annotations/instances_train2017.json', '/data1/wzydatasets/yuanle/coco/train2017/')
register_coco_instances("my_eldataset_test", {}, '/data1/wzydatasets/yuanle/coco/annotations/instances_test2017.json', '/data1/wzydatasets/yuanle/coco/test2017/')

MetadataCatalog.get("my_eldataset_train").thing_classes = ['duanshan', 'heiban', 'xianzhuangquexian', 'xuhan']
MetadataCatalog.get("my_eldataset_test").thing_classes = ['duanshan', 'heiban', 'xianzhuangquexian', 'xuhan']

dataloader.train = L(build_detection_train_loader)(
dataset=L(get_detection_dataset_dicts)(names="my_eldataset_train"),
mapper=L(DetrDatasetMapper)(
augmentation=[
L(T.ResizeShortestEdge)(
short_edge_length=600,
max_size=600,
),
L(T.RandomFlip)(),
L(T.ResizeShortestEdge)(
short_edge_length=(320, 480, 512, 544, 576, 608,),
# max_size=1333,
max_size=640,
sample_style="choice",
),
],
augmentation_with_crop=[
L(T.RandomFlip)(),
L(T.ResizeShortestEdge)(
short_edge_length=600,
max_size=600,
),
L(T.RandomCrop)(
crop_type="absolute_range",
crop_size=(300, 400), # 必须是列表,必须有两个参数,否则会报错。
),
L(T.ResizeShortestEdge)(
# short_edge_length=(480, 512, 544, 576, 608, 640, 672, 704, 736, 768, 800),
short_edge_length=(320, 480, 512, 544, 576, 608,),
# max_size=1333,
max_size=640,
sample_style="choice",
),
],
is_train=True,
mask_on=False,
img_format="RGB",
),
total_batch_size=16,
num_workers=4,
)

dataloader.test = L(build_detection_test_loader)(
dataset=L(get_detection_dataset_dicts)(names="my_eldataset_test", filter_empty=False),
mapper=L(DetrDatasetMapper)(
augmentation=[
L(T.ResizeShortestEdge)(
short_edge_length=600,
max_size=640,
),
],
augmentation_with_crop=None,
is_train=False,
mask_on=False,
img_format="RGB",
),
num_workers=4,
)

dataloader.evaluator = L(COCOEvaluator)(
dataset_name="${..test.dataset.names}",
)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.