facebookresearch / facebookresearch/SlowFast

Training on Custom Dataset

Open
#480 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7.4k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

Hi Team,
I'm using Custom Dataset to train the model. I put the videos and csv files according to [issue149](https://github.com/facebookresearch/SlowFast/issues/149). Like this:
```
SlowFast/
├── configs/
│ └── MyData/
│ └── I3D_8x8_R50.yaml
├── data/
│ └── MyData/
│ ├── ClassA/
│ │ └── ins.mp4
│ ├── ClassB/
│ │ └── kep.mp4
│ ├── ClassC/
│ | └── tak.mp4
│ ├── train.csv
│ ├── test.csv
│ ├── val.csv
│ └── classids.json
├── slowfast/
│ └── datasets/
│ ├── __init__.py
│ ├── mydata.py
│ └── ...
└── ...
```
When I tried to run. It said can't find the train.csv.
```
[09/28 16:15:53][INFO] mydata.py: 73: Constructing Mydata train...
Traceback (most recent call last):
File "tools/run_net.py", line 44, in
main()
File "tools/run_net.py", line 25, in main
launch_job(cfg=cfg, init_method=args.init_method, func=train)
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/slowfast/utils/misc.py", line 297, in launch_job
func(cfg=cfg)
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/tools/train_net.py", line 392, in train
train_loader = loader.construct_loader(cfg, "train")
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/slowfast/datasets/loader.py", line 83, in construct_loader
dataset = build_dataset(dataset_name, cfg, split)
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/slowfast/datasets/build.py", line 31, in build_dataset
return DATASET_REGISTRY.get(name)(cfg, split)
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/slowfast/datasets/mydata.py", line 74, in __init__
self._construct_loader()
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/slowfast/datasets/mydata.py", line 83, in _construct_loader
assert g_pathmgr.exists(path_to_file), "{} dir not found".format(
AssertionError: train.csv dir not found
```

Then I tried to put the csv files under the SlowFast, like this:
```
SlowFast/
├── configs/
│ └── MyData/
│ └── I3D_8x8_R50.yaml
data/
│ └── MyData/
│ ├── ClassA/
│ │ └── ins.mp4
│ ├── ClassB/
│ │ └── kep.mp4
│ ├── ClassC/
│ | └── tak.mp4

│ └── classids.json
├── slowfast/
│ └── datasets/
│ ├── __init__.py
│ ├── mydata.py
│ └── ...
├──train.csv
├── test.csv
├── val.csv
└── ...
```
This time the it successfully finds the csv files and start the epoch 1. But it failed to load the video through the address in csv files.
```
[09/28 16:23:45][INFO] mydata.py: 73: Constructing Mydata train...
[09/28 16:23:45][INFO] mydata.py: 111: Constructing Mydata dataloader (size: 150) from train.csv
[09/28 16:23:45][INFO] mydata.py: 73: Constructing Mydata val...
[09/28 16:23:45][INFO] mydata.py: 111: Constructing Mydata dataloader (size: 50) from val.csv
[09/28 16:23:45][INFO] mydata.py: 73: Constructing Mydata train...
[09/28 16:23:45][INFO] mydata.py: 111: Constructing Mydata dataloader (size: 150) from train.csv
[09/28 16:23:45][INFO] train_net.py: 417: Start epoch: 1
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing012_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 26 from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4; trial 0
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 124 from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4; trial 0
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing012_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 26 from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4; trial 1
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 124 from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4; trial 1
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing012_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 26 from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4; trial 2
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 124 from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4; trial 2
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing012_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 124 from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4; trial 3
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 26 from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4; trial 3
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing012_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 26 from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4; trial 4
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 124 from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4; trial 4
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing012_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 26 from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4; trial 5
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 124 from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4; trial 5
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing012_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 26 from /slowfast/data/MyData/Vandalism/Vandalism029_x264.mp4; trial 6
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 124 from /slowfast/data/MyData/Stealing/Stealing012_x264.mp4; trial 6
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing089_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing089_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Shoplifting/Shoplifting006_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Shoplifting/Shoplifting006_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 109 from /slowfast/data/MyData/Stealing/Stealing089_x264.mp4; trial 7
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 35 from /slowfast/data/MyData/Shoplifting/Shoplifting006_x264.mp4; trial 7
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Normal/Normal_Videos_478_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Normal/Normal_Videos_478_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Shoplifting/Shoplifting043_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Shoplifting/Shoplifting043_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 78 from /slowfast/data/MyData/Normal/Normal_Videos_478_x264.mp4; trial 8
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 42 from /slowfast/data/MyData/Shoplifting/Shoplifting043_x264.mp4; trial 8
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Stealing/Stealing065_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Stealing/Stealing065_x264.mp4'
[09/28 16:23:45][INFO] mydata.py: 205: Failed to load video from /slowfast/data/MyData/Normal/Normal_Videos_905_x264.mp4 with error [Errno 2] No such file or directory: '/slowfast/data/MyData/Normal/Normal_Videos_905_x264.mp4'
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 111 from /slowfast/data/MyData/Stealing/Stealing065_x264.mp4; trial 9
[09/28 16:23:45][WARNING] mydata.py: 212: Failed to meta load video idx 84 from /slowfast/data/MyData/Normal/Normal_Videos_905_x264.mp4; trial 9
Traceback (most recent call last):
File "tools/run_net.py", line 44, in
main()
File "tools/run_net.py", line 25, in main
launch_job(cfg=cfg, init_method=args.init_method, func=train)
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/slowfast/utils/misc.py", line 297, in launch_job
func(cfg=cfg)
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/tools/train_net.py", line 450, in train
train_epoch(
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/tools/train_net.py", line 49, in train_epoch
for cur_iter, (inputs, labels, _, meta) in enumerate(train_loader):
File "/home/zhaoyan/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
data = self._next_data()
File "/home/zhaoyan/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/zhaoyan/.local/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/zhaoyan/.local/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "/home/zhaoyan/.local/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/zhaoyan/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhaoyan/.local/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/zhaoyan/Desktop/UCF Crime/ActionRecognition/slowfast/slowfast/datasets/mydata.py", line 269, in __getitem__
raise RuntimeError(
RuntimeError: Failed to fetch video after 10 retries.
```
Then I tried to edit the videos address in csv files. I didn't move these csv files, I just changed the videos address inside the csv files. But once I edited, it will back to "can't find the train.csv".
Now I'm confused, can you help me figure out where I should put the csv files or what videos address should I put. Thank you so much!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.