MegEngine / MegEngine/Models

量化模型的时候出现不预期的问题

Open
#103 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
306
Forks
96
PR merge metrics
No merged PRs in 30d

Description

## 环境
1.系统环境:ubuntu1804
2.MegEngine版本:1.4.0
3.python版本:3.6.8

## 复现步骤
1. git clone https://github.com/MegEngine/Models.git
2. python3 train.py -a resnet18 -d /path/to/imagenet --mode normal 出现此问题

## 请提供关键的代码片段便于追查问题
1、train.py 和 finetune.py 同样出现此问题

## 请提供完整的日志及报错信息
Traceback (most recent call last):
File "finetune.py", line 315, in
main()
File "finetune.py", line 69, in main
train_proc(world_size, args)
File "finetune.py", line 159, in worker
data.RandomSampler(train_dataset, batch_size=cfg.BATCH_SIZE, drop_last=True)
File "/usr/local/python/lib/python3.6/site-packages/megengine/data/sampler.py", line 322, in __init__
self.sampler_iter = iter(self.sampler)
File "/usr/local/python/lib/python3.6/site-packages/megengine/data/sampler.py", line 100, in __iter__
return self.batch()
File "/usr/local/python/lib/python3.6/site-packages/megengine/data/sampler.py", line 146, in batch
if self.drop_last and len(batch_index[-1]) < self.batch_size:
IndexError: list index out of range

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the command from the issue in Models/train.py or finetune.py, then inspect the caller at finetune.py line 159 and MegEngine's data/sampler.py around lines 100 and 146. Confirm why RandomSampler reaches the empty batch case; done means the reported command no longer raises IndexError and the behavior is covered by an appropriate regression check.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.