mapbox / mapbox/robosat

Division by zero when training with small datasets caused by incomplete batch dropping

Open
#228 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.1k
Forks
385
PR merge metrics
No merged PRs in 30d

Description

I am aware that this repo is no longer maintained but I recently had to blow the dust off of it to replicate some stuff I did a few years ago. When setting up the environment I kept getting a division by 0 [here](https://github.com/mapbox/robosat/blob/cbb1c73328183afd2d6351b7bfa3f430b73103ea/robosat/tools/train.py#L197). I finally resolved it and thought I should leave a note here in case anyone finds themself in a similar situation.

It turns out it was because I was using a trivially small datasetr (only a few features) because I was just trying to get the entire pipeline working correctly. Specifically, when initializing the DataLoaders, the `drop_last` parameter is hardcoded to `True`, which will always drop the last batch if it is incomplete. So if you only have one (tiny) batch, you end up with an empty data loader, causing the aforementioned division by zero error.

Change this parameter to `False` [here](https://github.com/mapbox/robosat/blob/master/robosat/tools/train.py#L273) and [here](https://github.com/mapbox/robosat/blob/master/robosat/tools/train.py#L274) to fix it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in robosat/tools/train.py, especially the division at line 197 and the DataLoader initialization around lines 273-274. Reproduce training with a dataset smaller than one batch, then verify that incomplete final batches are retained and training no longer fails with division by zero.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
1/5
Estimated time
Under an hour
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.