mapbox / mapbox/robosat

Save all-background mask in `rs rasterize` for hard-negative mining

Open
#43 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.1k
Forks
385
PR merge metrics
No merged PRs in 30d

Description

At the moment rs rasterize generates masks which always have a feature in them.

When we initially train on these masks and their corresponding images, we don't have background-only images in the training set. The images come with background pixels near the features, but the model might still pick up false positives.

Once we have a model trained on this initial dataset we use it to predict on tiles we know don't have a feature in their image and use false positives to feed them back into the dataset (it's important to put them into all splits, not just the training dataset).

This is called hard-negative mining where we "mine" for model mistakes a couple of times, put false positives back into the dataset, and then re-train, and repeat until we have a solid dataset.

Another option is to start with random negatives which is easier to do but the dataset then can get pretty big and often contains images which do not really help the model to train.

In both cases we need to background images. And we need to add them back into the dataset with a corresponding all-background mask. We don't provide such a mask at the moment.

Task

  • write out a all-background mask for users to do hard negative mining
  • document hard-negative mining process and how rs compare and rs subset helps

Workaround:

import numpy as np
from PIL import Image
from robosat.colors import make_palette

bg = np.zeros(shape=(512,512), dtype=np.uint8)                                                         
img = Image.fromarray(bg, mode='P')
img.putpalette(make_palette('denim', 'orange'))                                                      
img.save('bg.png', optimize=True)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the rs rasterize entry point and compare its output with the provided NumPy/PIL workaround for an all-background mask. Then review how rs compare and rs subset support the documented hard-negative mining process, including adding background images and masks to all dataset splits; done means both the mask output and the workflow documentation are provided.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
cli, documentation, machine-learning
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.