Save all-background mask in `rs rasterize` for hard-negative mining
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.1k
- Forks
- 385
- PR merge metrics
- No merged PRs in 30d
Description
At the moment rs rasterize generates masks which always have a feature in them.
When we initially train on these masks and their corresponding images, we don't have background-only images in the training set. The images come with background pixels near the features, but the model might still pick up false positives.
Once we have a model trained on this initial dataset we use it to predict on tiles we know don't have a feature in their image and use false positives to feed them back into the dataset (it's important to put them into all splits, not just the training dataset).
This is called hard-negative mining where we "mine" for model mistakes a couple of times, put false positives back into the dataset, and then re-train, and repeat until we have a solid dataset.
Another option is to start with random negatives which is easier to do but the dataset then can get pretty big and often contains images which do not really help the model to train.
In both cases we need to background images. And we need to add them back into the dataset with a corresponding all-background mask. We don't provide such a mask at the moment.
Task
- write out a all-background mask for users to do hard negative mining
- document hard-negative mining process and how
rs compareandrs subsethelps
Workaround:
import numpy as np
from PIL import Image
from robosat.colors import make_palette
bg = np.zeros(shape=(512,512), dtype=np.uint8)
img = Image.fromarray(bg, mode='P')
img.putpalette(make_palette('denim', 'orange'))
img.save('bg.png', optimize=True)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the rs rasterize entry point and compare its output with the provided NumPy/PIL workaround for an all-background mask. Then review how rs compare and rs subset support the documented hard-negative mining process, including adding background images and masks to all dataset splits; done means both the mask output and the workflow documentation are provided.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- numpy, python
- Domain
- cli, documentation, machine-learning
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100