mapbox / mapbox/robosat

Handle instance segmentation by adding second output channel for touching borders

Open
#52 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.1k
Forks
385
PR merge metrics
No merged PRs in 30d

Description

At the moment we don't do instance segmentation. This allows our model to be very simple while still achieving amazing results for our current use-cases. Where it breaks down, though, is use cases like extracting buildings in very densely populated areas.

See for example a quick prototype for Tanzania where the segmentation mask alone can not distinguish between touching buildings.

![dense](https://user-images.githubusercontent.com/527241/41907204-6e6bc1e8-7940-11e8-9b5d-3a0d01355de9.png)

And while proper instance segmentation models are much more complicated, there is one trick we can pull off based on what the folks in https://arxiv.org/abs/1806.00844 propose.
- [ ] Add a second channel to the output. The first channel will be the segmentation mask as it is right now. The second channel will represent touching features - and only the border between features.
- [ ] Train with ground truth segmentation masks and compute and rasterize borders where features touch.
- [ ] After prediction feed results through the watershed transform to divide touching features into multiple features.

As a result we will get instance segmentation and can distinguish between buildings in the use-case above.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the linked paper and the issue checklist, focusing on the proposed second output channel and watershed transform. No repository files or tests are named, so the implementation entry points must be located first. Done means training can produce touching-border masks and post-processing separates touching buildings into distinct instances.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.