carpentries-incubator / carpentries-incubator/bioimage-analysis-python

Episode 9, Exercise 1: Concept of Validation proposal - "Segmentation Challenge"

Open
#21 3 comments 2 reactions 0 assignees View on GitHub
exercise-proposal
Dominant language
HTML
Stars
12
Forks
6
Avg merge
1m
Merged PRs (30d)
1

Description

### Brief description
This exercise takes the form of a "segmentation challenge": a few _challenge_ images are presented to learners, who are invited to come up with their best workflow to segment these images.

The images contain multiple recognizable objects, suitable for instance segmentation. For example:

Image

Each image is also accompanied with a carefully curated, manually edited segmentation mask to use as a "ground truth" for evaluating segmentation accuracy.

The evaluation metrics are:
- (1) object count error (relative to the true object count)
- (2) intersection over union (computed on the binary version of the masks)

Learners are invited to load and process the images using scikit-image and the tools and techniques they've learned during the course. When they feel like they have come to an acceptable solution, they can calculate the evaluation metrics for their solution against the ground truth mask.

**➡️ Setup with an interactive "leaderboard"**

To make it a more fun and engaging session:

- Learners can "submit" their proposed segmentation masks by uploading them to a shared folder.
- A little script automatically retrieves the masks in the shared folder, computes the metrics, and updates a "leaderboard" panel displayed on the main screen.

### Learning objective(s)
Related episode: [Concept of Validation](https://github.com/carpentries-incubator/bioimage-analysis-python/wiki/Learning-Objectives#9-validation)

Main objectives:

- Understand how segmentation masks can be compared to a ground truth (or other segmentation masks) quantitatively.
- Practice concepts and techniques seen during the rest of the course.

### Volunteer(s)
@MalloryWittwer

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the Episode 9 Concept of Validation learning objectives and the existing lesson structure. Define the challenge images, curated ground-truth masks, and scikit-image workflow, then determine how submissions in the shared folder will be evaluated for object count error and intersection over union and shown on the leaderboard. Done means the exercise is documented and the interactive evaluation flow is usable by learners.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, content
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.