weecology / weecology/DeepForest

Integrate segment anything for post model bounding box to polygons.

Open
#460 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Ideas for Machine Learning!
Dominant language
Python
Stars
774
Forks
265
PR merge metrics
No merged PRs in 30d

Description

https://segment-anything.com/ was a new transformer object detection model. @MarconiS showed good results for bounding box to polygon workflows. The main problem was it was too slow. There is a new CNN module that claims similar performance at 50x speed. This needs to be investigated.

https://arxiv.org/pdf/2306.12156.pdf

Screenshot from 2023-07-05 21-04-27 (1)

Next steps.

[ ] @MarconiS can you give us links and a couple code snippets to assess next steps for getting this into main branch.
[ ] Determine how to evaluate results given bounding box test data we have on hand
[ ] Identify potential workflow: after prediction (predict_image or predict_tile), take bounding box coordinates as prompts for segment anything. Do we use the 4 vertexes of the bounding box, or the centroids of the boxes?
[ ] Add fastSA model weights to git download. Use for either bird or tree release.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the Segment Anything site and the linked paper, then inspect the existing predict_image and predict_tile workflows. Use the available bounding box test data to define how results should be evaluated and decide how boxes prompt segmentation; done means a documented integration path, evaluation approach, and plan for adding fastSA model weights.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.