developmentseed / developmentseed/segment-anything-services
Support customizable prompts, multiple points and bbox for decoder
- Dominant language
- Jupyter Notebook
- Stars
- 96
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
with the current service, you can only supply one point. But SAM supports supplying N points, or a bbox, which should greatly improve mask detection accuracy.
We can support more prompts by adapting `handler_decode.py` to accept a different payload and pass the prompt inputs more flexibly to ort.InferenceSession. This would involve
1. changing the structure of the expected payload. currently handler_decode.py expects
```
{
"image_embeddings": encoded_embedding_string,
"image_shape": img_shape,
"input_label": input_label, # needs to be updated and made more flexible
"input_point": input_point # needs to be updated and made more flexible
}
```
2. Changing how we pass this information to decoding inference. I think we might just be able to pass the same input shapes for two parameters, since the empty dimension I'm adding in both cases probably represents the potential for multiple points.
3. Handle a bbox input, not sure how to set this up yet but can look into it when it becomes a priority. I think handling N points is higher priority since it's easier on the user.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.