developmentseed / developmentseed/segment-anything-services
torch.compile for faster n=1 batch runtime after the first inference
Open
- Dominant language
- Jupyter Notebook
- Stars
- 96
- Forks
- 10
- PR merge metrics
- No merged PRs in 30d
Description
TODO
* test first inference time with torch.compile
* test time for subsequent inferences on V100s
* see if we get a 20% speedup
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files or tests. Start by locating the image-embedding, prompting, and mask-generation service entry points, then run benchmarks on V100s for the first and subsequent inferences with torch.compile. Done means recording whether subsequent n=1 inference achieves the proposed 20% speedup and comparing first-inference cost.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pytorch
- Domain
- backend, machine-learning, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100