developmentseed / developmentseed/segment-anything-services

torch.compile for faster n=1 batch runtime after the first inference

Open
#24 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
96
Forks
10
PR merge metrics
No merged PRs in 30d

Description

TODO

* test first inference time with torch.compile
* test time for subsequent inferences on V100s
* see if we get a 20% speedup

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files or tests. Start by locating the image-embedding, prompting, and mask-generation service entry points, then run benchmarks on V100s for the first and subsequent inferences with torch.compile. Done means recording whether subsequent n=1 inference achieves the proposed 20% speedup and comparing first-inference cost.

Written by the indexing model from the issue text.

Assessment

Tech stack
pytorch
Domain
backend, machine-learning, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.