`ctl run-benchmark --local` OOM on local infer
- Dominant language
- Go
- Stars
- 714
- Forks
- 64
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 58
Description
our inference process has the tendency to OOM given it loads full git repos into memory. there are various ways we could mitigate this (e.g. with file-backed storage at the cost of speed) but the true fix would be to move inference to a separate process and detect/recover from OOM.
one additional thing to note is that we might want to refactor the inference server (and others) to accept a port flag instead of hard-coding the Cloud Run default. for ctl, we can use the port picker logic from internal/firestoretest/emulator.go to pass in an available port.
And thanks @giacomobenedetti for reporting this issue!
Contributor guide
Research direction
Start with the ctl run-benchmark --local entry point and inspect the inference server; use internal/firestoretest/emulator.go as the reference for selecting an available port. Done should address local inference OOM through process isolation and recovery, while allowing the inference server to accept a chosen port instead of the Cloud Run default.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100