Add denormalized Eval.DequeueID RPC
- Dominant language
- Go
- Stars
- 17k
- Forks
- 2.1k
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 105
Description
#5407 made me realize that thanks to #5381 there's no reason to serialize the whole Evaluation over the Dequeue RPC because the worker will wait until the Eval is available locally before processing it. #5411 makes retrieving the Eval from the state store even easier.
## Proposal
A new RPC could be added - `DequeueID` - that replaces the `*structs.Evaluation` in the response with just an `EvalID string`. Workers would then retrieve the Eval by EvalID after waiting on the WaitIndex.
## Performance
`struct.Evaluation` is small, so unless `ClassEligibility` or `FailedTGAllocs` are large, the performance impact may be negligible.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the existing Dequeue RPC and the worker path that waits on the WaitIndex before retrieving an Eval from the state store. Implement the proposed DequeueID response with an EvalID string and update workers to retrieve the Eval after waiting; done means the worker flow works without serializing the full Evaluation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100