Scaletests: `aibridged`
- Dominant language
- No language data
- Stars
- 3
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
We need to scaletest `aibridged` to assess its impact on the larger `coderd` process of which it is part.
The main resources of concern / likely culprits are:
- shared database (underlying resources like CPU, RAM, disk)
- shared database connection pool
- aibridge is a write-heavy application
- process file handles - since many network conns are made
- CPU/RAM
- possibly indirect impact due to increased allocs leading to increased GC
- goroutines inflight -> RAM growth, scheduler thrashing (manifesting as high CPU)
- in-memory datastrucures in https://github.com/coder/coder/blob/main/enterprise/aibridged/pool.go
- JSON marshaling causing high allocs
- response buffers either using or allocating large amounts of memory
The above list is non-exhaustive.
We must take care to control the full roundtrip: i.e. we must create a mock upstream which exhibits certain behaviours to provoke scaling concerns. For example, a pathologically slow upstream which leads to many requests inflight. The upstream will need to be a service in the scaletest package; see `cli/exp_scaletest_smtp.go` in coder/coder for an illustrative example.
The mock upstream must handle both streaming and blocking requests. You might consider reusing the fixtures from https://github.com/coder/aibridge/tree/main/fixtures.
Regular AI interactions + injected tool calls should be handled in order to exercise the majority of code paths.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.