coder / coder/internal

Scaletests: `aibridged`

Open
#1,152 0 comments 0 reactions 1 assignee Claimed by @SasSwart View on GitHub
Dominant language
No language data
Stars
3
Forks
0
PR merge metrics
No merged PRs in 30d

Description

We need to scaletest `aibridged` to assess its impact on the larger `coderd` process of which it is part.

The main resources of concern / likely culprits are:
- shared database (underlying resources like CPU, RAM, disk)
- shared database connection pool
- aibridge is a write-heavy application
- process file handles - since many network conns are made
- CPU/RAM
- possibly indirect impact due to increased allocs leading to increased GC
- goroutines inflight -> RAM growth, scheduler thrashing (manifesting as high CPU)
- in-memory datastrucures in https://github.com/coder/coder/blob/main/enterprise/aibridged/pool.go
- JSON marshaling causing high allocs
- response buffers either using or allocating large amounts of memory

The above list is non-exhaustive.

We must take care to control the full roundtrip: i.e. we must create a mock upstream which exhibits certain behaviours to provoke scaling concerns. For example, a pathologically slow upstream which leads to many requests inflight. The upstream will need to be a service in the scaletest package; see `cli/exp_scaletest_smtp.go` in coder/coder for an illustrative example.

The mock upstream must handle both streaming and blocking requests. You might consider reusing the fixtures from https://github.com/coder/aibridge/tree/main/fixtures.

Regular AI interactions + injected tool calls should be handled in order to exercise the majority of code paths.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.