AltimateAI / AltimateAI/altimate-code

Events dropped due to telemetry buffer overflow under heavy tool usage

Abierto
#449 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
TypeScript
Estrellas
811
Forks
134
Merge medio
3 d 2 h
PR fusionados (30 d)
50

Descripción

## Problem

During sessions with high tool call volume, the event buffer overflows and drops events silently. The error message logged is:

```
N events dropped due to buffer overflow
```

This means event counts are undercounted for power users, and we may be missing important failure/error events during the heaviest usage periods.

## Impact by version

| CLI Version | Occurrences | Total Events Dropped | Avg Dropped/Occurrence | Max Single Drop |
|-------------|-------------|----------------------|------------------------|-----------------|
| `0.5.7` | 167 | 36,968 | 221 | 3,901 |
| `0.5.1` | 43 | 7,614 | 177 | 706 |
| `0.5.3` | 14 | 2,691 | 192 | 646 |
| `0.5.2` | 9 | 931 | 103 | 290 |
| `0.5.4` | 7 | 511 | 73 | 283 |
| `0.5.5` | 2 | 9 | 5 | 5 |

**Total: ~48,800 events dropped across all versions in the last 7 days.**

## Observations

- The problem is **getting worse with newer versions** — `0.5.7` accounts for 75% of all drops
- The worst single overflow dropped **3,901 events** in one batch
- Overflows correlate with sessions that have very high `tool_call` volume (1,000+ calls in a session), especially `sql_execute` and `todowrite` heavy workflows
- `0.5.5` had the fewest drops (avg 5), suggesting something regressed in `0.5.7`

## Suggested investigation

1. Check the telemetry buffer size and flush interval — the current settings can't keep up with heavy `sql_execute` loops
2. Consider increasing buffer capacity or flushing more aggressively
3. Consider back-pressure or sampling instead of silent drops
4. Investigate why `0.5.7` is significantly worse than `0.5.5`

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.