agent-substrate / agent-substrate/substrate

[Design] Filtered ListActors

Aberta
#1,256 0 comentários 0 reações 0 responsáveis Ver no GitHub
area/api area/api-machinery kind/feature
Linguagem predominante
Go
Estrelas
1.8k
Forks
316
Merge médio
2d 43min
PRs com merge (30d)
287

Descrição

Splitting this out of #796, since it's a bigger topic than the original one about the metric.

**Please use this document for the discussion to keep everything in the same place**, then I'll sumamize the discussion and decisions on this issue: https://docs.google.com/document/d/1CqWlPjgF6NHmZJJvs5t8EcrqIvFWO60BKvxos7Q61So/edit?tab=t.0#heading=h.fmhjlo7dlovg

## Rationale

Operators will want to answer questions like:

- show me every actor currently in CRASHED, so I can inspect or clean them up
- show me actors of template X that are stuck in DELETING
- list actors in an atespace by state, e.g. for `kubectl-ate` view

These are ad-hoc queries against actor state, not strictly telemetry, so let's design it holistically.

## Current state

- `ListActors` takes an atespace, a page size and a page token
- The only way to answer any of the above is to page the entire fleet and filter client side, decoding every actor proto

## Requirements

- We are targeting 1B actors, so we cannot just list everthing and filter later
- We need a design where cost scales with total actors instead of the matches

## Rough ideas

- projected columns + indices on the fields we filter by
- secondary index tables maintained during writes
- a filtered watch, so consumers subscribe to a slice instead of everything
- keeping it out of the API entirely and answering these from a downstream consumer

These have a diff cost on the write path.

## Scope for this issue/design discussion

Let's agree on the requirements and the access patterns first, e.g.
- which fields we actually need to filter on
- do watch needs filtering too or only list
- what latency and consistency operators expect

cc @jpbetz @juli4n @maxsmythe @EItanya @git286 @JeffLuoo

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.