agent-substrate / agent-substrate/substrate

Deliver egress policy efficiently to enforcement points

Aperta
#1,323 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
area/network kind/feature
Lingua principale
Go
Stelle
1.8k
Fork
316
Merge medio
2g 43m
PR unite (30g)
287

Descrizione

## Summary

We need an efficient way to deliver Actor egress policy to policy enforcement points (PEPs).

Fetching the Actor and policy for every CONNECT is simple, but adds control-plane latency and database load to every request. Distributing every policy eagerly is also undesirable because a gateway may only see a small portion of all Actors.

The delivery mechanism must also propagate policy and Actor lifecycle changes so the PEP can terminate tunnels authorized by an obsolete policy.

## Requirements

- Load policy only when a PEP receives traffic for an Actor.
- Avoid control-plane reads for requests after the initial lookup.
- Bind policy to the Actor UID, not only its name.
- Propagate policy and relevant Actor lifecycle changes.
- Terminate tunnels when their policy generation is replaced or removed.
- Bound subscriptions and cached state at each PEP.
- Deliver one atomic effective policy, even if multiple policy documents are supported later.
- Work across multiple PEP and `ate-api` replicas.

## Approaches

### Unary lookup for every CONNECT

The PEP calls an internal API that resolves Actor eligibility and policy in one query.

This is the simplest approach and every new CONNECT observes current state. However, every request pays the RPC and database cost, and an established tunnel cannot be revoked without adding another mechanism.

### Unary lookup with a local cache

The PEP caches unary responses for a short period and coalesces concurrent misses.

This substantially reduces reads and is straightforward to implement. Its main limitation is update propagation: policy changes are only observed after refresh or expiry. Short refresh intervals increase database load, while longer intervals delay revocation.

### Custom bidirectional stream

The PEP opens one stream and sends typed subscribe and unsubscribe requests for Actors as they are encountered. The server pushes effective-policy replacements and removals.

This provides on-demand loading and prompt updates with a domain-specific API. It can use typed Actor references and explicit per-policy acknowledgements, but it requires us to define our own subscription, versioning, reconnection, and flow-control protocol.

### Delta xDS

The PEP uses one Delta xDS stream and subscribes to named Actor policy resources on demand. Resource updates and `removed_resources` propagate changes.

Delta xDS already defines subscriptions, versions, acknowledgements, removals, and reconnection behavior. It could also become a shared delivery mechanism for other runtime configuration. The tradeoff is that the PEP is not Envoy, so we still implement the client-side cache and enforcement behavior, and Actor identity must be encoded into resource names.

## Proposed direction

Use on-demand Delta xDS.

When the first CONNECT for an Actor arrives:

1. Authenticate the Actor certificate.
2. Derive a resource name from the certified Atespace, Actor name, and Actor UID.
3. Subscribe or join an existing pending subscription.
4. Wait for the initial policy response.
5. Cache the result and retain the subscription for updates.

Subsequent requests use the cached policy without another control-plane lookup.

Each PEP maintains one stream, coalesces concurrent misses, and removes idle subscriptions through a bounded LRU cache. Active tunnels pin their policy entry.

## Effective policy resource

The PEP receives one resolved policy resource:

```proto
message EffectiveEgressPolicy {
repeated ateapi.EgressRule rules = 1;
}
```

If multiple policy documents are supported later, `ate-api` combines them into one ordered rule list. A change to any contributing document creates a new effective-policy generation.

The resource name identifies the Actor and includes its UID. The resource version represents Actor eligibility and all contributing policy document UIDs and versions.

## Updates and revocation

A resource is available only while the Actor exists, its UID matches, it is running, and it has an egress policy.

Policy changes and relevant Actor lifecycle changes update or remove the resource. The PEP associates every tunnel with the policy generation that authorized it. Replacing or removing that generation prevents new use and terminates its existing tunnels.

Policy and Actor mutations must produce transactional change events. Each delivery server resolves changed resources once and fans them out only to interested PEPs.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.