JanssenProject / JanssenProject/jans
feat(jans-cedarling): unify token entity-type representation between JWT and custom issuers
- Dominant language
- Java
- Stars
- 647
- Forks
- 173
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 110
Description
# Unify token entity-type representation between JWT and custom issuers
## Summary
JWT issuers and custom (non-JWT) issuers describe per-token metadata in two
different shapes:
- **JWT** (`TrustedIssuer`): `token_metadata: HashMap`
where the key is a *token key* from a fixed vocabulary (`"access_token"`,
`"id_token"`), and the Cedar entity type is a separate field
`TokenEntityMetadata::entity_type_name` (`token_entity_metadata.rs:24`,
`policy_store.rs:193`).
- **Custom** (`CustomIssuerMetadata`): `tokens_mappings: HashMap`
where the key **is** the Cedar entity type (`custom_issuer_metadata.rs:19-29`).
This asymmetry is historical, not accidental: custom issuers key by the mapping
as a single source of truth so duplicate types within one issuer are
unrepresentable (`custom_issuer_metadata.rs:22-28`).
Today the JWT side pays for the asymmetry with a reverse scan: the public request
identifies tokens by Cedar entity type (`TokenInput.mapping`, `authz/request.rs:63-65`),
so the pipeline maps entity type → token key via `find_token_metadata_key`
(`issuer_index.rs:62-75`), an O(n) scan over all issuers with a TODO to index it.
## Proposal
### Phase 1 — non-breaking prep (can land independently)
- Introduce an `EntityTypeName` newtype that **wraps `cedar_policy::EntityTypeName`**
and parses at parse time, used as `TokenEntityMetadata::entity_type_name` and as the
`tokens_mappings` key type.
- Rationale: the request side already parses `TokenInput.mapping` through
`cedar_policy::EntityTypeName::from_str` (`authz/mod.rs:1231-1234`), while the
config side only does an empty-check (`issuer_parser.rs:171`,
`custom_issuer_parser.rs:71`, `custom_token.rs:247-259`). Wrapping the Cedar type
makes both sides agree on what a valid entity type is, and removes the duplicated
empty-check.
- The wrapper exists so we can attach our own error type / `Deserialize` and avoid
colliding with the widely-imported `cedar_policy::EntityTypeName`.
- **Do not** share a claim-enforcement type between the two structs. The two
`required_claims` sets only share a name; their validation domains differ:
- JWT: presence **plus** RFC 7519 registered-claim value validation
(`token_entity_metadata.rs:29-30`, `trusted_issuer_validator.rs:118-135`).
- Custom: presence only in processor output; Cedarling does not validate the
custom payload (`jwt/mod.rs:706-710`).
They are expected to diverge, so a shared `TokenRequirements` struct would couple
their evolution for no real DRY win (the validation functions already differ).
Keep separate fields; at most share a tiny stateless presence-check helper, and
only if it removes actual duplication.
- Do **not** merge the value structs: `token_id` is JWT-only (custom processors
supply it) and the `required_claims` semantics differ (above). `trusted` / `required`
are removed in Phase 2, not unified.
### Phase 2 — key JWT `token_metadata` by entity type (breaking)
Blocked by the format-versioning issue.
- Change `token_metadata` to `HashMap` and drop
the `entity_type_name` field.
- Remove `find_token_metadata_key` (`issuer_index.rs:62`) and the `jwt/mod.rs:797`
wrapper; lookups become direct O(1).
- `determine_token_entity_type` (`build_multi_issuer_entity.rs:254-278`) becomes a
direct lookup.
- `token.name` is already the mapping in the multi-issuer path
(`jwt/mod.rs:620-624`), so the request model does not change; only the config
shape does.
- Agama YAML (`LegacyTrustedIssuer`, `legacy_store/mod.rs:54-97`) carries the same
shape. Per the versioning issue it is test-only: change it directly and fix its
fixtures in the same PR — it gets no migration.
#### Rename the custom issuer's `tokens_mappings` to `token_metadata`
Once Phase 2 lands, both issuer kinds hold "token metadata keyed by Cedar entity
type", so one name for one concept:
`custom-issuers/*.json` → `tokens_mappings` becomes `token_metadata`.
- The remaining value-struct difference is small (`token_id` is JWT-only; `trusted` /
`required` are gone), and the two live in different directories, so the shared name
is not misleading.
- Breaking, but trivial: a key rename in the custom issuer files, in the same format
bump as everything else here. The runtime field on `CustomIssuerMetadata` and the
index copy (`custom_token.rs:169`) rename with it.
- Struct names (`CustomIssuerMetadata` / `CustomTokenMetadata`) stay as they are.
#### Remove `trusted` (JWT) and `required` (custom)
Both are per-token booleans, and both go away in the same format bump.
| Field | Store JSON location | Structs holding it |
|---|---|---|
| `trusted` | `trusted-issuers/*.json` → `token_metadata..trusted` | `TokenEntityMetadata` (`token_entity_metadata.rs:19-22`); Agama: `LegacyTokenEntityMetadata` (`legacy_store/mod.rs:54-63`) |
| `required` | `custom-issuers/*.json` → `token_metadata..required` (see the rename above) | `CustomTokenMetadata` (`custom_issuer_metadata.rs:34-38`); Agama reuses the same struct (`legacy_store/mod.rs:453,518`) |
- **`trusted`** — default `true`. Its only effect: `trusted: false` skips validator creation
(`validator_cache.rs:62`), so a token of that type fails validation and is skipped.
The docs describe it as a way to disable an entry without deleting it — that is
config commenting-out disguised as a field. To disable a token type, remove the
entry.
- **`required`** — default `false`. `true` turns a custom-token processing failure/timeout into a hard
authorization error instead of skip-and-continue
(`CustomIssuerIndex::mapping_required`, `custom_token.rs:302`; call sites
`jwt/mod.rs:488,556`). The JWT side has no such switch: an invalid JWT is always
logged and skipped (`jwt/mod.rs:648-657`). After removal both paths behave the same.
Migration:
- `trusted: false` → drop the **whole** token entry, not just the field; behavior is
preserved.
- `required: true` → behavior changes (hard error → token skipped, request continues).
Migration emits a warning (warning enum from the versioning issue), and the upgrade
guide must call it out.
#### Every token becomes optional — what a failed token means
After this change JWT and custom tokens behave the same (JWT already does today):
- A token that fails validation/processing is logged (WARN) + counted in metrics and
**dropped**; the request continues (`jwt/mod.rs:648-657`).
- If **no** token survives → `TokenValidationFailed` error (`jwt/mod.rs:458-466`).
- If some survive → authorization runs without the dropped ones: they are absent from
`context.tokens`, and `total_token_count` is lower. Multi-issuer has no principal
(`authz/mod.rs:291-296`), and the recommended schema makes tokens optional, so
nothing else catches the absence.
- The caller cannot see what was dropped: `MultiIssuerAuthorizeResult` carries only
`response`, `decision`, `request_id`. Only the log knows.
Impact on policies:
- `permit` that needs a dropped token → does not fire (via `has` → `false`, or via
direct access → evaluation error → policy skipped) → deny. Safe.
- **`forbid` that reads a dropped token → does not fire either → can flip the decision
to ALLOW** if another `permit` holds. This is the fail-open case.
Fail-closed for a mandatory token moves into policy, e.g.:
```cedar
forbid (principal, action, resource)
unless { context has tokens._ };
```
#### Validation moves to runtime conversion
`validate_trusted_issuers_config` (`policy_store.rs:104-140`) currently enforces that
each Cedar entity type is owned by exactly one token **across all issuers**, not just
within one. After Phase 2 the within-one-issuer half becomes unrepresentable, but the
**cross-issuer check must survive**. Move it into the parse → runtime conversion /
init step, where the full issuer set is known.
#### Precompute what is currently derived per request
`simplify_token_type` (`build_multi_issuer_entity.rs:65`) downgrades an entity type to
a lowercased token-ish string (`Acme::DolphinToken` → `dolphintoken`) and is called on
the request path (`build_multi_issuer_entity.rs:515`, `custom_token.rs:251`). It is
already the de-facto answer to "what is the human-readable token kind" once the token
key is gone.
Compute it once during conversion to the runtime type and store it alongside the
metadata, instead of recomputing per request. That also gives logs, telemetry and the
validator-cache key (`validator_cache.rs:61`, `trusted_issuer_validator.rs:124`) a
stable string without reintroducing a configured token key.
Note: `token_kind` is only ever a lookup/cache key (`jwt/mod.rs:285-293`) — no
validation behavior is selected by the token key's *value*, so dropping the configured
key is safe on that axis.
#### `default_tkn_entity_name` probably goes away
`default_tkn_entity_name` (`entity_builder/mod.rs:214-221`) matches *token keys*
(`"access_token"`, `"id_token"`, `"userinfo_token"`). After Phase 2 there are no token
keys in config, and it only survives as a fallback for a request that sends a bare
mapping with no `::`. Likely deletable — decide during implementation whether that
fallback is still wanted.
### Why not fully merge into one struct
- Different map-key semantics (token selector vs entity type).
- Different fields (`token_id` is JWT-only) — still true after `trusted` /
`required` are removed.
- Breaking, user-facing policy-store JSON change.
- Different lifecycles (JWT validator pipeline vs `CustomTokenProcessor`).
## Open questions
- [ ] Is `mapping` allowed to differ from the configured entity type? Today it can
(reverse lookup); after Phase 2 it must be equal. Confirm this is desired.
- [ ] How does this interact with #14747 (custom `entity_type_name` assumed globally
unique, `custom_token.rs:69-72,179-185,300-320`)? If #14747 keeps a runtime
discriminator, does it affect the key choice?
- [ ] Deprecation window for the old key shape, and the exact migration mapping.
- [ ] Keep `default_tkn_entity_name` as a bare-mapping fallback, or delete it?
## Identified code changes
Phase 1:
- [ ] Add `EntityTypeName` newtype wrapping `cedar_policy::EntityTypeName`; use it in
both metadata structs; parse-time validation.
- [ ] Remove the duplicated empty-check validation.
- [ ] Decision: keep `required_claims` as separate fields (documented rationale); no
shared claim-enforcement type.
Phase 2 (after format versioning):
- [ ] Change `TrustedIssuer::token_metadata` key type; drop `entity_type_name`.
- [ ] Remove `find_token_metadata_key` / `IssuerIndex::find_token_metadata_key`.
- [ ] Simplify `determine_token_entity_type`.
- [ ] Remove `trusted` from `TokenEntityMetadata` and `LegacyTokenEntityMetadata`
(+ the skip branch in `validator_cache.rs`).
- [ ] Remove `required` from `CustomTokenMetadata` (covers Agama too)
(+ `CustomIssuerIndex::mapping_required` and its call sites).
- [ ] Rename `CustomIssuerMetadata::tokens_mappings` to `token_metadata` (parse +
runtime + `CustomIssuerIndex`).
- [ ] Move the cross-issuer duplicate-entity-type check into runtime conversion.
- [ ] Precompute `simplify_token_type` at conversion time.
- [ ] Update the dir-format issuer parser and the Agama legacy parser/converter.
- [ ] Re-evaluate `default_tkn_entity_name`.
- [ ] Update all fixtures that use `token_metadata` keys + `entity_type_name`
(`loader_tests.rs`, `manager.rs`, `test_utils.rs`, `issuer_parser.rs`,
`validator.rs`, `jwt/mod.rs` tests).
## Tests
- [ ] JWT `required_claims`: presence + RFC 7519 registered-claim validation.
- [ ] Custom `required_claims`: presence enforced on processor output.
- [ ] Invalid entity type name (bad Cedar syntax, not just empty) is rejected at parse.
- [ ] Duplicate entity type within one issuer is unrepresentable; across issuers is
rejected at conversion.
- [ ] Multi-issuer lookup by mapping works with non-default entity type names.
- [ ] Migration from the old key shape produces an equivalent runtime store.
- [ ] Migration: `trusted: false` entry is dropped; `required: true` emits a warning.
- [ ] Custom-token processing failure/timeout is skipped, not a hard error (same as JWT).
## Docs
This is the first real breaking format change, so it is also the first entry in the
upgrade guide introduced by the versioning issue.
- [ ] **Upgrade guide entry** (`docs/cedarling/reference/cedarling-policy-store.md`,
"Format versions and upgrading"): what changed in `token_metadata`, why, a
before/after trusted-issuer example, and step-by-step how to rewrite an existing
store (move `entity_type_name` into the key, drop the old token key, bump
`format_version`). Call out the new constraint: request `mapping` must equal the
configured entity type.
- [ ] Same entry: the custom issuer `tokens_mappings` → `token_metadata` rename,
removal of `trusted` (how to disable a token type now: delete the entry) and of
`required` (failed custom token is skipped; policies decide).
- [ ] Update the `token_metadata` schema/examples on the same page (Trusted Issuer
Files, Trusted Issuers Schema, Token Metadata Schema) and the custom issuer
section (`required`, `tokens_mappings` → `token_metadata`).
- [ ] Update other pages and tutorials with `token_metadata` examples, so no docs show
the old shape (`cedarling-multi-issuer.md`, `cedarling-jwt-validation.md`,
`tutorials/*`, `integrations/terraform-authz-jwt.md`).
- [ ] Agama Lab generator emits the new shape (external follow-up).
Token-failure semantics (`cedarling-multi-issuer.md`, and linked from the upgrade guide
entry for `required`):
- [ ] What happens when a token fails: dropped + WARN; all failed → error; partial →
authorization continues without it; dropped tokens are visible only in logs.
- [ ] How to make a token mandatory now: the `forbid … unless { context has tokens.… }`
pattern — the replacement for `required: true`.
- [ ] Authoring rule: a `forbid` that reads a token must use `has` and be written so
that a missing token cannot turn the decision into ALLOW. Explain why (erroring
or `false` forbid conditions simply do not fire).
## Dependency
Phase 2 requires the format-versioning infrastructure (separate issue) so the
breaking key change has a versioned migration path for the directory/`.cjar` format.
Agama YAML is test-only and is changed in place.
Contributor guide
Assessment
This issue has not been assessed yet.