finos / finos/architecture-as-code

Support items-based (zero-or-more) candidate catalogs in pattern decisions

Open
#2,859 0 comments 0 reactions 1 assignee Claimed by @YoofiTT96 View on GitHub
Dominant language
TypeScript
Stars
399
Forks
138
Avg merge
2d 14h
Merged PRs (30d)
37

Description

## Feature Proposal

### Target Project:
`shared` (Spectral pattern rules) and `calm-hub-ui` (pattern visualizer)

### Description of Feature:
CALM patterns already allow node/relationship candidates to be declared inside a `prefixItems` slot's `oneOf`/`anyOf` (positional tuple JSON Schema), and decisions (`relationship-type.options`) can reference any of those candidates by unique-id. This works well when a decision is "pick exactly one alternative to fill a specific slot."

It does not work for patterns that need to express "zero or more selections, in any combination, from an open catalog of candidate nodes" — e.g. a set of optional add-on nodes where any subset (including none, including skipping the middle one) may be present. JSON Schema already has the right construct for this: `items: { oneOf: [...] }` with `minItems: 0` validates a catalog of candidates matched against however many array elements are actually present, with no positional constraint. This is standard JSON Schema and requires **no change to the CALM meta-schema** — it already validates correctly today.

The gap is entirely in the tooling: pattern validation (Spectral rules in `shared/src/spectral/`) and pattern rendering (`calm-hub-ui`'s `patternTransformer.ts`) only ever look at `prefixItems`, so a pattern authored with `items`-declared candidates validates a subset of what it should, and renders no decision box at all.

### User Stories:
- As a pattern author, I want to declare a set of optional candidate nodes as an `items`-based catalog (rather than one `prefixItems` slot per candidate), so that my pattern accurately expresses "zero or more, any combination" instead of a fixed-position tuple that doesn't match the real architecture.
- As a pattern author, I want `calm validate` to catch duplicate unique-ids and unreferenced nodes inside an `items`-declared catalog, the same way it already does for `prefixItems`, so I get consistent validation regardless of which construct I use.
- As a calm-hub-ui user, I want a decision that references `items`-declared candidates to render as a visible decision box with its prompt and choices, instead of silently rendering nothing.

### Current Limitations:
- `shared/src/spectral/functions/pattern/ids-are-unique.ts` only scans `nodes.prefixItems[*]` / `relationships.prefixItems[*]` / interface `prefixItems[*]` for unique-id duplication — duplicate ids inside an `items.oneOf`/`anyOf` catalog currently pass validation silently.
- `shared/src/spectral/rules-pattern.ts`'s `pattern-nodes-must-be-referenced` rule's `given` selector is scoped to `nodes.prefixItems[*]` only, so an unreferenced `items`-declared node never triggers the existing "must be referenced" warning.
- `calm-hub-ui/src/visualizer/components/reactflow/utils/patternTransformer.ts`'s `getPrefixItems` helper (and everything downstream of it — `extractNodesFromPattern`, `extractRelationshipsFromPattern`) only ever reads `prefixItems`. A decision's metadata (`parsePatternData`'s `nodeToGroupMap`/`groupOptionsMap`) is only ever *attached* to a pre-existing node-level decision group built from `prefixItems` oneOf/anyOf wrapping — it never creates a new group, so a decision referencing only `items`-declared ids currently computes its metadata and then silently drops it, rendering no box at all.

### Minimal Example:
A pattern with two mandatory nodes (`webapp`, `database`) plus an open catalog of optional add-ons (`cache`, `queue`) declared via `items` instead of `prefixItems`, and a decision that lets an architecture instance pick any subset of those add-ons:

```json
{
"$schema": "https://calm.finos.org/release/1.0/meta/calm.json",
"properties": {
"nodes": {
"type": "array",
"prefixItems": [
{
"$ref": "https://calm.finos.org/release/1.0/meta/core.json#/defs/node",
"properties": {
"unique-id": { "const": "webapp" },
"node-type": { "const": "service" },
"name": { "const": "Web Application" },
"description": { "const": "Front-end web application" }
}
},
{
"$ref": "https://calm.finos.org/release/1.0/meta/core.json#/defs/node",
"properties": {
"unique-id": { "const": "database" },
"node-type": { "const": "database" },
"name": { "const": "Database" },
"description": { "const": "Primary data store" }
}
}
],
"items": {
"oneOf": [
{
"$ref": "https://calm.finos.org/release/1.0/meta/core.json#/defs/node",
"properties": {
"unique-id": { "const": "cache" },
"node-type": { "const": "service" },
"name": { "const": "Cache" },
"description": { "const": "Optional caching layer" }
}
},
{
"$ref": "https://calm.finos.org/release/1.0/meta/core.json#/defs/node",
"properties": {
"unique-id": { "const": "queue" },
"node-type": { "const": "service" },
"name": { "const": "Queue" },
"description": { "const": "Optional async messaging layer" }
}
}
]
},
"minItems": 2
},
"relationships": {
"type": "array",
"prefixItems": [
{
"$ref": "https://calm.finos.org/release/1.0/meta/core.json#/defs/relationship",
"properties": {
"unique-id": { "const": "webapp-to-database" },
"relationship-type": {
"const": {
"connects": {
"source": { "node": "webapp" },
"destination": { "node": "database" }
}
}
}
}
},
{
"$ref": "https://calm.finos.org/release/1.0/meta/core.json#/defs/relationship",
"properties": {
"unique-id": { "const": "optional-add-ons" },
"description": { "const": "Choose any optional add-ons for this deployment" },
"relationship-type": {
"properties": {
"options": {
"prefixItems": [
{
"anyOf": [
{
"properties": {
"description": { "const": "Add a caching layer" },
"nodes": { "const": ["cache"] },
"relationships": { "const": [] }
}
},
{
"properties": {
"description": { "const": "Add an async messaging queue" },
"nodes": { "const": ["queue"] },
"relationships": { "const": [] }
}
}
]
}
]
}
}
}
}
}
]
}
}
}
```

`webapp` and `database` are mandatory; `cache` and `queue` are an open catalog — zero, one, or both may appear, in any combination. The `optional-add-ons` decision lets an architecture instance pick either, both, or neither add-on.

**Observed today:**
1. **Validation gap** — duplicate a `unique-id` inside `nodes.items.oneOf` (e.g. give `queue` the id `cache` by mistake) and run `calm validate`: `unique-ids-must-be-unique-in-pattern` does not catch it, because its JSONPath queries only scan `nodes.prefixItems[*]`. The same duplicate inside a `prefixItems` slot is correctly rejected.
2. **Rendering gap** — load this pattern in calm-hub-ui: `webapp`, `database`, and the connecting edge render correctly, but the `optional-add-ons` decision box never appears, and `cache`/`queue` are never drawn at all. `patternTransformer.ts`'s extraction (`getPrefixItems`) only reads `prefixItems`, so the `items`-declared candidates and the decision's prompt/choices are silently dropped rather than rendered.

### Proposed Implementation:
- **`ids-are-unique.ts`**: extend the three JSONPath queries to also cover `items.oneOf[*]`/`items.anyOf[*]` equivalents for nodes, relationships, and node interfaces.
- **`rules-pattern.ts`**: widen the `given` selector on `pattern-nodes-must-be-referenced` to also walk `nodes.items.oneOf[*]`/`anyOf[*]`. No function change needed — `node-has-relationship.ts`'s function body already scans the whole document unscoped.
- **`patternTransformer.ts`**:
- Extend node/relationship extraction to also read `items.oneOf`/`anyOf` candidates (in addition to the existing `prefixItems` path), producing `ExtractedNode`/`ExtractedRelationship` entries the same way `prefixItems` candidates do today.
- In `parsePatternData`, change the decision-to-group mapping from "look up the first existing group that matches" to: fold any decision-referenced ids not already in a group into the matching group (if one exists), or create a new decision group from the decision's own id list if none of its referenced ids match an existing group. Only fold in/create using ids that resolve to a real extracted node — ids that don't resolve to anything (e.g. a typo) must be skipped so a dangling reference still renders as nothing, matching today's behavior, rather than producing an empty box.
- Thread the options-relationship's own `unique-id` through `OptionsMetadata` so a newly created group has a stable, deterministic id (rather than one derived from array index).

### Alternatives Considered:
A new CALM vendor-extension construct for expressing optional candidate sets was considered and rejected: decisions already reference candidates by unique-id, and `items` already expresses "zero or more, any combination" natively in JSON Schema. No schema change is needed; this is purely a tooling gap.

### Testing Strategy:
- `shared`: new Spectral rule-function unit tests for `ids-are-unique` and `pattern-nodes-must-be-referenced` covering `items`-declared duplicate/unreferenced ids; confirm `nodes-referenced-in-pattern-decision-must-be-in-oneof-or-anyof-block` and the two options-shape rules are unaffected (existing tests should continue to pass unchanged).
- `calm-hub-ui`: new fixtures/tests in `patternTransformer.test.ts` covering (a) a decision referencing only `items`-declared candidates (new group created), (b) a decision referencing a mix of `prefixItems`- and `items`-declared candidates (fold-in), (c) a decision referencing a dangling/typo'd id (must still render nothing, not an empty box). Confirm the existing `handles options metadata on decision groups` test is unaffected.
- Since `shared` is a dependency of `cli` and the VSCode extension, run the full `npm test` from repo root after the `shared` changes, not just `--workspace shared`.

### Documentation Requirements:
- Update pattern-authoring docs to describe `items`-based optional candidate catalogs as a supported alternative to `prefixItems`-per-slot, with a worked example.
- Release notes: call out the one behavior change with real teeth — duplicate unique-ids inside an `items` catalog, previously silent, now fail `calm validate` with an error. The "must be referenced" extension is a `warn`-level addition and lower priority to call out but should still be mentioned.

### Implementation Checklist:
- [ ] Design reviewed and approved
- [ ] Implementation completed
- [ ] Tests written and passing
- [ ] Documentation updated
- [ ] Relevant workflows updated (if needed)
- [ ] Performance impact assessed

### Additional Context:
No CALM meta-schema changes are required — `items` is already valid, already-supported JSON Schema. This is entirely a tooling gap between what CALM's schema foundation already permits and what `shared`'s Spectral rules and `calm-hub-ui`'s renderer currently understand.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.