finos / finos/architecture-as-code
Referencing documents in CalmHub
- Dominant language
- TypeScript
- Stars
- 399
- Forks
- 138
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 37
Description
## Feature Proposal
### Target Project:
shared, cli
### Description of Feature:
#### Current situation
The CLI has the capability to resolve references against a CalmHub configured in its settings.
See #887 #1209 #1227 #1243 #1784 #1836 . The document-loader currently expects to resolve URL references of the form `calm://` as being a suffix that can be appended to a configured CalmHub URL.
#### Proposal
This feature proposal discusses what that suffix should be, and if documents the CLI is handling should directly contain `calm://` or whether the CLI should attempt to locate documents referenced with other forms in CalmHub, if a CalmHub is configured in the CLI user settings or command line.
Consider that artefacts may contain the following references (list may be non-exhaustive)
* Architectures have $schema attributes which may be the CALM schema or a pattern
* Architectures have references to:
* other architectures (`detailed-architecture`) and `required-pattern`
* to adrs (`/adrs/*`)
* to control `requirement-url`, and `config-url`
* to flow `requirement-url`
* to interface `definition-url`
* Patterns can reference Standards via `$ref` attributes.
What should these references look like?
#### Decision Drivers
The following properties of references have been considered:
1. [calm-resolvable] References MUST be resolvable by the CALM suite of products
* Goes without saying, CALM tools must be able to resolve CALM references
2. [non-calm-resolvable] References, especially $schema, SHOULD resolve with non-CALM tooling, e.g. a standard JSON validator.
* This is a usability benefit. Consider both artifacts that a user is actively editing on a desktop in VSCode etc, processing using `jq`, and those viewable in a Git repo.
3. [no-edits-during-copy] References within an artefact SHOULD NOT require updating if it and any artefacts it references are published to CalmHub from a Git Repo, or copied from one CalmHub to another.
* If a pattern exists on FINOS CalmHub and makes reference to controls on FINOS CalmHub, it should be possible to copy these artifacts to an internal Organization CalmHub without modification.
4. [no-edits-on-copy] References to a CalmHub artefact SHOULD NOT require updating if the referenced artefact is copied to another CalmHub.
* A slight extension of [3], suggesting that an artifact not in CalmHub might refer to an artifacts canonical (FINOS) CalmHub location, despite it being resolved against a local (Organizational) one.
5. [no-undue-restriction-on-api] References to CalmHub artifacts MUST NOT rely on paths that may change.
* As soon as an CalmHub API path is referenced by an artifact, it must remain supported unless there is a breaking change between CalmHub and all published artifacts.
#### Option 1 - `calm://calm/namespace////version/`
* Requires CLI and VSCode support for clickable extension.
* Clear what is in CalmHub or Not, therefore clear what is a published immutable artifact
* Sets in stone the API structure of CalmHub
#### Option 2 - `https://calmhub.example.org/calm////version/`
* Only ever resolves against the original CalmHub location.
* May fail if firewall/proxy in way
* Tooling could attempt to replace domain name with local CalmHub - what if artifacts are available in multiple CalmHub, e.g. copied from calmhub.finos.org to calmhub.myorg.com?
* Still sets in stone the API structure of CalmHub
* `` must be restricted to never be the value of any existing API path under `/calm/` (and we can never add any either)
* Is there a point of the domain in this situation... May as well do calm:// ?
#### Option 3 - `calm:///`
* Arbitrary ID, which has is not resolvable.
* Parseable to namespace, type, name, version.
* Tools are required to translate this to a resolvable location, in a CALMhub.
#### Option 4 - `https://example.com/patterns/my-pattern.json`
* Is either the location of a document not in a CalmHub, or is the $id of a document (hopefully) in the configured CalmHub
* If the former, the CLI can just access the document
* If the latter, CalmHub needs a method to allow the CLI etc to search for documents based on an $id.
* Does allow documents to be located anywhere, but may optionally be published to a local CalmHub (potentially inside an organisation), but with no change to any references.
* Tooling needs to attempt a CalmHub access prior to a Web access (may be a good thing for organizations)
#### Summary
| Criteria | Option 1 | Option 2 | Option 3 | Option 4 |
|------|-----|-----|-----|-----|
| calm-resolvable | yes | yes | yes | yes |
| non-calm-resolvable | no | yes | no | yes |
| no-edits-during-copy | yes | no | yes | yes |
| no-edits-on-copy | yes | no | yes | yes |
| no-undue-restriction-on-api | no? | no? | yes | yes |
Much of this comes down to whether an organisation's CalmHub can transparently resolve artifacts authored and published elsewhere (Web, another CalmHub) like a proxy-cache, or whether all users of CLI etc need to maintain a URL mapping file of every document they reference, especially if it's not from the local CalmHub.
### User Stories:
* As a pattern author, I need to reference standards in $ref attributes that a later `calm validate` can load.
* As an architect, I need to reference patterns, controls and interface definitions. This may be in a CalmHub, or as a JSON document on any other website.
* As a CalmHub maintainer in an organisation, I may need to make available in my org's CalmHub patterns, controls etc, previously published in another CalmHub, in a way that requires minimal or zero change to nested references within those artifacts. For example, if I need to copy a pattern and all controls it references, the references either need to resolve against local CalmHub transparently, or be changed.
### Current Limitations:
The current functionality uses Option 1
### Proposed Implementation:
...provide details of the intended implementation approach, including:
- Technical design considerations
- API changes (if applicable)
- Data model changes (if applicable)
- Dependencies on other components
### Alternatives Considered:
...what other approaches were considered and why were they rejected?
### Testing Strategy:
...how will this feature be tested? Include unit, integration, and any other testing considerations...
### Documentation Requirements:
...what documentation will need to be created or updated?
### Implementation Checklist:
- [ ] Design reviewed and approved
- [ ] Implementation completed
- [ ] Tests written and passing
- [ ] Documentation updated
- [ ] Relevant workflows updated (if needed)
- [ ] Performance impact assessed
### Additional Context:
...add any other context, diagrams, mockups, or screenshots about the feature request here...
Contributor guide
Research direction
Start with the CLI document-loader and its current Option 1 behavior, then trace how calm:// references are resolved and how calm validate loads $ref and other URLs. Compare the four options against the stated decision drivers and user stories; done requires an approved design covering implementation, testing, and documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100