finos / finos/architecture-as-code
Authentication Support for Direct URL Document Loading
- Dominant language
- TypeScript
- Stars
- 399
- Forks
- 138
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 37
Description
### Target Project:
`architecture-as-code` monorepo, primarily affecting:
* `@finos/calm-cli`
* `@finos/calm-shared`
### Description of Feature:
Add authentication support to the `DirectUrlDocumentLoader` so CALM can retrieve **read-only CALM artifacts** from HTTP/HTTPS repositories that require authentication.
The primary goal is to let organizations use CALM with artifacts that are already stored in existing authenticated repositories, without first moving those artifacts into CALM Hub.
The two main use cases are:
* Retrieve a CALM artifact from an authenticated repository
* Use `calm validate` on artifacts stored in an authenticated repository
This feature is limited to **read-only retrieval of CALM artifacts** through `DirectUrlDocumentLoader`. It does not propose CALM Hub features such as publishing, visualization, versioning, diff, timeline, or repository management.
Authentication for direct URL access should remain separate from the existing CALM Hub authentication path. If direct URL authentication is not configured, `DirectUrlDocumentLoader` should continue to behave exactly as it does today.
Any support added to the `architecture-as-code` repository should be based on **industry-standard authentication mechanisms and extension points**, not on embedding end-user organization Identity Provider (IDP) implementations in this repository. Organization-specific IDP implementations should remain outside the `architecture-as-code` repo.
Related Issue #2944
### User Stories:
As a CALM user, I want to retrieve a CALM artifact from an authenticated repository so that I can work with architecture content already stored in my organization.
As a CALM user, I want to run `calm validate` against artifacts stored in an authenticated repository so that I can validate those artifacts without copying them to another location first.
As an enterprise user, I want CALM CLI to work with my organization’s existing authenticated repository so that adopting CALM does not require changing existing repository or access-control arrangements.
As an automation or CI/CD user, I want `calm validate` to retrieve CALM artifacts from a protected repository using non-interactive authentication so that validation can run in pipelines.
As an interactive CLI user, I want CALM CLI to retrieve protected CALM artifacts using my organization’s authentication flow so that I can validate or inspect those artifacts from my local environment.
As an organization adopting CALM Hub while continuing to use an existing repository, I want direct URL authentication and CALM Hub authentication to remain independently configurable so that each can use its own credentials and access model.
### Current Limitations:
`DirectUrlDocumentLoader` currently performs unauthenticated HTTP requests. As a result, CALM artifacts stored in authenticated repositories cannot be retrieved through direct URLs.
This prevents organizations from using CALM directly against protected repositories that already contain relevant artifacts unless they make those artifacts public or relocate them.
There is currently no direct way to use `calm validate` on artifacts that are only available through authenticated direct URLs.
The existing `authPluginPath` mechanism is associated with CALM Hub authentication and does not address the separate need to retrieve protected artifacts through `DirectUrlDocumentLoader`.
### Proposed Implementation:
Introduce authentication support specifically for `DirectUrlDocumentLoader`.
The CLI should allow users to configure authentication for direct URL access independently from CALM Hub authentication.
When configured, `DirectUrlDocumentLoader` should apply the necessary authentication to retrieve protected CALM artifacts over HTTP/HTTPS.
This capability should support the two main use cases in this feature request:
* Retrieving a CALM artifact from an authenticated repository
* Running `calm validate` on artifacts stored in an authenticated repository
The feature should remain read-only. It should only enable retrieval of artifacts needed by CALM CLI and should not introduce repository write operations.
The implementation should preserve backward compatibility. When direct URL authentication is not configured, existing unauthenticated direct URL behavior should remain unchanged.
The implementation should also preserve separation from the existing CALM Hub authentication path so that authenticated direct URL retrieval and CALM Hub access can be configured independently.
Any changes made in the `architecture-as-code` repo should focus on industry-standard protocol support and stable extension points. End-user organization IDP implementations, proprietary authentication logic, and vendor-specific integrations should be kept outside this repository.
### Alternatives Considered:
**Require users to make protected artifacts public.** This would weaken existing access controls and is not acceptable for many enterprise repositories.
**Require users to migrate artifacts into CALM Hub first.** This creates an unnecessary adoption barrier for organizations that already store CALM artifacts in an authenticated repository.
**Reuse the existing CALM Hub authentication mechanism directly.** This would couple two different access scenarios that may need different credentials, policies, or operational ownership.
**Continue supporting only unauthenticated direct URLs.** This does not address the main adoption problem for organizations whose repositories already require authentication.
**Use existing authenticated methods to download CALM artifacts and run `calm validate` with `url-mapping`, if needed, on the downloaded files.** Complicates the workflow. This would be the fallback if this feature request does not workout.
### Testing Strategy:
Unit tests should verify that authentication can be applied to `DirectUrlDocumentLoader` requests and that existing unauthenticated behavior still works when no configuration is provided.
`DirectUrlDocumentLoader` tests should verify that protected CALM artifacts can be retrieved when authentication is configured and cannot be retrieved when required authentication is missing or invalid.
Integration tests should verify at minimum:
1. A CALM artifact stored in an authenticated repository can be retrieved through `DirectUrlDocumentLoader`.
2. `calm validate` can validate an artifact stored in an authenticated repository.
3. Direct URL authentication and CALM Hub authentication can be configured independently in the same execution.
4. Existing configurations without direct URL authentication continue to operate unchanged.
### Documentation Requirements:
Update the CALM CLI documentation to describe how authentication can be configured for `DirectUrlDocumentLoader`.
Document the primary supported use cases:
* Retrieve a CALM artifact from an authenticated repository
* Use `calm validate` on artifacts stored in an authenticated repository
Provide examples showing how direct URL authentication is configured separately from CALM Hub authentication.
Document that this feature is limited to read-only retrieval of CALM artifacts and does not add CALM Hub-style repository capabilities.
Document that organization-specific IDP implementations must live outside the `architecture-as-code` repository, and that changes within this repository should remain standards-based rather than encoding end-user-specific implementation details.
### Implementation Checklist:
* [ ] Design reviewed and approved
* [ ] Define direct URL authentication configuration
* [ ] Add authentication support to `DirectUrlDocumentLoader`
* [ ] Preserve independent CALM Hub authentication behavior
* [ ] Implementation completed
* [ ] Unit tests written and passing
* [ ] Integration tests written and passing
* [ ] Backward compatibility verified
* [ ] Documentation updated
* [ ] Relevant workflows updated (if needed)
### Additional Context:
The main purpose of this feature is to reduce adoption friction for organizations that already store CALM artifacts in authenticated repositories.
CALM should be able to retrieve those artifacts in a read-only way through `DirectUrlDocumentLoader`, and `calm validate` should be able to operate on them without requiring migration into CALM Hub.
This keeps the scope focused on authenticated retrieval of CALM artifacts rather than expanding into broader repository-management capabilities.
It should also keep a clean boundary between the public CALM codebase and end-user organization authentication implementations. The `architecture-as-code` repo should add standards-based support for authenticated retrieval, while proprietary or organization-specific IDP implementations should remain in external repositories or packages controlled by the end user organization.
Illustrative Example
```json
{
"calmHubUrl": "https://calm.example.com",
"authPluginPath": "~/company-calmhub-plugin.js",
"directUrlAuth": {
"module": "~/plugins/acme-direct-url-auth.js",
"options": {
"tokenUrl": "https://idp.acme.example.com/oauth/token",
"clientId": "calm-direct-url",
"clientSecret": "${ACME_IDP_CLIENT_SECRET}",
"scopes": ["calm:read", "calm:documents"],
"headerName": "Authorization",
"headerPrefix": "Bearer "
}
}
}
```
Contributor guide
Assessment
This issue has not been assessed yet.