finos / finos/morphir-dotnet

Fix network-dependent schema validation to work offline

Open
#168 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
F#
Stars
14
Forks
12
PR merge metrics
No merged PRs in 30d

Description

## Problem

The test `Handle_ShouldAutoDetectVersion_WhenVersionNotSpecified` intermittently fails on Windows CI with a network error:

```
RefResolutionException: Could not resolve 'https://finos.github.io/morphir/schemas/morphir-ir-v3.yaml#/definitions/PackageName'
```

**Failing Test**: [`tests/Morphir.Tooling.Tests/Features/VerifyIR/VerifyIRHandlerTests.cs:60`](tests/Morphir.Tooling.Tests/Features/VerifyIR/VerifyIRHandlerTests.cs#L60)

**Root Cause**: The `SchemaLoader` class loads schemas from embedded resources (which is good!), but the JSON Schema library (`json-everything`) tries to resolve `$ref` references to external URLs at validation time. When schemas reference other schemas via URLs like `https://finos.github.io/morphir/schemas/morphir-ir-v3.yaml#/definitions/PackageName`, it attempts network fetches.

**Impact**:
- ❌ Tests fail intermittently on CI (especially Windows)
- ❌ Schema validation requires network connectivity
- ❌ Slower validation due to network calls
- ❌ Potential security concerns with external dependencies at runtime

## Current Implementation

**File**: [`src/Morphir.Tooling/Infrastructure/JsonSchema/SchemaLoader.cs`](src/Morphir.Tooling/Infrastructure/JsonSchema/SchemaLoader.cs)

The schemas are already embedded as resources:
- `morphir-ir-v1.json/yaml`
- `morphir-ir-v2.json/yaml`
- `morphir-ir-v3.json/yaml`

But the schema registry uses remote URLs as IDs (line 40):
```csharp
var schemaId = new Uri($"{SchemaBaseUri}/morphir-ir-v{version}.yaml");
```

Where `SchemaBaseUri = "https://finos.github.io/morphir/schemas"`

## Desired Solution

Make schema validation work **completely offline** by:

1. **Register all embedded schemas** with the JSON Schema global registry using their canonical URLs
2. **Pre-load schema references** so `$ref` resolution works without network calls
3. **Configure the JSON Schema library** to use only the local registry

### Implementation Approach

```csharp
public class SchemaLoader
{
private static bool _globalRegistryInitialized = false;

public SchemaLoader()
{
// Initialize global registry once
if (!_globalRegistryInitialized)
{
InitializeGlobalRegistry();
_globalRegistryInitialized = true;
}
}

private static void InitializeGlobalRegistry()
{
// Register all embedded schemas with their canonical URLs
foreach (var version in new[] { "1", "2", "3" })
{
var schemaUri = new Uri($"{SchemaBaseUri}/morphir-ir-v{version}.yaml");
var schema = LoadSchemaFromEmbeddedResource(version);

// Register in global registry for $ref resolution
Json.Schema.SchemaRegistry.Global.Register(schemaUri, schema);
}
}
}
```

## Acceptance Criteria

- [ ] All tests pass on Windows, macOS, and Linux without network connectivity
- [ ] Schema validation works with `$ref` references resolved from embedded resources
- [ ] No network calls during schema validation
- [ ] Tests run reliably in CI without intermittent failures
- [ ] Documentation updated to reflect offline-first design

## References

- Failing CI run: https://github.com/finos/morphir-dotnet/actions/runs/20246302445/job/58127135609
- json-everything docs: https://docs.json-everything.net/schema/basics/
- Related: #167 (Dependency updates PR where this was discovered)

## Priority

**High** - This is causing CI failures and makes the tool dependent on network availability.

---

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Contributor guide

Open the contributing guide

Research direction

Start with src/Morphir.Tooling/Infrastructure/JsonSchema/SchemaLoader.cs and the failing test at tests/Morphir.Tooling.Tests/Features/VerifyIR/VerifyIRHandlerTests.cs:60. Run the test with network access disabled and inspect how embedded schemas and their canonical URLs are registered. Done means all embedded schema references resolve locally, validation makes no network calls, and the test suite passes across Windows, macOS, and Linux.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
devtools, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.