apache / apache/polaris

Semantic model object documents bypass dataset source validation

Open
#5,405 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
522
Avg merge
1d 22h
Merged PRs (30d)
137

Description

## Describe the bug

The semantic-model create and update paths accept any well-formed JSON value, but dataset-source validation only runs when the parsed `semantic_model` value is an array. A JSON object therefore bypasses every `dataset.source` check and is persisted even when it references a missing table.

The OpenAPI example currently documents exactly this invalid object form, so clients copying the published example can persist a semantic model that does not conform to the Apache Ossie document shape and contains dangling dataset sources.

This is narrower than #5205: it does not implement full Ossie JSON Schema validation, schema-version selection, or document-size limits. It restores the array-root invariant already assumed by the existing source-validation loop and corrects the API example.

## To Reproduce

Using Polaris `main` at `7bb319e281225b2b63c78f1cbc87726a1c9a7ece` with the beta semantic-model API enabled:

1. Create a catalog and a `sales` namespace without a `sales.missing_table` table.
2. Create a semantic model in that namespace with the object-form document shown by the current OpenAPI example:

```json
{
"name": "model_with_missing_source",
"document": {
"version": "0.1.1",
"semantic_model": "{\"name\":\"model_with_missing_source\",\"datasets\":[{\"name\":\"missing\",\"source\":\"sales.missing_table\"}]}"
}
}
```

3. Load the created semantic model.

The same bypass is present on update.

## Actual Behavior

The create or update succeeds and persists the object-form document. `SemanticModelCatalog.resolveAndValidateSources` returns immediately because the parsed root is not an array, so `sales.missing_table` is never resolved.

## Expected Behavior

Create and update should return HTTP 400 when `semantic_model` is not a JSON array. The OpenAPI example should show the supported Apache Ossie array form. Valid array-form documents should continue through the existing dataset-source validation.

## Additional context

- The current Apache Ossie JSON Schema defines the semantic-model document root as an array.
- A focused regression test can submit an object with a dangling source and assert that persistence is rejected before any source lookup.
- No dependency or full-schema validator is needed for this correctness fix; full document validation remains tracked by #5205.
- This is a correctness bug in the beta semantic-model API, not a security report.

## System information

- OS: macOS 26.6.2, arm64
- Polaris Catalog Version: `main` at `7bb319e281225b2b63c78f1cbc87726a1c9a7ece` (the semantic-model API is not present in a stable release yet)
- Object storage & setup: not applicable; the failure occurs before storage access

Contributor guide

Open the contributing guide

Research direction

Start with the semantic-model create and update paths and SemanticModelCatalog.resolveAndValidateSources, then inspect the current OpenAPI example. Add a focused regression test for an object-root document with a dangling dataset source. Done means both paths return HTTP 400 for non-array semantic_model values, while valid array documents still perform dataset-source validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.