apache / apache/ossie

Clarify query-backed Dataset.source representation and SQL dialect

Open
#298 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.1k
Forks
267
Avg merge
4d 20h
Merged PRs (30d)
24

Description

## Problem

The [core specification](https://github.com/apache/ossie/blob/main/core-spec/spec.md#datasets)
defines `Dataset.source` as a string that may identify a physical table/view or contain a query.

For a query-backed source, the specification provides neither:

- a discriminator that distinguishes a relation identifier from SQL text; nor
- a SQL dialect for parsing or validating the query.

Consumers therefore cannot interpret query-backed sources consistently.

## Questions

1. Is a query-backed `Dataset.source` intended to be portable core behavior?
2. If so, should the specification provide a structured form such as:

```yaml
source:
type: query
dialect: ANSI_SQL
query: SELECT ...
```

3. Should the existing string form remain the relation-identifier form for compatibility?
4. If query interpretation is intentionally consumer-specific, should the specification
explicitly allow consumers to reject query-backed sources?

## Context

Gravitino maps an OSI dataset source to a catalog Table or View. Because the query dialect is
currently unspecified, its proposed structured Metric View model requires a query to be registered
as a logical View before it can be referenced as a dataset source.

Related discussions: #52, #294, and #104.

Contributor guide

Open the contributing guide

Research direction

Start with the Dataset section in core-spec/spec.md, then review related discussions #52, #294, and #104 for prior decisions. Determine whether the specification can define the source representation and SQL dialect, or must document consumer-specific handling; done means the open questions are resolved and the relevant specification text is updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.