feldera / feldera/feldera

Upgrade to OpenAPI v3.1 or v3.2

Open
#5,264 0 comments 1 reaction 0 assignees View on GitHub
Pipeline manager
Dominant language
Rust
Stars
2.1k
Forks
154
Avg merge
2d 19h
Merged PRs (30d)
133

Description

Currently we are using OpenAPI 3.0 (i.e., the format of `openapi.json` at the root of the repository).
It would be preferable to upgrade to the next up version (3.1) or even 3.2

- 3.0 -> 3.1 is not backward compatible for some reason: https://learn.openapis.org/upgrading/v3.0-to-v3.1.html
- 3.1 -> 3.2 is backward compatible: https://learn.openapis.org/upgrading/v3.1-to-v3.2.html

**Tasks for this:**
- [ ] Upgrade client dependency
- [ ] Upgrade progenitor once they support 3.1
- [ ] Upgrade utoipa, in particular this requires the following changes.

(a) Add to `ColumnType`, `RngFieldSettings`, `CalciteId` the following annotation:
```
#[schema(no_recursion)] // API schema generation will otherwise stack overflow due to recursive fields in this struct
```

(b) Use the various types across the API endpoints as they are now auto-detected

(c) New `ToSchema` implementation for `StartFromCheckpoint`
```
impl ToSchema for StartFromCheckpoint {
fn name() -> Cow<'static, str> {
Cow::Borrowed("StartFromCheckpoint")
}
}

impl PartialSchema for StartFromCheckpoint {
fn schema() -> RefOr {
RefOr::T(Schema::OneOf(
OneOfBuilder::new()
.item(
ObjectBuilder::new()
.schema_type(SchemaType::new(Type::String))
.enum_values(Some(["latest"].into_iter()))
.build(),
)
.item(
ObjectBuilder::new()
.schema_type(SchemaType::new(Type::String))
.format(Some(utoipa::openapi::SchemaFormat::KnownFormat(
utoipa::openapi::KnownFormat::Uuid,
)))
.build(),
)
.item(
ObjectBuilder::new()
.schema_type(SchemaType::new(Type::Null))
.build(),
)
.build(),
))
}
}
```

(d) New `none_as_string_schema`
```
fn none_as_string_schema() -> Schema {
Schema::OneOf(
OneOfBuilder::new()
.item(RefOr::Ref(Ref::new(format!(
"#/components/schemas/{}",
T::name()
))))
.item(
ObjectBuilder::new()
.schema_type(SchemaType::new(Type::String))
.enum_values(Some(vec!["none"])),
)
.default(Some(
serde_json::to_value(T::default()).expect("Failed to serialize default value"),
))
.build(),
)
}
```

(e) New `components(schemas(...))`
```
components(schemas(
// Enumeration types passed as query parameters are not automatically detected by utoipa
// and added to the OpenAPI spec. As a workaround, they are manually added below.
// Tracking issue: https://github.com/juhaku/utoipa/issues/1425
feldera_types::runtime_status::BootstrapPolicy,
feldera_types::query::AdHocResultFormat,
feldera_types::format::json::JsonUpdateFormat,
feldera_types::transport::datagen::DatagenStrategy,
feldera_types::program_schema::SqlType,
feldera_types::query_params::MetricsFormat,
feldera_types::config::FtModel,
crate::api::endpoints::pipeline_management::PipelineFieldSelector,
crate::api::endpoints::cluster::ClusterMonitorEventFieldSelector,
),),
```

Contributor guide

Open the contributing guide

Research direction

Start with the root openapi.json and the dependency configuration for the client, progenitor, and utoipa versions. Review ColumnType, RngFieldSettings, CalciteId, StartFromCheckpoint, none_as_string_schema, and the listed components(schemas(...)) changes. Done means the dependencies and schema generation support OpenAPI 3.1 or 3.2 without recursion failures, with the documented types represented in the generated specification.

Written by the indexing model from the issue text.

Assessment

Tech stack
openapi, rust
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.