ethereum / ethereum/beacon-APIs

Derive OpenAPI type schema from pyspec

Open
#402 7 comments 4 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
389
Forks
224
Avg merge
8h 29m
Merged PRs (30d)
3

Description

Thanks to [ethereum/consensus-specs/pull/3506](https://github.com/ethereum/consensus-specs/pull/3506) we have canonical mapping of SSZ -> YAML, so it is possible to derive OpenAPI schemas for all the spec types.

Since the vast majority of API types are spec types auto-deriving the types would reduce future fork development and maintenance. Plus it paves the way to SSZ-ing most API routes.

We can define API-only extra types in SSZ syntax, such as:

```markdown
// extra_types/bellatrix.md

## Containers

### Registration

From the Builder API specification

```python
class ValidatorRegistration(Container):
fee_recipient: ExecutionAddress # Address to receive fees from the block
gas_limit: uint64 # Preferred gas limit of validator
timestamp: uint64 # Unix timestamp of registration
pubkey: BLSPubkey # BLS public key of validator

class SignedValidatorRegistration(Container):
message: ValidatorRegistration
signature: BLSSignature

```

### Current status

I've created a simple python script to translate the pyspec to OpenAPI schemas https://pypi.org/project/pyspec2openapi. Here is a demo integration into the beacon-APIs repo https://github.com/dapplion/eth2.0-APIs/pull/1

The **diff** is quite large since it merges all types into a single output file.

One can compare the bundled outputs with

```
git checkout master
swagger-cli bundle ./beacon-node-oapi.yaml -r -t yaml -o ./deploy/beacon-node-oapi-master.yaml
git checkout dapplion/derive-spec-types
swagger-cli bundle ./beacon-node-oapi.yaml -r -t yaml -o ./deploy/beacon-node-oapi-derive.yaml
diff -l -u deploy/beacon-node-oapi-master.yaml deploy/beacon-node-oapi-derive.yaml | colordiff | more -R
```

The bundled diff is also quite substantial due to some key characteristics
- Usage of allOf to define block types. Generated spec does not de-duplicate keys, resulting in less indentations
```
BeaconBlock:
allOf:
- $ref: '.../BeaconBlockCommon'
- $ref: '#/Deneb/BeaconBlockBody'
```
- Lots of comments per-property and per type not present in the generated specs
https://github.com/ethereum/beacon-APIs/blob/4882aa0803b622b75bab286b285599d70b7a2429/types/block.yaml#L24

### Next steps

- [ ] Get buy-in from repo mantainer's, or how could I tune this to align with your prefered DX

If yes, decide how to merge

- One big diff PR switching directly to generated types, a-la https://github.com/dapplion/eth2.0-APIs/pull/1
- Big diff PR merging existing existing schemas into a single spec.yaml file, then replace with generated schemas for a clearer diff
- Modify current master branch to align with generated output (remove block allOf usage, remove comments), then merge a smaller diff PR switching to generated types

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with beacon-node-oapi.yaml and the issue's pyspec2openapi demo integration, then run the listed swagger-cli bundle and diff commands against the bundled outputs in deploy/. Review the generated allOf structure and omitted comments before discussing which of the three merge paths maintainers prefer. Done requires an agreed integration approach and accepted generated schemas.

Written by the indexing model from the issue text.

Assessment

Tech stack
openapi, python
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.