OpenAPITools / OpenAPITools/openapi-generator

[BUG][Protobuf] Generates circular file imports that don't compile with protoc

Open
#15,461 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Issue: Bug
Dominant language
Java
Stars
26.8k
Forks
7.7k
PR merge metrics
PR metrics pending

Description

Bug Report Checklist
  • Have you provided a full/minimal spec to reproduce the issue?
  • Have you validated the input using an OpenAPI validator (example)?
  • Have you tested with the latest master to confirm the issue still exists?
  • Have you searched for related issues/PRs?
  • What's the actual output vs expected output?
  • [Optional] Sponsorship to speed up the bug fix or feature request (example)
Description

Converting Teamcity's swagger.json to Protocol Buffers results in protobuf specification that doesn't compile with protoc. Specifically because of circular/recursive file references/imports.

openapi-generator version

--version output:

openapi-generator-cli 6.6.0-SNAPSHOT
  commit : ff9b384
  built  : -999999999-01-01T00:00:00+18:00
  source : https://github.com/openapitools/openapi-generator
  docs   : https://openapi-generator.tech/
OpenAPI declaration file content or url

https://teamcity.jetbrains.com/app/rest/swagger.json

Generation Details
docker run -m 24g -it --rm -v "$(pwd):/local" openapitools/openapi-generator-cli java -Xmx20G -jar ./opt/openapi-generator/modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -i https://teamcity.jetbrains.com/app/rest/swagger.json -g protobuf-schema -o /local/out/teamcity

Needs extra mem otherwise fails with Exception in thread "main" java.lang.OutOfMemoryError: Java heap space.

Steps to reproduce
  1. Download/extract protobuf compiler v23.0 (the latest release) for win64 (link)
  2. cd ./out/teamcity
  3. .\protoc-23.0-win64\bin\protoc.exe --csharp_out=./csharp .\models/*
  4. Note resulting errors:
models/agent.proto:15:1: File recursively imports itself: models/agent.proto -> models/agent_pool.proto -> models/agents.proto -> models/agent.proto
models/agents.proto:15:1: Import "models/agent.proto" was not found or had errors.
models/agents.proto:27:12: "Agent" is not defined.
<SNIP>
models/build_types.proto:15:1: Import "models/build_type.proto" was not found or had errors.
models/build_types.proto:27:12: "BuildType" is not defined.
models/cloud_image.proto:15:1: File recursively imports itself: models/cloud_image.proto -> models/cloud_instances.proto -> models/cloud_instance.proto -> models/cloud_image.proto
models/cloud_instance.proto:15:1: Import "models/agent.proto" was not found or had errors.
models/cloud_instance.proto:16:1: Import "models/cloud_image.proto" was not found or had errors.
models/cloud_instance.proto:32:3: "CloudImage" is not defined.
models/cloud_instance.proto:34:3: "Agent" is not defined.
models/cloud_instances.proto:15:1: Import "models/cloud_instance.proto" was not found or had errors.
models/cloud_instances.proto:27:12: "CloudInstance" is not defined.
<SNIP>
Related issues/PRs
Suggest a fix

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by generating the protobuf-schema output from TeamCity's swagger.json with the documented openapi-generator command, then run protoc v23.0 against the generated models. Trace the imports shown in agent.proto, agent_pool.proto, agents.proto, cloud_image.proto, cloud_instances.proto, and cloud_instance.proto. Done means the generated protobuf files compile without recursive-import or undefined-symbol errors.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.