OpenAPITools / OpenAPITools/openapi-generator
[BUG] Error generating models with types referencing to attributes types
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 26.8k
- Forks
- 7.7k
- PR merge metrics
- PR metrics pending
Description
Bug Report Checklist
- Have you provided a full/minimal spec to reproduce the issue?
- Have you validated the input using an OpenAPI validator (example)?
This API is valid, but it cannot be shown because it contains circular references
- Have you tested with the latest master to confirm the issue still exists?
Yes, with openapi-generator-cli-7.0.0-20220719.043604-2.jar, valides says "OK" but generates crashes: Exception in thread "main" java.lang.RuntimeException: Could not process model 'EventFilter_exptUeBehav_expectedUmts_inner_geographicAreas_inner_anyOf_1'.Please make sure that your schema is correct!
- Have you searched for related issues/PRs?
- What's the actual output vs expected output?
A python-flask server that can be launched
- [Optional] Sponsorship to speed up the bug fix or feature request (example)
Description
I'm triying to generate a OpenAPI server from 3GPP 5G specs. I got this specs from this repository https://github.com/jdegre/5GC_APIs and trying to build the .yaml called: TS29520_Nnwdaf_AnalyticsInfo.yaml
openapi-generator-cli generate -g python-flask -i TS29520_Nnwdaf_AnalyticsInfo.yaml -o output
Then I tried to install dependencies:
cd output
python3 -m venv .
bin/pip install -r requirements.txt
bin/python -m openapi_server
And got the first error:
ImportError: cannot import name 'escape' from 'jinja2' (~/openapi/5GC_APIs/output/lib/python3.8/site-packages/jinja2/__init__.py)
I searched for this and is for the flask version (1.1.2): https://stackoverflow.com/questions/71718167/importerror-cannot-import-name-escape-from-jinja2
I updated the Flask version:
bin/pip install Flask==2.1.0
And now it starts with the generated models problems:
bin/python -m openapi_server
ImportError: cannot import name 'NnwdafEventsSubscription' from partially initialized module 'openapi_server.models.nnwdaf_events_subscription' (most likely due to a circular import) (~/openapi/5GC_APIs/output/openapi_server/models/nnwdaf_events_subscription.py)
Reading some bugs opened here I found "swagger-cli bundle", I tried this:
swagger-cli bundle -o b_TS29520_Nnwdaf_AnalyticsInfo.yaml -r -t yaml TS29520_Nnwdaf_AnalyticsInfo.yaml
But "dereference" doesn't work because there is a "Circular $ref pointer", but if I remove this I can generate a single .yaml with all the spec:
swagger-cli bundle -o b_TS29520_Nnwdaf_AnalyticsInfo.yaml -t yaml TS29520_Nnwdaf_AnalyticsInfo.yaml
I uploaded the output .yaml as gziped file (to use github upload): b_TS29520_Nnwdaf_AnalyticsInfo.yaml.gz
Now repeat the code generation using this new bundled yaml:
rm -rf output
openapi-generator-cli generate -g python-flask -i b_TS29520_Nnwdaf_AnalyticsInfo.yaml -o output
cd output
python3 -m venv .
bin/pip install -r requirements.txt
bin/pip install Flask==2.1.0
bin/python -m openapi_server
And now the real question comes: It raises new errors based on genereated models from the yaml:
from openapi_server.models.network_area import NetworkArea
ModuleNotFoundError: No module named 'openapi_server.models.network_area'
If we look the spec we can see where this come from some attributes referencing other types:
networkArea:
$ref: '#/components/schemas/EventFilter/properties/networkArea'
And this adds the "import openapi_server.models.network_area" but this file doesn't exists. The real module is:
models/event_filter_network_area.py
Found here:
$ find -iname "*network_area*"
./openapi_server/models/event_filter_network_area_g_ran_node_ids_inner_gnb_id.py
./openapi_server/models/event_filter_network_area_g_ran_node_ids_inner.py
./openapi_server/models/event_filter_network_area_ecgis_inner.py
./openapi_server/models/event_filter_network_area_ncgis_inner.py
./openapi_server/models/event_filter_network_area_tais_inner.py
./openapi_server/models/event_filter_network_area.py <------------------------------------
So this is my first problem or bug: When a type references a subtype like "#/components/schemas/EventFilter/properties/networkArea" code adds a import to "openapi_server.models.network_area" but this doesn't exists, it is "models.event_filter_network_area import EventFilterNetworkArea"
I tried to fix this case with a string replace: Search for "models.network_area import Network area" and replace this to "model.event_filter_network_area import EventFilterNetworkArea as NetworkArea":
sed -i "s/models.network_area import NetworkArea/models.event_filter_network_area import EventFilterNetworkArea as NetworkArea/g" *.py
And it worked... but now I have a second situation, similar but different:
from openapi_server.models.start import Start
ModuleNotFoundError: No module named 'openapi_server.models.start'
This is more funny, because there isn't any "start" model. What happens here is initially the same: Types referencing other types.
startTs:
$ref: '#/components/schemas/AnalyticsData/properties/start'
And if we go to "AnalyticsData" we can see the definition as a string with format date-time:
properties:
start:
format: date-time
type: string
description: string with format 'date-time' as defined in OpenAPI.
The problem here is that "AnalyticsData" doesn't generates any class to contain this "start" property, it keeps the datetime directly:
:param start: The start of this AnalyticsData. # noqa: E501
:type start: datetime
.....
self.openapi_types = {
'start': datetime,
So no "analytics_data_start" module is generated and nothing can be imported.
The openapi-generator should 2 one of this things:
- See that the referenced type is a basic type and inline it in all other models.
- Always generate a "model.*" for all attributes (or al least all referenced attributes)
In the same "AnalyticsData" reference we have the two situations: the start that defines that is a string with "format: date-time" and then "expiry" that references to start to define his type:
:param start: The start of this AnalyticsData. # noqa: E501
:type start: datetime <---- Inlined datetime
:param expiry: The expiry of this AnalyticsData. # noqa: E501
:type expiry: Start <---- Using inexistent "model.start" instead of use "datetime"
Is this a bug? Is a openapi-generator limitation? Maybe a incorrect spec? I'm not and OpenAPI expert, but reading the yaml I can see that OpenAPI has the information to generate the code.
openapi-generator version
$ openapi-generator-cli version
6.0.1
Downloaded using npm.
OpenAPI declaration file content or url
b_TS29520_Nnwdaf_AnalyticsInfo.yaml.gz
Generation Details
openapi-generator-cli generate -g python-flask -i b_TS29520_Nnwdaf_AnalyticsInfo.yaml -o output
Steps to reproduce
Described in "Description", but basically:
openapi-generator-cli generate -g python-flask -i b_TS29520_Nnwdaf_AnalyticsInfo.yaml -o output
cd output
python3 -m venv .
bin/pip install -r requirements.txt
bin/pip install Flask==2.1.0
bin/python -m openapi_server
Suggest a fix
For the first case: Don't import the "last slash" (or generate this model). It is importing the inexistent "NetworkArea" (generatedfrom /NetworkArea) when the real import is EventFilterNetworkArea.
For the second case:
- See that the referenced type is a basic type and inline it in all other models.
- Always generate a "model.*" for all attributes (or al least all referenced attributes)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce with the bundled TS29520_Nnwdaf_AnalyticsInfo.yaml using the python-flask generation command, then inspect the generated openapi_server/models files and imports. Compare references such as EventFilter/properties/networkArea and AnalyticsData/properties/start with generated files such as event_filter_network_area.py. Done means the generated server imports successfully and resolves both referenced model and primitive types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- flask, java, openapi, python
- Domain
- api, backend, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100