OpenAPITools / OpenAPITools/openapi-generator

[BUG] [openapi-generator-cli] BATCH mode does not recognize multiple or nested '!include' directives.

Open
#7,311 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Issue: Bug
Dominant language
Java
Stars
26.8k
Forks
7.7k
PR merge metrics
PR metrics pending

Description

Bug Report Checklist
  • [✅] Have you provided a full/minimal spec to reproduce the issue?
  • [✅] Have you validated the input using an OpenAPI validator (example)?
  • [✅ ] What's the version of OpenAPI Generator used?
  • [✅] Have you search for related issues/PRs?
  • [✅] What's the actual output vs expected output?
  • [Optional] Bounty to sponsor the fix (example)
Description

openapi-generator-cli BATCH mode does not recognize multiple or nested '!include' directives.

openapi-generator version

4.3.1

OpenAPI declaration file content or url

InputSpec is your petstore-v3.0.yaml

Generation Details

NESTED FAILURE EXAMPLE

# java-common.yaml
generatorName: java
groupId: org.thisisatest.digital
configOptions:
  dateLibrary: java8
# java-jersey2.yaml
'!include': 'java-common.yaml'
library: jersey2
# test-config.yaml
'!include': 'java-jersey2.yaml'
inputSpec: petstore-v3.0.yaml
outputDir: generated/test

Command line: java -jar openapi-generator-cli-4.3.1.jar batch --verbose test_config.yaml

output:

[main] INFO  o.o.codegen.cmd.GenerateBatch - Batch generation using up to 16 threads.
Includes: /Users/fred/openapi-generator-cli/flat
Root: /Users/fred/openapi-generator-cli/flat
[pool-1-thread-1] Generation failed: (NullPointerException) language/generatorName must be specified
java.lang.NullPointerException: language/generatorName must be specified
	at org.apache.commons.lang3.Validate.notEmpty(Validate.java:395)
	at org.openapitools.codegen.config.CodegenConfigurator.toContext(CodegenConfigurator.java:423)
	at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:507)
	at org.openapitools.codegen.cmd.GenerateBatch$GenerationRunner.run(GenerateBatch.java:189)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
COMPLETE.
mbp-fred:flat fred$

SEQUENTIAL FAILURE EXAMPLE

# java-common.yaml
generatorName: java
groupId: org.thisisatest.digital
configOptions:
  dateLibrary: java8
# java-jersey2.yaml
#### REMOVED NESTED INCLUDE
library: jersey2
# test-config.yaml
#### MOVED IT HERE
'!include': 'java-common.yaml'
'!include': 'java-jersey2.yaml'
inputSpec: petstore-v3.0.yaml
outputDir: generated/test

Command line: java -jar openapi-generator-cli-4.3.1.jar batch --verbose test_config.yaml

java -jar openapi-generator-cli-4.3.1.jar batch --verbose test_config.yaml
[main] INFO  o.o.codegen.cmd.GenerateBatch - Batch generation using up to 16 threads.
Includes: /Users/fred/openapi-generator-cli/flat
Root: /Users/fred/openapi-generator-cli/flat
[pool-1-thread-1] Generation failed: (NullPointerException) **language/generatorName must be specified**
java.lang.NullPointerException: language/generatorName must be specified
	at org.apache.commons.lang3.Validate.notEmpty(Validate.java:395)
	at org.openapitools.codegen.config.CodegenConfigurator.toContext(CodegenConfigurator.java:423)
	at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:507)
	at org.openapitools.codegen.cmd.GenerateBatch$GenerationRunner.run(GenerateBatch.java:189)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
COMPLETE.
mbp-fred fred$

REVERSAL OF INCLUDEs WORKS?

If you reverse the two includes in test-config.yaml, the generation seems to work, but the settings in the first included file are ignored.

# java-common.yaml
generatorName: java
groupId: org.thisisatest.digital
configOptions:
  dateLibrary: java8
#java-jersey2.yaml
library: jersey2
groupId: org.thisisatest.injersey2
#test_config.yaml
'!include': 'java-jersey2.yaml'
'!include': 'java-common.yaml'
inputSpec: petstore-v3.0.yaml
outputDir: generated/test

The generation succeeds, but note that the date library is threetenp, and the http/json libraries are the default instead of those specified in java-jersey2.yaml:

dependencies {
    compile 'io.swagger:swagger-annotations:1.5.24'
    compile "com.google.code.findbugs:jsr305:3.0.2"
    compile 'com.squareup.okhttp3:okhttp:3.14.7'
    compile 'com.squareup.okhttp3:logging-interceptor:3.14.7'
    compile 'com.google.code.gson:gson:2.8.6'
    compile 'io.gsonfire:gson-fire:1.8.4'
    compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.10'
    compile 'org.threeten:threetenbp:1.4.3'
    testCompile 'junit:junit:4.13'
}

ACTUAL RESULTS WITHOUT INCLUDES

# test_all_config.yaml
generatorName: java
groupId: org.thisisatest.digital
configOptions:
  dateLibrary: java8
library: jersey2
inputSpec: petstore-v3.0.yaml
outputDir: generated/test

java -jar openapi-generator-cli-4.3.1.jar batch --verbose test_all_config.yaml

dependencies {
    compile "io.swagger:swagger-annotations:$swagger_annotations_version"
    compile "com.google.code.findbugs:jsr305:3.0.2"
    compile "org.glassfish.jersey.core:jersey-client:$jersey_version"
    compile "org.glassfish.jersey.media:jersey-media-multipart:$jersey_version"
    compile "org.glassfish.jersey.media:jersey-media-json-jackson:$jersey_version"
    compile "com.fasterxml.jackson.core:jackson-core:$jackson_version"
    compile "com.fasterxml.jackson.core:jackson-annotations:$jackson_version"
    compile "com.fasterxml.jackson.core:jackson-databind:$jackson_databind_version"
    compile "org.openapitools:jackson-databind-nullable:$jackson_databind_nullable_version"
    compile "com.github.joschi.jackson:jackson-datatype-threetenbp:$threetenbp_version"
    compile "com.brsanthu:migbase64:2.2"
    testCompile "junit:junit:$junit_version"
}
``

##### Steps to reproduce

* Create the yaml files, above. 
* run the given java command.

##### Related issues/PRs

I didn't find any.

##### Suggest a fix

I think the code in [GenerateBatch.java](https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/cmd/GenerateBatch.java), class DynamicSettingsRefSupport, method deserialize doesn't support multiple includes as it uses the TreeNode.get(String) method that can only return a single node.

Also, the code that reads in the included file does not then parse the included data for nested '!include' tokens.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/cmd/GenerateBatch.java, specifically DynamicSettingsRefSupport.deserialize, and reproduce the failure with the nested and multiple-include YAML files and the batch command shown. Done means batch mode recognizes multiple and nested !include directives and preserves settings from all included files.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.