swagger-api / swagger-api/swagger-codegen

[ANY LANG] The incoming YAML document exceeds the limit: 3145728 code points.

Open
#11,980 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Mustache
Stars
17.8k
Forks
6k
No merged PRs in 30d

Description

Description

When executing the following command:
java -jar modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate -i ../swagger.json -l html2 -o target

The following exception occurs, no mather which language (-l) is exported to:

io.swagger.v3.parser.util.DeserializationUtils$SnakeException: Exception safe-checking yaml content  (maxDepth 2000, maxYamlAliasesForCollections 2147483647)
	at io.swagger.v3.parser.util.DeserializationUtils$CustomSnakeYamlConstructor.getSingleData(DeserializationUtils.java:438)
	at org.yaml.snakeyaml.Yaml.loadFromReader(Yaml.java:477)
	at org.yaml.snakeyaml.Yaml.load(Yaml.java:406)
	at io.swagger.v3.parser.util.DeserializationUtils.readYamlTree(DeserializationUtils.java:211)
	at io.swagger.v3.parser.util.DeserializationUtils.deserializeIntoTree(DeserializationUtils.java:143)
	at io.swagger.v3.parser.OpenAPIV3Parser.readContents(OpenAPIV3Parser.java:165)
	at io.swagger.v3.parser.OpenAPIV3Parser.readContents(OpenAPIV3Parser.java:104)
	at io.swagger.v3.parser.converter.SwaggerConverter.readResult(SwaggerConverter.java:111)
	at io.swagger.v3.parser.converter.SwaggerConverter.readLocation(SwaggerConverter.java:85)
	at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
	at io.swagger.codegen.v3.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:612)
	at io.swagger.codegen.v3.cli.cmd.Generate.run(Generate.java:386)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points.
	at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:342)
	at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:263)
	at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:694)
	at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:185)
	at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:57)
	at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:43)
	at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:136)
	at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:116)
	at org.yaml.snakeyaml.composer.Composer.composeScalarNode(Composer.java:239)
	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:208)
	at org.yaml.snakeyaml.composer.Composer.composeKeyNode(Composer.java:347)
	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:332)
	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:311)
	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:212)
	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:357)
	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:336)
	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:311)
	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:212)
	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:357)
	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:336)
	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:311)
	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:212)
	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:357)
	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:336)
	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:311)
	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:212)
	at org.yaml.snakeyaml.composer.Composer.composeValueNode(Composer.java:357)
	at org.yaml.snakeyaml.composer.Composer.composeMappingChildren(Composer.java:336)
	at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java:311)
	at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:212)
	at org.yaml.snakeyaml.composer.Composer.getNode(Composer.java:134)
	at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:160)
	at io.swagger.v3.parser.util.DeserializationUtils$CustomSnakeYamlConstructor.getSingleData(DeserializationUtils.java:415)
	... 12 common frames omitted
14:39:19.137 [Thread-0] ERROR i.s.v.p.util.DeserializationUtils - Error parsing content
com.fasterxml.jackson.dataformat.yaml.JacksonYAMLParseException: The incoming YAML document exceeds the limit: 3145728 code points.
 at [Source: (StringReader); line: 99391, column: 16]
	at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:425)
	at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:539)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:98)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:23)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
	at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4772)
	at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3124)
	at io.swagger.v3.parser.util.DeserializationUtils.readYamlTree(DeserializationUtils.java:232)
	at io.swagger.v3.parser.util.DeserializationUtils.deserializeIntoTree(DeserializationUtils.java:143)
	at io.swagger.v3.parser.OpenAPIV3Parser.readContents(OpenAPIV3Parser.java:165)
	at io.swagger.v3.parser.OpenAPIV3Parser.readContents(OpenAPIV3Parser.java:104)
	at io.swagger.v3.parser.converter.SwaggerConverter.readResult(SwaggerConverter.java:111)
	at io.swagger.v3.parser.converter.SwaggerConverter.readLocation(SwaggerConverter.java:85)
	at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
	at io.swagger.codegen.v3.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:612)
	at io.swagger.codegen.v3.cli.cmd.Generate.run(Generate.java:386)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points.
	at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:342)
	at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:263)
	at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:694)
	at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:185)
	at org.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:195)
	at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:419)
	... 16 common frames omitted
14:39:19.311 [Thread-0] WARN  io.swagger.v3.parser.OpenAPIV3Parser - Exception while parsing:
com.fasterxml.jackson.dataformat.yaml.JacksonYAMLParseException: The incoming YAML document exceeds the limit: 3145728 code points.
 at [Source: (StringReader); line: 99391, column: 16]
	at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:425)
	at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer._deserializeContainerNoRecursion(JsonNodeDeserializer.java:539)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:98)
	at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:23)
	at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
	at com.fasterxml.jackson.databind.ObjectMapper._readTreeAndClose(ObjectMapper.java:4772)
	at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:3124)
	at io.swagger.v3.parser.OpenAPIV3Parser.readContents(OpenAPIV3Parser.java:167)
	at io.swagger.v3.parser.OpenAPIV3Parser.readContents(OpenAPIV3Parser.java:104)
	at io.swagger.v3.parser.converter.SwaggerConverter.readResult(SwaggerConverter.java:111)
	at io.swagger.v3.parser.converter.SwaggerConverter.readLocation(SwaggerConverter.java:85)
	at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:16)
	at io.swagger.codegen.v3.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:612)
	at io.swagger.codegen.v3.cli.cmd.Generate.run(Generate.java:386)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.yaml.snakeyaml.error.YAMLException: The incoming YAML document exceeds the limit: 3145728 code points.
	at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:342)
	at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:263)
	at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:694)
	at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:185)
	at org.yaml.snakeyaml.parser.ParserImpl.getEvent(ParserImpl.java:195)
	at com.fasterxml.jackson.dataformat.yaml.YAMLParser.nextToken(YAMLParser.java:419)
	... 14 common frames omitted
14:39:19.314 [Thread-0] INFO  i.s.c.v.i.CodegenIgnoreProcessor - No .swagger-codegen-ignore file found.
Exception in thread "Thread-0" java.lang.RuntimeException: missing OpenAPI input!
	at io.swagger.codegen.v3.DefaultGenerator.generate(DefaultGenerator.java:777)
	at io.swagger.codegen.v3.cli.cmd.Generate.run(Generate.java:388)
	at java.base/java.lang.Thread.run(Thread.java:833)

The swagger.json for our monolith of an application is 145 288 lines long.

Swagger-codegen version

3.0.36

Swagger declaration file content or url

I am not a liberty to share this .json file due to an NDA.
The file has 145 288 lines and is about 4MB in size.

Command line used for generation

java -jar modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate -i ../swagger.json -l html2 -o target

openjdk 17.0.5 2022-10-18
Ubuntu 22.04.1 LTS x86_64

Steps to reproduce
  1. execute the command above with a large swagger.json file
  2. see the exception occur that's mentioned above
Related issues/PRs

N/A

Suggest a fix/enhancement

Increase upper limit or make configurable.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with io.swagger.v3.parser.util.DeserializationUtils.java and the OpenAPIV3Parser.readContents entry point shown in the stack trace. Reproduce the failure using the generate command and a YAML or JSON document larger than 3,145,728 code points. Done means the limit is increased or configurable and the large OpenAPI input no longer fails during parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.