swagger-api / swagger-api/swagger-parser

OAS3 Swagger parser: Error on external http references

Open
#1,142 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
867
Forks
560
Avg merge
2d 21h
Merged PRs (30d)
7

Description

I have my API definition in a YAML file and I tried to use absolute references to make references to common or shared API "components" (or fragments) hosted on a dedicated repository and publicly accessible via http in order to reuse those components in several API definitions.

But it looks that the generator (openapi-generator-maven-plugin) fails to reach these remote API definitions when I run it in a corporate network. I've tried the same outside the work network and it works fine. So that's why I think it could be a problem with the proxy. I must point that other maven plugins such as WGET:

<groupId>com.googlecode.maven-download-plugin</groupId>
<artifactId>download-maven-plugin</artifactId>
<version>1.4.1</version>

do work with the same network and setup, that's what I think it is not a Maven or network issue.

The result is that it can't reach the referenced components via http and generation fails.

I'm using the Maven openapi-generator-maven-plugin:3.3.4 to generate my API artifacts from YAML definitions.

The maven is run by corporate Jenkins CI (but I insist that maven wget plugin works fine in the same setup)

The remote component URL is something like this:
http://myprivateorganization.foo/bitbucket/projects/FOO/repos/api-foo/raw/api-foo.yaml?at=refs%2Ftags%2Fv1.0 (which is a versioned tag of the API), so I reference it inside API definition (yaml) like the following:

components:
  schemas:
    Foo:
      type: object
      properties:
        status:
          $ref: 'http://myorganization.com/bitbucket/projects/FOO/repos/api-foo/raw/api-foo.yaml?at=refs%2Ftags%2Fv1.0#/components/schemas/SomeComponent'

I provide the stack-trace of the generation in case can help:

[INFO] --- openapi-generator-maven-plugin:3.3.4:generate (default) @ psd2-vas-contract ---
[ERROR] unable to read
java.io.FileNotFoundException: http://myprivateorganization.foo/bitbucket/projects/FOO/repos/api-foo/raw/api-foo.yaml
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0 (HttpURLConnection.java:1835)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream (HttpURLConnection.java:1440)
    at io.swagger.v3.parser.util.RemoteUrl.urlToString (RemoteUrl.java:148)
    at io.swagger.v3.parser.util.RefUtils.readExternalRef (RefUtils.java:156)
    at io.swagger.v3.parser.ResolverCache.loadRef (ResolverCache.java:118)
    at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema (ExternalRefProcessor.java:48)
    at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema (SchemaProcessor.java:201)
    at io.swagger.v3.parser.processors.SchemaProcessor.processPropertySchema (SchemaProcessor.java:102)
    at io.swagger.v3.parser.processors.SchemaProcessor.processSchemaType (SchemaProcessor.java:53)
    at io.swagger.v3.parser.processors.SchemaProcessor.processSchema (SchemaProcessor.java:38)
    at io.swagger.v3.parser.processors.ComponentsProcessor.processSchemas (ComponentsProcessor.java:221)
    at io.swagger.v3.parser.processors.ComponentsProcessor.processComponents (ComponentsProcessor.java:71)
    at io.swagger.v3.parser.OpenAPIResolver.resolve (OpenAPIResolver.java:50)
    at io.swagger.v3.parser.OpenAPIV3Parser.readLocation (OpenAPIV3Parser.java:53)
    at io.swagger.parser.OpenAPIParser.readLocation (OpenAPIParser.java:19)
    at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput (CodegenConfigurator.java:552)
    at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:607)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:497)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

[WARNING] Exception while reading:
java.lang.RuntimeException: Unable to load URL ref: http://myprivateorganization.foo/bitbucket/projects/FOO/repos/api-foo/raw/api-foo.yaml path: C:\Users\user\repos\myproject\generator-api-contracts-java\target
    at io.swagger.v3.parser.util.RefUtils.readExternalRef (RefUtils.java:183)
    at io.swagger.v3.parser.ResolverCache.loadRef (ResolverCache.java:118)
    at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema (ExternalRefProcessor.java:48)
    at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema (SchemaProcessor.java:201)
    at io.swagger.v3.parser.processors.SchemaProcessor.processPropertySchema (SchemaProcessor.java:102)
    at io.swagger.v3.parser.processors.SchemaProcessor.processSchemaType (SchemaProcessor.java:53)
    at io.swagger.v3.parser.processors.SchemaProcessor.processSchema (SchemaProcessor.java:38)
    at io.swagger.v3.parser.processors.ComponentsProcessor.processSchemas (ComponentsProcessor.java:221)
    at io.swagger.v3.parser.processors.ComponentsProcessor.processComponents (ComponentsProcessor.java:71)
    at io.swagger.v3.parser.OpenAPIResolver.resolve (OpenAPIResolver.java:50)
    at io.swagger.v3.parser.OpenAPIV3Parser.readLocation (OpenAPIV3Parser.java:53)
    at io.swagger.parser.OpenAPIParser.readLocation (OpenAPIParser.java:19)
    at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput (CodegenConfigurator.java:552)
    at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:607)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:497)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: java.io.FileNotFoundException: http://myprivateorganization.foo/bitbucket/projects/FOO/repos/api-foo/raw/api-foo.yaml
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0 (HttpURLConnection.java:1835)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream (HttpURLConnection.java:1440)
    at io.swagger.v3.parser.util.RemoteUrl.urlToString (RemoteUrl.java:148)
    at io.swagger.v3.parser.util.RefUtils.readExternalRef (RefUtils.java:156)
    at io.swagger.v3.parser.ResolverCache.loadRef (ResolverCache.java:118)
    at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema (ExternalRefProcessor.java:48)
    at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema (SchemaProcessor.java:201)
    at io.swagger.v3.parser.processors.SchemaProcessor.processPropertySchema (SchemaProcessor.java:102)
    at io.swagger.v3.parser.processors.SchemaProcessor.processSchemaType (SchemaProcessor.java:53)
    at io.swagger.v3.parser.processors.SchemaProcessor.processSchema (SchemaProcessor.java:38)
    at io.swagger.v3.parser.processors.ComponentsProcessor.processSchemas (ComponentsProcessor.java:221)
    at io.swagger.v3.parser.processors.ComponentsProcessor.processComponents (ComponentsProcessor.java:71)
    at io.swagger.v3.parser.OpenAPIResolver.resolve (OpenAPIResolver.java:50)
    at io.swagger.v3.parser.OpenAPIV3Parser.readLocation (OpenAPIV3Parser.java:53)
    at io.swagger.parser.OpenAPIParser.readLocation (OpenAPIParser.java:19)
    at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput (CodegenConfigurator.java:552)
    at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:607)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:497)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with io.swagger.v3.parser.util.RemoteUrl.urlToString and follow its use from RefUtils.readExternalRef, ResolverCache.loadRef, and OpenAPIV3Parser.readLocation. Reproduce the external HTTP reference failure through openapi-generator-maven-plugin:3.3.4 in a proxied environment, then verify that generation can retrieve the referenced YAML and resolve its component.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.