swagger-api / swagger-api/swagger-client

Extensive memory consumption when using OpenAPI 3.1 spec

Open
#3,805 4 comments 8 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

cat: usability P2 type: bug
Dominant language
JavaScript
Stars
2.7k
Forks
765
Avg merge
1d 1h
Merged PRs (30d)
6

Description

Q&A (please complete the following information)
  • OS: macOS
  • Environment: Node.js v10.9.2
  • Method of installation: npm
  • Swagger-Client version: 3.34.0
  • Swagger/OpenAPI version: OpenAPI 3.1
Content & configuration

Swagger/OpenAPI definition:
https://petstore31.swagger.io/api/v31/openapi.json

Swagger-Client usage:

const swaggerClient = require('swagger-client')

const specFile = require('./petstore-spec')

async function main() {
    const specVersion = process.argv[2]
    const executions = Number(process.argv[3] || '10')
    console.log(`Using spec version: ${specVersion}, executions: ${executions}`)

    const iterator = Array.from(Array(executions).keys());
    const spec = Object.assign(specFile, {openapi: specVersion})
    const specClients = await Promise.all(
        iterator.map(idx => swaggerClient({
            spec: Object.assign(
                {...spec},
                {servers: [{url: 'http://localhost:8989/example-api-' + idx}]}
            )
        }))
    )
    return specClients.length
}

main()
    .then(res => {
        console.log(res);
    })
    .catch(err => {
        console.error(err);
    })
Describe the bug you're encountering

After migration to OpenAPI 3.1 specs we are experiencing memory consumption spikes leading to OOMs in our Node.js applications. Our investigation revealed that creating Swagger clients from OpenAPI 3.1 spec requires about 20% more memory than it was for 3.0.1 spec. For large specs (186 kB) it's even six times more and we have plenty of such specs. Unfortunately, we cannot execute promises one-by-one to limit the consumption because it would take ages to build all clients (sometimes even 50).

Petstore spec:

% /usr/bin/time -l node app.js 3.0.1 40 
Using spec version: 3.0.1, executions: 40
40
        0.70 real         0.35 user         0.09 sys
           138412032  maximum resident set size

% /usr/bin/time -l node app.js 3.1.0 40
Using spec version: 3.1.0, executions: 40
40
        1.39 real         1.59 user         0.07 sys
           160399360  maximum resident set size

Large spec:

% /usr/bin/time -l node app.js 3.0.1 40
Using spec version: 3.0.1, executions: 40
40
        0.85 real         0.60 user         0.09 sys
           144490496  maximum resident set size

% /usr/bin/time -l node app.js 3.1.0 40
Using spec version: 3.1.0, executions: 40
40
       30.93 real        32.17 user         0.48 sys
           903905280  maximum resident set size
To reproduce...

Steps to reproduce the behavior:

  1. (Optional) Replace petstore-spec file with some large spec file
  2. Execute /usr/bin/time -l node app.js 3.0.1 40 to get results for 3.0.1 spec
  3. Execute /usr/bin/time -l node app.js 3.1.0 40 to get results for 3.1.0 spec
Expected behavior

Memory consumption for large 3.1.0 specs is as close as possible to 3.0.1 specs

Additional context or thoughts

Is there a way to configure swagger-client so it consumes less memory for 3.1.0 specs?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the reported app.js reproduction, petstore-spec, and the provided OpenAPI 3.1 definition; run the 3.0.1 and 3.1.0 commands while profiling memory and comparing client construction. Trace the difference into swagger-client's OpenAPI 3.1 processing, and consider the issue done when large specifications no longer cause the reported memory spike while the reproduction still builds all clients.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, node.js, openapi
Domain
api, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.