swagger-api / swagger-api/swagger-client
Extensive memory consumption when using OpenAPI 3.1 spec
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 2.7k
- Forks
- 765
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 6
Description
Q&A (please complete the following information)
- OS: macOS
- Environment: Node.js v10.9.2
- Method of installation: npm
- Swagger-Client version: 3.34.0
- Swagger/OpenAPI version: OpenAPI 3.1
Content & configuration
Swagger/OpenAPI definition:
https://petstore31.swagger.io/api/v31/openapi.json
Swagger-Client usage:
const swaggerClient = require('swagger-client')
const specFile = require('./petstore-spec')
async function main() {
const specVersion = process.argv[2]
const executions = Number(process.argv[3] || '10')
console.log(`Using spec version: ${specVersion}, executions: ${executions}`)
const iterator = Array.from(Array(executions).keys());
const spec = Object.assign(specFile, {openapi: specVersion})
const specClients = await Promise.all(
iterator.map(idx => swaggerClient({
spec: Object.assign(
{...spec},
{servers: [{url: 'http://localhost:8989/example-api-' + idx}]}
)
}))
)
return specClients.length
}
main()
.then(res => {
console.log(res);
})
.catch(err => {
console.error(err);
})
Describe the bug you're encountering
After migration to OpenAPI 3.1 specs we are experiencing memory consumption spikes leading to OOMs in our Node.js applications. Our investigation revealed that creating Swagger clients from OpenAPI 3.1 spec requires about 20% more memory than it was for 3.0.1 spec. For large specs (186 kB) it's even six times more and we have plenty of such specs. Unfortunately, we cannot execute promises one-by-one to limit the consumption because it would take ages to build all clients (sometimes even 50).
Petstore spec:
% /usr/bin/time -l node app.js 3.0.1 40
Using spec version: 3.0.1, executions: 40
40
0.70 real 0.35 user 0.09 sys
138412032 maximum resident set size
% /usr/bin/time -l node app.js 3.1.0 40
Using spec version: 3.1.0, executions: 40
40
1.39 real 1.59 user 0.07 sys
160399360 maximum resident set size
Large spec:
% /usr/bin/time -l node app.js 3.0.1 40
Using spec version: 3.0.1, executions: 40
40
0.85 real 0.60 user 0.09 sys
144490496 maximum resident set size
% /usr/bin/time -l node app.js 3.1.0 40
Using spec version: 3.1.0, executions: 40
40
30.93 real 32.17 user 0.48 sys
903905280 maximum resident set size
To reproduce...
Steps to reproduce the behavior:
- (Optional) Replace
petstore-specfile with some large spec file - Execute
/usr/bin/time -l node app.js 3.0.1 40to get results for 3.0.1 spec - Execute
/usr/bin/time -l node app.js 3.1.0 40to get results for 3.1.0 spec
Expected behavior
Memory consumption for large 3.1.0 specs is as close as possible to 3.0.1 specs
Additional context or thoughts
Is there a way to configure swagger-client so it consumes less memory for 3.1.0 specs?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the reported app.js reproduction, petstore-spec, and the provided OpenAPI 3.1 definition; run the 3.0.1 and 3.1.0 commands while profiling memory and comparing client construction. Trace the difference into swagger-client's OpenAPI 3.1 processing, and consider the issue done when large specifications no longer cause the reported memory spike while the reproduction still builds all clients.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, openapi
- Domain
- api, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100