OpenAPITools / OpenAPITools/openapi-generator

[BUG] Uploading large file from client code as multipart/form-data leads to high memory usage

Open
#15,028 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Issue: Bug
Dominant language
Java
Stars
26.8k
Forks
7.7k
PR merge metrics
PR metrics pending

Description

Bug Report Checklist
  • Have you provided a full/minimal spec to reproduce the issue?
  • Have you validated the input using an OpenAPI validator (example)?
  • Have you tested with the latest master to confirm the issue still exists?
  • Have you searched for related issues/PRs?
  • What's the actual output vs expected output?
  • [Optional] Sponsorship to speed up the bug fix or feature request (example)
Description

Hello,
We're observing OutOfMemoryErrors when uploading a large file from a client application as multipart/form-data. It appears the complete file is read into memory, however, I think this should be done streamed. Uploading a ~300MB file seems to use ~1000MB of heap memory.

Stacktrace
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.base/java.lang.StringCoding.decodeUTF8_0(StringCoding.java:777) at java.base/java.lang.StringCoding.decodeUTF8(StringCoding.java:734) at java.base/java.lang.StringCoding.decode(StringCoding.java:257) at java.base/java.lang.String.<init>(String.java:507) at java.base/java.lang.String.<init>(String.java:561) at okio.Buffer.readString(Buffer.kt:306) at okio.Buffer.readUtf8(Buffer.kt:290) at foo.bar.baz.ApiClient.requestBodyToString(ApiClient.java:1558) at foo.bar.baz.ApiClient.buildRequest(ApiClient.java:1209) at foo.bar.baz.ApiClient.buildCall(ApiClient.java:1157)

openapi-generator version

6.4.0

OpenAPI declaration file content or url

OpenAPI specification (simplified)

paths:
  /upload:
    post:
      tags:
      - foo
      summary: bar
      description: |
        baz
      requestBody:
        content:
          multipart/form-data:
            schema:
              body:
                 properties:
                    file:
                      type: string
                      format: binary
        required: true
Generation Details

Using Java generator with Maven.

Steps to reproduce
  • generate client code
  • start application with limited amount of heap memory
  • call client code with significantly large file
Related issues/PRs
Suggest a fix

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated ApiClient.java methods requestBodyToString, buildRequest, and buildCall named in the stack trace, then trace how the Java generator handles multipart/form-data file uploads. Reproduce the upload with a roughly 300 MB file under a limited heap; done means the client uploads it without converting the complete file into an in-memory string or raising OutOfMemoryError.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.