OpenAPITools / OpenAPITools/openapi-generator

[BUG] content-type application/json using iso-8859-1 instead of utf-8 (accent and special char not supported)

Open
#12,797 11 comments 13 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Issue: Bug
Dominant language
Java
Stars
26.8k
Forks
7.7k
PR merge metrics
PR metrics pending

Description

  • Have you provided a full/minimal spec to reproduce the issue?
  • [ x Have you validated the input using an OpenAPI validator (example)?
  • Have you tested with the latest master to confirm the issue still exists?
  • Have you searched for related issues/PRs?
  • What's the actual output vs expected output?
  • [Optional] Sponsorship to speed up the bug fix or feature request (example)
Description

Using the plugin org.openapitools:openapi-generator-maven-plugin:6.0.1 to generator a Java client from an openapi3.yml spec file, I have noticed requestBody content-type application/json generate a Java Api that does not support accent or special characters.

For example )çàç!è!§è(‘“é‘(§‘“é&“’(§è!çà send to the Java API is transformed to )???!?!??(????(????&??(??! in the HTTP call (wireshark inspection).

Problem : request content-type application/json is parsed as application/json with no charset. Therfore, using apache-httpclient, the generator use org.apache.httpcomponents:httpcore where the default charset iso-8859-1 which does not support accent.

application/json with ISO-8859-1 it is not compliant with RFC4627. In my case it lead to unwanted behavior in Java HTTP call such as unexpected character replacement.

openapi-generator version

org.openapitools:openapi-generator-maven-plugin:6.0.1
library : apache-httpclient

OpenAPI declaration file content or url
openapi: 3.0.1
info:
  version: 1.0.0
  title: Example
paths:
  /test-body:
    put:
      tags:
        - Test
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/TestBodys'
      responses:
        200:
          description: OK
          content:
            text/plain:
              schema:
                type: string
                example: pong
components:
  schemas:
    TestBodys:
      type: object
      properties:
        test_body:
          type: array
          items:
            $ref: '#/components/schemas/TestBody'

    TestBody:
      type: object
      properties:
        code:
          type: string
        type:
          type: string
        value:
          type: object
Generation Details

pom.xml

<plugin>
    <groupId>org.openapitools</groupId>
    <artifactId>openapi-generator-maven-plugin</artifactId>
    <version>6.0.1</version>
    <executions>
        <execution>
            <phase>generate-sources</phase>
            <goals>
                <goal>generate</goal>
            </goals>
            <configuration>
                <generatorName>java</generatorName>
                <inputSpec>${project.basedir}/src/main/resources/openapi/connect/openapi3.yml</inputSpec>
                <!-- api final path: output + sourceFolder + apiPackage -->
                <apiPackage>com.test.client.generated.api</apiPackage>
                <!-- model final path: output + sourceFolder + modelPackage -->
                <modelPackage>com.test.client.generated.model</modelPackage>
                <!-- disable unused code generation -->
                <generateApiTests>false</generateApiTests>
                <generateModelTests>false</generateModelTests>
                <generateApiDocumentation>false</generateApiDocumentation>
                <generateModelDocumentation>false</generateModelDocumentation>
                <typeMappings>
                    <typeMapping>OffsetDateTime=java.time.Instant</typeMapping>
                </typeMappings>
                <!-- disable pom.xml and other unwanted file generation -->
                <supportingFilesToGenerate>
                    ApiClient.java,ServerConfiguration.java,ServerVariable.java,JavaTimeFormatter.java,StringUtil.java,Authentication.java,HttpBasicAuth.java,HttpBearerAuth.java,ApiKeyAuth.java,ApiException.java,Configuration.java,Pair.java,auth/Authentication.java,RFC3339DateFormat.java
                </supportingFilesToGenerate>
                <configOptions>
                    <library>apache-httpclient</library>
                    <dateLibrary>java8</dateLibrary>
                    <sourceFolder>src/main/java/</sourceFolder>
                </configOptions>
                <output>${project.basedir}</output>
            </configuration>
        </execution>
    </executions>
</plugin>
Steps to reproduce

Generator the Java client using the OpenApi plugin generator.
Then use the ApiClient TestBody api to send special character such as )çàç!è!§è(‘“é‘(§‘“é&“’(§è!çà
Using wireshark or another tool inspect the payload request content.
=> Special caracters has been replaces by interrogation point.

Related issues/PRs

Not found

Suggest a fix

Use UTF-8 charset proposed by apache httpcore such as here in httpcore.

Inside generated ApiClient.java, the getContentType method is :

private ContentType getContentType(String headerValue) throws ApiException {
        try {
            return ContentType.parse(headerValue);
        } catch (org.apache.http.ParseException var3) {
            throw new ApiException("Could not parse content type " + headerValue);
        }
    }

And can be replaced by the following to retrieve an utf8 charset inside the content-type :

private ContentType getContentType(String headerValue) {
       return ContentType.getByMimeType(headerValue);
    }

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the generated ApiClient.java getContentType method and the Java client produced from the supplied openapi3.yml and pom.xml configuration. Reproduce the request with accented characters using the apache-httpclient library, then verify that the generated client sends those characters without replacement characters and add coverage for the corrected content type handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.