OpenAPITools / OpenAPITools/openapi-generator

[BUG] Stripping ECMA Regex Leading and Trailing `/` Causes Errors

Open
#10,395 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Issue: Bug
Dominant language
Java
Stars
26.8k
Forks
7.7k
PR merge metrics
PR metrics pending

Description

Bug Report Checklist
  • Have you provided a full/minimal spec to reproduce the issue?
  • Have you validated the input using an OpenAPI validator (example)?
  • Have you tested with the latest master to confirm the issue still exists?
  • Have you searched for related issues/PRs?
  • What's the actual output vs expected output?
Description

Python code generation tries to strip leading and trailing / from regex patterns, but the pattern its using to do so matches when it shouldn't.

For example this pattern in my spec: [0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12} has no leading and trailing slashes, but the code linked above would still match and end up stripping off the trailing } which causes an error from the https://github.com/curious-odd-man/RgxGen library.

import java.util.regex.Pattern;
import java.util.regex.Matcher;


public class Example {

    public static String SHOULD_NOT_MATCH = "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}";
    public static String SHOULD_MATCH = "/^abc$/i";
    
    public static void main(String args[]) {
        Pattern valueExtractor = Pattern.compile("^/?(.+?)/?(.?)$");
        
        Matcher m = valueExtractor.matcher(SHOULD_NOT_MATCH);
        System.out.println(m.find());
        System.out.println(m.group(1));
        
        Matcher m2 = valueExtractor.matcher(SHOULD_MATCH);
        System.out.println(m2.find());
        System.out.println(m2.group(1));
    }
}

Screen Shot 2021-09-14 at 10 26 33 AM

Screen Shot 2021-09-14 at 10 26 57 AM

To be clear: this is causing an error for me, but it's possible this silently changing a lot of other regex and using that changed regex to generate examples.

openapi-generator version

v5.2.1

Don't think this is a regression, I got the same error on v5.1.X as well

OpenAPI declaration file content or url
openapi: 3.0.0
info:
  title: 'Example'
  version: v1
servers:
  -
    url: 'https://example.com'
    description: 'Example API API'
paths:
  '/{id}':
    get:
      operationId: example
      parameters:
        -
          name: id 
          in: path
          required: true
          schema:
            type: string
            pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
      responses:
        '200':
          description: example
          content:
            application/json:
              schema:
                type: string
Generation Details

I ran this with the docker container:

    docker run \
        --rm \
        -it \
        --volume "$(pwd)/generator-workspace:/workspace" \
        'openapitools/openapi-generator-cli:v5.2.1' \
        generate \
        --generator-name python \
        --output "/workspace/example" \
        --remove-operation-id-prefix \
        --invoker-package "$namespace" \
        --global-property modelTests=false,apiTests=false \
        --additional-properties packageName=alli.$namespace,library=asyncio \
        --input-spec "/workspace/openapi.yml"
Steps to reproduce
  1. Try to generate a schema with the example above
Related issues/PRs

n/a

Suggest a fix

I don't think the / should be optional here, you only want to match if they are present so having a ? behind each making them optional doesn't make sense.

import java.util.regex.Pattern;
import java.util.regex.Matcher;


public class Example {

    public static String SHOULD_NOT_MATCH = "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}";
    public static String SHOULD_MATCH = "/^abc$/i";
    
    public static void main(String args[]) {
        Pattern valueExtractor = Pattern.compile("^/(.+?)/(.?)$");
        
        Matcher m = valueExtractor.matcher(SHOULD_NOT_MATCH);
        if (m.find()) {
            System.out.println("found");
            System.out.println(m.group(1));
        }
        
        Matcher m2 = valueExtractor.matcher(SHOULD_MATCH);
        if (m2.find()) {
            System.out.println("found");
            System.out.println(m2.group(1));
        }
    }
}

The current regex only works with a single flag as well, if multiple flags need to be supported: Pattern.compile("^/(.+?)/(.+)?$").

I've only experiened this with the python generator, but I suspect the same bug is present everywhere RgxGen library is being used.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in modules/openapi-generator/src/main/java/org/openapitools/codegen/languages/PythonClientCodegen.java at the linked regex handling around lines 1167-1172, then reproduce the issue with the supplied OpenAPI schema and Python generator Docker command. Done means regex patterns without surrounding slashes are preserved and slash-delimited patterns still generate successfully without the RgxGen error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
api, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.