NaturalIntelligence / NaturalIntelligence/fast-xml-parser

Carriage Return characters (\r) are incorrectly replaced with Newline characters (\n)

Open
#512 12 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Pending
Dominant language
JavaScript
Stars
3.1k
Forks
395
Avg merge
1d 20h
Merged PRs (30d)
1

Description

  • Are you running the latest version?
  • Have you included sample input, output, error, and expected output?
  • Have you checked if you are using correct configuration?
  • Did you try online tool?
Description

Carriage Return characters (\r) found within parsed XML are incorrectly converted to newline characters (\n).

I did a quick scan of the source code and found a likely culprit:

const parseXml = function(xmlData) {
  xmlData = xmlData.replace(/\r\n?/g, "\n"); //TODO: remove this line

That pretty clearly replaces any \r character (and possibly \r\n pair) with \n.

The online tool does not exhibit this issue because browser node access APIs encode Carriage Return "strings" (the character \ followed by r) as \\r. The regular expression no longer matches. See:

> "1) \r 2) \\r 3) \n 4) \\n 5) \r\n 6) \\r\\n".replace(/\r\n?/g, "\n")
  '1) \n 2) \\r 3) \n 4) \\n 5) \n 6) \\r\\n'
Input

The XML that exhibits this issue is of the form:

<properties object="" engine="">
    <property type="string" name="x" state="changed">
        <![CDATA[This is a carriage return \r...]]>
    </property>
    <property type="string" name="y" state="changed">
        <![CDATA[\r]]>
    </property>
</properties>
Code

The code is pretty straightforward.

const XML_OPTIONS_NO_TAG_PARSE: fastXMLParser.X2jOptionsOptional = {
    attributeNamePrefix: "@",
    ignoreAttributes: false,
    parseAttributeValue: false,
    parseTagValue: false,
    textNodeName: "#value",
};
const XML_PARSER_NO_TAG_PARSE = new fastXMLParser.XMLParser(XML_OPTIONS_NO_TAG_PARSE);

// ...

const parsed = XML_PARSER_NO_TAG_PARSE.parse(xmlData);

After that code runs, the parsed text node content has \n instead of the expected \r.

Output

Running the above results in the following JSON:

{
    "properties": {
        "property": [
            {
                "#value": "This is a carriage return \n...",
                "@type": "string",
                "@name": "x",
                "@state": "changed"
            },
            {
                "#value": "\n",
                "@type": "string",
                "@name": "y",
                "@state": "changed"
            },
        ],
        "@object": "",
        "@engine": ""
    }
}
Expected Data

I expect the following output:

{
    "properties": {
        "property": [
            {
                "#value": "This is a carriage return \r...",
                "@type": "string",
                "@name": "x",
                "@state": "changed"
            },
            {
                "#value": "\r",
                "@type": "string",
                "@name": "y",
                "@state": "changed"
            },
        ],
        "@object": "",
        "@engine": ""
    }
}

Would you like to work on this issue?

  • Yes
  • No

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in src/xmlparser/OrderedObjParser.js at the parseXml function and inspect the normalization of carriage returns. Reproduce the issue with the CDATA XML shown in the report, then verify that parsed text preserves \r characters as expected without changing the other documented input behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.