microsoft / microsoft/STL

`<regex>`: Character ranges with collating symbol bounds are rejected

Open
#5,391 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug regex
Dominant language
C++
Stars
11.1k
Forks
1.7k
Avg merge
4d 15h
Merged PRs (30d)
22

Description

The regex parser currently rejects ranges in character classes the bounds of which are given by collating symbols.

Test case

#include <algorithm>
#include <locale>
#include <regex>
#include <string>

using namespace std;

class collating_symbols_regex_traits : public regex_traits<char> {

public:
    template <class FwdIt>
    string_type lookup_collatename(FwdIt first, FwdIt last) const {
        // from Hungarian
        const string_type collating_symbol1 = "cs";
        const string_type collating_symbol2 = "dzs";

        if (std::equal(first, last, begin(collating_symbol1), end(collating_symbol1))) {
            return collating_symbol1;
        }

        if (std::equal(first, last, begin(collating_symbol2), end(collating_symbol2))) {
            return collating_symbol2;
        }

        return regex_traits::lookup_collatename(first, last);
    }
};

int main() {
    basic_regex<char, collating_symbols_regex_traits> r;
    r.imbue(locale("hu_HU"));
    try {
        r.assign("[[.cs.]-[.dzs.]]");
        printf("construction succeeded");
    } catch (const regex_error& e) {
        printf("regex error thrown");
    }
    return 0;
}

No Godbolt link: This succeeds there because the latest provided MSVC STL doesn't include the fix for #4995 yet.

The construction of the regex object should succeed, but this actually throws a regex_error with code error_range.

Note: Fixing this for multi-character collating symbols requires changes to the layout of the NFA node that represents character classes. I think this is doable without breaking ABI, but the fix has to be carefully thought through.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the supplied C++ test case at basic_regex::assign with the custom collating-symbol traits and locale. Trace the regex parser and the NFA character-class representation, paying attention to multi-character collating symbols and ABI constraints. Done means constructing "[[.cs.]-[.dzs.]]" succeeds instead of throwing error_range.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.