boostorg / boostorg/regex

Assertion failure in `parse_perl_extension()` when `newline_alt` is combined with `no_except`

Open
#285 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
119
Forks
113
PR merge metrics
No merged PRs in 30d

Description

# Bug Report

A 6-byte pattern makes `basic_regex::assign()` abort on
`BOOST_REGEX_ASSERT` in `basic_regex_parser<>::parse_perl_extension()`
([basic_regex_parser.hpp:2613](https://github.com/boostorg/regex/blob/master/include/boost/regex/v5/basic_regex_parser.hpp#L2613)),
when the syntax options combine `newline_alt` with `no_except`.

`no_except` is documented as the way to ask Boost.Regex *not* to fail hard on an invalid
expression, so aborting the process on one is the opposite of what the flag is for.

**Scope, up front:** with `NDEBUG` defined the assertion is compiled out and nothing bad
happens — `assign()` returns and `status()` reports an error, which is the correct
behaviour. So this affects assertion-enabled builds only (debug builds, sanitizer builds,
OSS-Fuzz, anything using `BOOST_ENABLE_ASSERT_HANDLER`). I did not find any
memory-safety consequence in a release build; see "Release builds" below.

## Version

Reproduced against `boostorg/regex` master, commit `a640597` (2026-05-22), Boost.Regex v5.
I have not tested tagged releases, but the code at the assert is unchanged on master today.

## Reproducer

Self-contained — needs only this repository, in standalone mode, no other Boost libraries:

```cpp
#include
#include
#include

int main()
{
// 6 bytes: ( ? : \ R 0x0c
const char pat[] = { '(', '?', ':', '\\', 'R', '\x0c' };

const boost::regex_constants::syntax_option_type flags =
static_cast(
boost::regbase::newline_alt | boost::regex_constants::no_except);

boost::basic_regex re;
try {
re.assign(pat, pat + sizeof(pat), flags);
} catch (const std::exception& e) {
std::printf("threw: %s\n", e.what());
return 0;
}
std::printf("returned, status = %d\n", static_cast(re.status()));
return 0;
}
```

```console
$ g++ -std=c++17 -DBOOST_REGEX_STANDALONE -I regex/include -o repro repro.cpp
$ ./repro
repro: regex/include/boost/regex/v5/basic_regex_parser.hpp:2613:
bool boost::re_detail_600::basic_regex_parser::parse_perl_extension()
[with charT = char; traits = boost::regex_traits]:
Assertion `this->m_traits.syntax_type(*m_position) == regex_constants::syntax_close_mark' failed.
Aborted (core dumped)
```

The `basic_regex` constructor taking the same flags aborts identically; it is not specific
to `assign()`.

## What is required to trigger it

All of `(?:`, `\R`, a following byte that is not `)`, and both flags are needed. Same flags
throughout, only the pattern varies:

| pattern | result |
| --- | --- |
| `(?:\R\x0c` | **assertion failure** |
| `(?:\R` | returns, `status() == 13` |
| `(?:\R)` | returns, `status() == 13` |
| `\R\x0c` | returns, `status() == 13` |
| `(?:\N\x0c` | returns, `status() == 5` |
| `(?:x\x0c` | returns, `status() == 8` |

And with the pattern fixed at `(?:\R\x0c`, both flags are required — `newline_alt` alone
throws a `regex_error`, `no_except` alone returns cleanly. Neither aborts.

## Root cause

Boost.Regex expands `\R` internally into an atomic group. Its own diagnostic shows the
expansion, if you run the same pattern with `newline_alt` but *without* `no_except`
(non-printables escaped by me):

```
Invalid preceding regular expression prior to repetition operator.
The error occurred while parsing the regular expression fragment:
'?-x:(?>\x0d\x0a?>>>HERE>>>|[\x0a\x0b\x0c\x85]))'.
```

That is `\R` → `(?>\r\n?|[\n\v\f\x85])`. The `>>>HERE>>>` marker sits immediately after the
`\x0a`, i.e. **`newline_alt` is being applied to the parser's own synthesized expansion**:
the literal `\n` inside `(?>\r\n?|...)` is structural, but with `newline_alt` set it is
reinterpreted as an alternation `|`, leaving the `?` that follows with nothing to repeat.
This looks like the underlying defect — a user-supplied option is leaking into text the
parser generated itself.

From there the two flags interact:

- Without `no_except`, `fail()`
([basic_regex_parser.hpp:244](https://github.com/boostorg/regex/blob/master/include/boost/regex/v5/basic_regex_parser.hpp#L244))
raises `regex_error` and the parse unwinds.
- With `no_except`, `fail()` only records `m_status` and returns. Control returns to the
`parse_perl_extension()` frame handling the outer `(?:`, which checks for the two failure
shapes it knows about — `unwind_alts()` failing (line 2594) and running out of input
(line 2605) — and then asserts that what remains must be a `)`:

```cpp
if(m_position == m_end)
{
// Rewind to start of (? sequence:
...
return false;
}
BOOST_REGEX_ASSERT(this->m_traits.syntax_type(*m_position) == regex_constants::syntax_close_mark);
```

The assert encodes "if we did not run out of input and alternatives unwound, we are at
`)`", which no longer holds once an error can be recorded without unwinding.

## Release builds

With `-DNDEBUG` the assert disappears and every case above returns normally with an error
status (`13`), including the aborting one — so I could not produce a wrong-memory or
wrong-result outcome in a release build.

One separate oddity, present in both build configurations: `grep | no_except` on the same
pattern returns `status() == 0` with `size() == 6`, i.e. the expression is accepted as
*valid*, while `egrep | no_except` reports `status() == 13`. That may be worth a look
independently of the assert.

## Suggested direction

Two things look separable:

1. Stop `newline_alt` from applying to the internally generated `\R` expansion. That
removes the bogus parse error and, with it, this abort. It would also fix `(?:\R)` being
rejected under `newline_alt`.
2. Independently, make `parse_perl_extension()` tolerate the "an error was recorded but no
exception was thrown" state rather than asserting — e.g. check
`this->m_pdata->m_status` alongside the `m_position == m_end` test at line 2605 before
reaching line 2613. Otherwise the same shape is likely reachable from other error paths
whenever `no_except` is set.

## How this was found

Found by an automated fuzz-harness generation experiment built on OSS-Fuzz, then reduced
by hand to the 6-byte pattern and the two-flag combination above and re-verified with the
standalone program in this report — no fuzzing infrastructure, sanitizers, or OSS-Fuzz
build are needed to reproduce it.

I searched this repository's open and closed issues for `parse_perl_extension`,
`syntax_close_mark`, and `no_except`, and reviewed the recent issue list, and did not find
a match. Apologies if I missed one.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the standalone reproducer and read regex/include/boost/regex/v5/basic_regex_parser.hpp around fail() at line 244 and parse_perl_extension() at lines 2594-2613. Compare the parser state for newline_alt with and without no_except, including the synthesized R expansion. Done means the assertion-enabled reproducer no longer aborts and the documented error status remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.