Assertion failure in `parse_perl_extension()` when `newline_alt` is combined with `no_except`
- Dominant language
- C++
- Stars
- 119
- Forks
- 113
- PR merge metrics
- No merged PRs in 30d
Description
# Bug Report
A 6-byte pattern makes `basic_regex::assign()` abort on
`BOOST_REGEX_ASSERT` in `basic_regex_parser<>::parse_perl_extension()`
([basic_regex_parser.hpp:2613](https://github.com/boostorg/regex/blob/master/include/boost/regex/v5/basic_regex_parser.hpp#L2613)),
when the syntax options combine `newline_alt` with `no_except`.
`no_except` is documented as the way to ask Boost.Regex *not* to fail hard on an invalid
expression, so aborting the process on one is the opposite of what the flag is for.
**Scope, up front:** with `NDEBUG` defined the assertion is compiled out and nothing bad
happens — `assign()` returns and `status()` reports an error, which is the correct
behaviour. So this affects assertion-enabled builds only (debug builds, sanitizer builds,
OSS-Fuzz, anything using `BOOST_ENABLE_ASSERT_HANDLER`). I did not find any
memory-safety consequence in a release build; see "Release builds" below.
## Version
Reproduced against `boostorg/regex` master, commit `a640597` (2026-05-22), Boost.Regex v5.
I have not tested tagged releases, but the code at the assert is unchanged on master today.
## Reproducer
Self-contained — needs only this repository, in standalone mode, no other Boost libraries:
```cpp
#include
#include
#include
int main()
{
// 6 bytes: ( ? : \ R 0x0c
const char pat[] = { '(', '?', ':', '\\', 'R', '\x0c' };
const boost::regex_constants::syntax_option_type flags =
static_cast(
boost::regbase::newline_alt | boost::regex_constants::no_except);
boost::basic_regex re;
try {
re.assign(pat, pat + sizeof(pat), flags);
} catch (const std::exception& e) {
std::printf("threw: %s\n", e.what());
return 0;
}
std::printf("returned, status = %d\n", static_cast(re.status()));
return 0;
}
```
```console
$ g++ -std=c++17 -DBOOST_REGEX_STANDALONE -I regex/include -o repro repro.cpp
$ ./repro
repro: regex/include/boost/regex/v5/basic_regex_parser.hpp:2613:
bool boost::re_detail_600::basic_regex_parser::parse_perl_extension()
[with charT = char; traits = boost::regex_traits]:
Assertion `this->m_traits.syntax_type(*m_position) == regex_constants::syntax_close_mark' failed.
Aborted (core dumped)
```
The `basic_regex` constructor taking the same flags aborts identically; it is not specific
to `assign()`.
## What is required to trigger it
All of `(?:`, `\R`, a following byte that is not `)`, and both flags are needed. Same flags
throughout, only the pattern varies:
| pattern | result |
| --- | --- |
| `(?:\R\x0c` | **assertion failure** |
| `(?:\R` | returns, `status() == 13` |
| `(?:\R)` | returns, `status() == 13` |
| `\R\x0c` | returns, `status() == 13` |
| `(?:\N\x0c` | returns, `status() == 5` |
| `(?:x\x0c` | returns, `status() == 8` |
And with the pattern fixed at `(?:\R\x0c`, both flags are required — `newline_alt` alone
throws a `regex_error`, `no_except` alone returns cleanly. Neither aborts.
## Root cause
Boost.Regex expands `\R` internally into an atomic group. Its own diagnostic shows the
expansion, if you run the same pattern with `newline_alt` but *without* `no_except`
(non-printables escaped by me):
```
Invalid preceding regular expression prior to repetition operator.
The error occurred while parsing the regular expression fragment:
'?-x:(?>\x0d\x0a?>>>HERE>>>|[\x0a\x0b\x0c\x85]))'.
```
That is `\R` → `(?>\r\n?|[\n\v\f\x85])`. The `>>>HERE>>>` marker sits immediately after the
`\x0a`, i.e. **`newline_alt` is being applied to the parser's own synthesized expansion**:
the literal `\n` inside `(?>\r\n?|...)` is structural, but with `newline_alt` set it is
reinterpreted as an alternation `|`, leaving the `?` that follows with nothing to repeat.
This looks like the underlying defect — a user-supplied option is leaking into text the
parser generated itself.
From there the two flags interact:
- Without `no_except`, `fail()`
([basic_regex_parser.hpp:244](https://github.com/boostorg/regex/blob/master/include/boost/regex/v5/basic_regex_parser.hpp#L244))
raises `regex_error` and the parse unwinds.
- With `no_except`, `fail()` only records `m_status` and returns. Control returns to the
`parse_perl_extension()` frame handling the outer `(?:`, which checks for the two failure
shapes it knows about — `unwind_alts()` failing (line 2594) and running out of input
(line 2605) — and then asserts that what remains must be a `)`:
```cpp
if(m_position == m_end)
{
// Rewind to start of (? sequence:
...
return false;
}
BOOST_REGEX_ASSERT(this->m_traits.syntax_type(*m_position) == regex_constants::syntax_close_mark);
```
The assert encodes "if we did not run out of input and alternatives unwound, we are at
`)`", which no longer holds once an error can be recorded without unwinding.
## Release builds
With `-DNDEBUG` the assert disappears and every case above returns normally with an error
status (`13`), including the aborting one — so I could not produce a wrong-memory or
wrong-result outcome in a release build.
One separate oddity, present in both build configurations: `grep | no_except` on the same
pattern returns `status() == 0` with `size() == 6`, i.e. the expression is accepted as
*valid*, while `egrep | no_except` reports `status() == 13`. That may be worth a look
independently of the assert.
## Suggested direction
Two things look separable:
1. Stop `newline_alt` from applying to the internally generated `\R` expansion. That
removes the bogus parse error and, with it, this abort. It would also fix `(?:\R)` being
rejected under `newline_alt`.
2. Independently, make `parse_perl_extension()` tolerate the "an error was recorded but no
exception was thrown" state rather than asserting — e.g. check
`this->m_pdata->m_status` alongside the `m_position == m_end` test at line 2605 before
reaching line 2613. Otherwise the same shape is likely reachable from other error paths
whenever `no_except` is set.
## How this was found
Found by an automated fuzz-harness generation experiment built on OSS-Fuzz, then reduced
by hand to the 6-byte pattern and the two-flag combination above and re-verified with the
standalone program in this report — no fuzzing infrastructure, sanitizers, or OSS-Fuzz
build are needed to reproduce it.
I searched this repository's open and closed issues for `parse_perl_extension`,
`syntax_close_mark`, and `no_except`, and reviewed the recent issue list, and did not find
a match. Apologies if I missed one.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the standalone reproducer and read regex/include/boost/regex/v5/basic_regex_parser.hpp around fail() at line 244 and parse_perl_extension() at lines 2594-2613. Compare the parser state for newline_alt with and without no_except, including the synthesized R expansion. Done means the assertion-enabled reproducer no longer aborts and the documented error status remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100