operator>> does not restore the character that terminates a number

Summary

operator>> is documented to leave the stream "positioned right after" the value it parsed, so that it can be called repeatedly on a stream of concatenated JSON values. That holds for strings, arrays, objects and literals, but not for numbers: a number is only terminated by the character following it, and that character is consumed from the stream and never put back. The next extraction therefore starts one byte too late.

This came up while reviewing #5326, which documents the parse() vs operator>> strictness difference and repeats the positioning guarantee from operator>>'s notes. Before we document the guarantee more prominently, we should decide whether to make it true or to qualify it.

Reproduction

#include <nlohmann/json.hpp>
#include <iostream>
#include <sstream>

int main()
{
    std::istringstream input("1true");
    nlohmann::json j1;
    input >> j1;                       // j1 == 1

    std::string rest;
    char c;
    while (input.get(c)) { rest += c; }
    std::cout << j1 << " | rest = \"" << rest << "\"\n";
}

Output:

1 | rest = "rue"

Expected (per the documented guarantee): rest = "true", and a second input >> j2 yielding true. Instead the second extraction throws:

[json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: 'r'

More cases (current behaviour):

input	first value	remaining stream
1true	1	rue ❌
1 true	1	true ✔ (the eaten byte was whitespace)
truefalse	true	false ✔
[1][2]	[1]	[2] ✔
null null	null	null ✔

So the guarantee only breaks when a number is immediately followed by a non-whitespace character — 1true, 1[2], 1{}, 1"a". Concatenated values that are whitespace-separated, or whose first value is not a number, work as documented.

Cause

lexer::scan_number() reads one character past the number and ungets it:

https://github.com/nlohmann/json/blob/dd24e2dffd734c24d71cd79d09dee32d72dde7ca/include/nlohmann/detail/input/lexer.hpp#L1277-L1280

But lexer::unget() is deliberately simulated — it only rewinds the lexer's own bookkeeping, not the input:

We implement unget by setting variable next_unget to true. The input is not changed - we just simulate ungetting by modifying chars_read_total, chars_read_current_line, and token_string.

and input_stream_adapter::get_character() consumes via sbumpc() with no corresponding sungetc():

https://github.com/nlohmann/json/blob/dd24e2dffd734c24d71cd79d09dee32d72dde7ca/include/nlohmann/detail/input/input_adapters.hpp#L127-L136

The simulated unget is correct and sufficient for parse()/accept(), which require the input to end after the value anyway. It is only observable when the caller keeps using the stream afterwards — i.e. operator>>, and sax_parse with strict == false.

Options

1. Fix the adapter. Give input_stream_adapter a real unget (sungetc(), or sputbackc() for the character just read) and have lexer::unget() propagate it to the adapter when the adapter supports it. This makes the documented guarantee hold. It is a behaviour change: code that today relies on the extra byte being swallowed would see it again. sungetc() can also fail if the streambuf has no putback room, which needs a decision (currently sbumpc() is used precisely to avoid depending on std::istream state).
2. Qualify the documentation. Keep the implementation as is and state in operator>>'s notes and in features/parsing that concatenated values must be separated by whitespace when the preceding value is a number. Cheapest, no behaviour change, but leaves a sharp edge.
3. Both: qualify the docs now (so #5326 can land accurately) and treat the adapter fix as a candidate for the next major release.

I lean towards 3, but since option 1 changes observable behaviour of operator>>, marking this for discussion.

Which version of the library did you use?

develop (dd24e2dff)

---

This issue was written by Claude Code.

Start by reading scan_number() and unget() in include/nlohmann/detail/input/lexer.hpp, then inspect get_character() in include/nlohmann/detail/input/input_adapters.hpp. Reproduce the 1true case and determine whether the project will restore the character, qualify the operator>> documentation, or do both. Done should include the chosen behavior being implemented or documented and verified against concatenated stream values.