dmlc / dmlc/xgboost

Portable RNG state: measurements on the Philox throughput question from #12485

Open
#12,509 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
28.8k
Forks
8.9k
Avg merge
1d 12h
Merged PRs (30d)
54

Description

#12459 is still open. A Booster trained on Linux fails to unpickle on Windows, because `Context::rng_` is a `std::mt19937` and its serialized form is not portable across standard libraries. libstdc++ writes 624 raw words plus a position; libc++ and the MSVC STL write 624 rotated words with none. Windows fails loudly on a Linux checkpoint, macOS loads a structurally valid but rotated state and fails silently.

I opened #12485 for this and closed it after review. Two of the three points there were clearly right: the `SerializableRandomEngine` wrapper was unnecessary, since Philox's stream operator is well defined, and there was an inefficiency in the multiplies. The third, "the performance here is 1/10 of mt", I could not reproduce, and since it was the reason the approach was set aside, I went and measured it properly rather than assert otherwise.

Raw data and harness: https://github.com/sinhaparth5/xgboost/tree/philox-multiply-probe/ci_probe

## The 1/10 figure appears at -O0

Sapphire Rapids (Xeon 8481C), dedicated vCPU, pinned, 15 repetitions, median CV under 0.3%. ns per draw, steady-state sequential:

| build | `std::mt19937` | Philox | ratio |
|---|---|---|---|
| `g++ -O0` | 11.09 | 161.75 | **0.07x** |
| `g++ -O2` | 2.35 | 3.96 | 0.59x |
| `g++ -O3` | 1.84 | 3.07 | 0.60x |
| `clang -O0` | 7.89 | 195.22 | **0.04x** |
| `clang -O2` | 2.50 | 3.74 | 0.67x |
| `clang -O3` | 2.36 | 3.74 | 0.63x |

Unoptimised, `UMul64` stays an out-of-line call made twice per lane per round, and the ratio lands between 0.04x and 0.10x. Optimised, it is 0.59-0.67x. CMake emits no optimisation flags at all when `CMAKE_BUILD_TYPE` is unset.

Optimised Philox is still slower than mt19937 at drawing numbers in a loop, by roughly a quarter to a third. That cost is there and I am not arguing it away. It is not an order of magnitude.

## The multiply inefficiency is the duplicate product

`MulHi` and `MulLo` each compute the same `a * b`. Multiply instructions in the generated block function:

| compiler | as-is | `MulHi` specialised | single-product `MulHiLo` |
|---|---|---|---|
| GCC 13.3 `-O3` | 20 | 20 | 20 |
| clang 18.1.3 `-O3` | 40 | 40 | **20** |
| MSVC 19.51 `/O2` | 24 | 20 | **13** |

Ten rounds times two lanes means 20 is one multiply per lane per round. GCC already folds `UMul64` and CSEs the duplicate away, so a hand-written wide-multiply specialisation changes nothing there. clang folds but does not CSE, paying double. Taking both halves from a single product fixes all three, and is worth 1.22x on clang in wall time (0.63x -> 0.77x at `-O3`). It is ~15 lines and produces a bit-identical stream, verified against `[rand.predef]`'s 10000th-draw constant 1955073260 and 500k draws across variants.

## XGBoost does not use the engine in steady state

`ColumnSampler::ColSample` builds a fresh engine on every call:

```cpp
auto seed = ctx->Rng()();
RandomEngine rng(seed);
std::shuffle(new_features.HostVector().begin(), ..., rng);
```

So the relevant cost is construction plus a few hundred draws, not a long sequential run. Constructing `std::mt19937` initialises 624 words. Constructing Philox sets a key and a counter. ns per `ColSample` call, GCC `-O3`:

| n features | `std::mt19937` | Philox | |
|---|---|---|---|
| 16 | 1621 | 59 | **27.3x faster** |
| 64 | 1739 | 235 | **7.4x faster** |
| 256 | 2201 | 854 | **2.6x faster** |
| 1024 | 4014 | 3368 | 1.19x faster |
| 4096 | 12359 | 13681 | 0.90x slower |

mt19937 goes from 1621 ns to 1739 ns for four times the draws. Almost all of it is the state init. clang agrees within a few percent. The crossover is past ~2000 features, so for typical feature counts the counter-based engine is ahead at the site that actually matters, and the steady-state deficit does not reach the workload.

## What I would like to propose

`RandomEngine` becomes a Philox engine following `[rand.eng.philox]`, so `rng_state` serializes as a key and a counter and survives a platform hop. That is #12485 without the wrapper and with the single-product multiply, which addresses both of the points that were right.

End-to-end training time is in a comment below: 0.37% apart on a two-second run, which I read as no regression rather than a win. And I recognise the closing comment mentioned reworking the PRNG at a higher level; if that is still the plan, I would rather contribute to it than duplicate it, so say the word and I will hold off. Mainly I did not want #12459 to sit open on the assumption of a 10x cost that I could not find outside an unoptimised build.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with Context::rng_, RandomEngine, and ColumnSampler::ColSample to understand how rng_state is created, serialized, and consumed. Review the ci_probe harness and the findings from #12485, then verify the proposed Philox behavior against the reported draw constants and benchmark cases. Done means portable checkpoint state with the single-product multiply and no meaningful training-time regression.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.