`booster.predict(..., pred_leaf = True)` way slower than other flags.

Why is predicting the leaf **way** slower than actual predicting? Doesn't xgboost have to get the leaf anyway for the predicted value?

```python
X, y = make_regression(n_samples = 100000, n_features = 10)
dmatrix = xgb.DMatrix(X, label = y)

model = xgb.XGBRegressor(n_estimators = 20, max_depth = 6)
model.fit(X, y)
booster = model.get_booster()

%%timeit
booster.predict(dmatrix, pred_leaf = True)
# 22.9 ms ± 626 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

%%timeit
booster.predict(dmatrix)
# 385 µs ± 26 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)
```

(Isolating a single booster produces similar results)