huggingface / huggingface/xet-core
perf: download stream groups repeatedly rebuild HTTP clients and TLS roots
- Dominant language
- Rust
- Stars
- 592
- Forks
- 102
- Avg merge
- 5d 8h
- Merged PRs (30d)
- 9
Description
Creating short-lived download stream groups repeatedly rebuilds HTTP clients and reloads TLS trust roots, even when all groups share one `XetSession`. This is particularly expensive for random-access consumers that create a group for each reader/range operation.
Observed with `hf-xet = 1.6.0` on Linux (Rust 1.97.0, optimized build). The two construction patterns below are also present in current main at `7af65ba2989a6b386c61e71048309d1ea1472434`.
### Offline reproduction
No dataset, valid token, or network request is needed. The CAS endpoint and an unexpired dummy token are supplied, so the dummy refresh URL is never called. Do not replace the dummy values with real credentials.
```toml
[dependencies]
xet = { package = "hf-xet", version = "=1.6.0" }
http = "1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
```
```rust
use std::time::Instant;
use xet::xet_session::XetSessionBuilder;
#[tokio::main]
async fn main() -> Result<(), Box> {
let refresh = std::env::args().any(|arg| arg == "refresh");
let session = XetSessionBuilder::new().build()?;
let start = Instant::now();
for _ in 0..256 {
let mut builder = session
.new_download_stream_group()?
.with_endpoint("https://cas-server.xethub.hf.co")
.with_token_info("diagnostic-no-network", u64::MAX);
if refresh {
let mut headers = http::HeaderMap::new();
headers.insert(
http::header::AUTHORIZATION,
"Bearer diagnostic-no-network".parse()?,
);
builder = builder.with_token_refresh_url(
"https://example.invalid/xet-read-token",
headers,
);
}
let _group = builder.build().await?;
}
println!("refresh={refresh}, elapsed={:?}", start.elapsed());
Ok(())
}
```
Run `cargo run --release` and `cargo run --release -- refresh`. In our optimized, non-LTO build on an otherwise idle c7i.8xlarge, 256 groups took approximately **1.40 s without refresh** and **4.19 s with refresh**. Absolute timings depend on the environment, including its trust store. An offline constructor profile attributed substantial self CPU time to PEM/base64 decoding and trust-store filesystem work; no downloads or reconstruction were involved.
### Two sources of repeated construction
1. [`get_or_create_reqwest_client`](https://github.com/huggingface/xet-core/blob/7af65ba2989a6b386c61e71048309d1ea1472434/xet_runtime/src/core/common.rs) stores only one `(tag, client)` under `global_reqwest_client`. The refresh client uses default authorization headers, while CAS clients use a different header set. Group construction alternates tags A/B/A/B, so each group evicts the previous client rather than reusing both pools.
2. [`RemoteClient::new_with_socket`](https://github.com/huggingface/xet-core/blob/7af65ba2989a6b386c61e71048309d1ea1472434/xet_client/src/cas_client/remote_client.rs) eagerly constructs `shard_upload_http_client`, including for download groups. Its no-read-timeout HTTP-client path is explicitly uncached in [`http_client.rs`](https://github.com/huggingface/xet-core/blob/7af65ba2989a6b386c61e71048309d1ea1472434/xet_client/src/common/http_client.rs). This leaves a repeated construction cost even without token refresh.
A real consumer trigger is `object_store_opendal 0.60.1`: each `ObjectStore::get_ranges` constructs an OpenDAL reader. In `opendal-service-hf 0.59.1`, XET resolution and the download group are scoped to that reader. Sharing an operator or an XET session therefore does not amortize group construction across these calls. Many small range reads amplify the SDK cost. Reader/group reuse can reduce this overhead, but it also changes the consumer's resource and metadata lifetime; it should not be required just to reuse transport clients.
Could the SDK retain distinct compatible clients within the session/runtime cache, and initialize the upload-only client lazily or reuse an appropriately configured client? Client reuse must preserve isolation between different headers, sockets, and timeout configurations. This report concerns construction overhead, not a claim that XET reconstruction or network transfer is itself slow.
Contributor guide
Assessment
This issue has not been assessed yet.