eic / eic/xrootd-mcp-server

Avoid full ROOT file downloads for metadata queries (use JSROOT remote access or partial reads)

Open
#58 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1
Forks
1
Avg merge
15h 53m
Merged PRs (30d)
7

Description

## Background

Currently, the server downloads the entire remote ROOT file to answer questions about tree/event/collection metadata (e.g., entry counts, branch info, etc.) using `ROOTAnalyzer`. This happens whenever methods like `analyzeFile`, `getEventStatistics`, etc. are called, even though only a small portion of the file is needed for such queries.

This approach is inefficient, especially for large files or datasets with many files, since only the file header, directory structure, and key TTree objects need to be read. Downloading full files can be slow and resource‐intensive.

## Proposal

### 1. Use JSROOT's Remote-File Support
JSROOT supports reading ROOT files directly via HTTP/HTTPS or XRootD URLs, issuing byte-range requests to retrieve only the required metadata blocks. Instead of downloading the file into a buffer and passing a blob to `openFile`, pass the file URL directly to JSROOT:

```js
// Instead of:
const fileData = await this.xrootdClient.readFile(remotePath);
const blob = new Blob([new Uint8Array(fileData)]);
const file = await openFile(blob);

// Use:
const file = await openFile('https://xrootd-server.org/path/to/file.root'); // or root:// url, if JSROOT supports
```

This results in JSROOT fetching only the bytes necessary for metadata queries, significantly reducing transfer time and load.

### 2. (Alternative) Use xrdcp --range or other byte-range approaches
If HTTP endpoints are unavailable, implement logic to read only the file header and key/streamer/TTree objects using partial reads over `root://`, reusing existing range-support in `xrdcp`. This requires a more complex parser, but is possible.

### 3. Cache ROOT File Analysis Results
To avoid repeat downloads for the same (unchanged) file, implement a cache keyed on file path and modification time.

### 4. (Optional) Parallelize Dataset-Wide Operations
For dataset-wide stats, process files in parallel (up to a safe concurrency limit) to improve wall-clock performance.

## References
- [JSROOT Partial Read Example](https://jsroot.gsi.de/latest/examples.htm?file=https://xrootd-public.cern.ch//store/test/root_v6.14.00/Run2012B_SingleMu.root&item=Events)
- [uproot](https://github.com/scikit-hep/uproot5) for reference implementation in Python

## Impact
- Large reduction in bandwidth and latency for all metadata queries
- Makes interactive metadata browsing with large datasets practical
- If combined with caching and (if needed) parallel fetches, the server would be much more scalable for production use

---

**Summary:** Instead of always downloading full ROOT files for metadata queries, support partial-IO approaches (via JSROOT with remote URLs or range reads), and cache results for repeated queries.

Contributor guide

Open the contributing guide

Research direction

Trace analyzeFile and getEventStatistics through ROOTAnalyzer and the current openFile path to identify where complete remote files are downloaded. Compare JSROOT remote URLs with the proposed xrdcp range approach, then define which metadata operations, caching behavior, and validation tests constitute completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.