dmwm / dmwm/CMSSpark

Consider file pfn vs lfn in schemas

Open
#153 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4
Forks
1
PR merge metrics
No merged PRs in 30d

Description

thanks to some debugging, I notice that data in /project/monitoring/archive/xrootd does not contain lfns, but rather some mix of pfn and lfn. (presumably whatever name the user used to open the file). My guess is that this is more widespread than just this one monitoring source. Consider updating schema column names to reflect this to improved readability.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the schema definitions used for /project/monitoring/archive/xrootd and checking whether its values are PFNs, LFNs, or both. Compare other monitoring sources before deciding whether column names should change; done means the schema names accurately describe the data across the affected sources.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, spark
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.