christophertubbs / christophertubbs/PyRDB

Metadata is overwritten on data that shares pcodes

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
bug good first issue help wanted
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

From https://waterservices.usgs.gov/nwis/iv/?format=rdb&sites=08025360&siteStatus=all¶meterCd=00060:

```
#
# TS_ID - An internal number representing a time series.
#
# Data provided for site 08025360
# TS_ID Parameter Description
# 249485 00060 Discharge, cubic feet per second, [Total Spillway Releases]
# 249487 00060 Discharge, cubic feet per second, [Total Discharge from Reservoir]
# 271825 00060 Discharge, cubic feet per second, [Tailrace]
#
# Data-value qualification codes included in this output:
# P Provisional data subject to revision.
#
agency_cd site_no datetime tz_cd 249485_00060 249485_00060_cd 249487_00060 249487_00060_cd 271825_00060 271825_00060_cd
5s 15s 20d 6s 14n 10s 14n 10s 14n 10s
USGS 08025360 2025-05-27 10:15 CDT 13000 P 12700 P
USGS 08025360 2025-05-27 10:30 CDT 364 P
```

There are three sets of values represented that are represented by different `TS_ID`s, but identical parameter codes. When loaded via:

```python
data = pyrdb.RDB.from_url(
"https://waterservices.usgs.gov/nwis/iv/?format=rdb&sites=08025360&siteStatus=all¶meterCd=00060",
frame_processing_functions=[
pyrdb.NormalizeTimeZoneUnawareTransformation(
to_column_name='datetime',
date_column='datetime',
timezone_code_column='tz_cd'
)
]
)
```

You get:

```
>>> data['08025360'].frame
agency_cd site_no datetime tz_cd 00060 00060_cd 00060.1 00060_cd.1 00060.2 00060_cd.2
0 USGS 08025360 2025-05-27 15:15:00 CDT 356.0 P 13000.0 P 12700.0 P
```

If you look at the columns, though, you see:
```
>>> data['08025360'].columns
{'00060': 'Discharge, cubic feet per second, [Tailrace]'}
```
Meaning no explanation for what 00060.1 or 00060.2 are.

The following may be more ideal:

```
>>> data['08025360'].frame
agency_cd site_no datetime tz_cd discharge_total_spillway_releases discharge_total_spillway_releases_cd total_discharge_from_reservoir total_discharge_from_reservoir_cd discharge_tailwind discharge_tailwind_cd
0 USGS 08025360 2025-05-27 15:15:00 CDT 356.0 P 13000.0 P 12700.0 P
```

There may be a better way to do it since this format is a bit wordy. `data["id"].columns` probably also needs to be a bit better represented as well.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.