HydrologicEngineeringCenter / HydrologicEngineeringCenter/cwms-python
JSON data from fetch can't be directly used to store again
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 16
- Forks
- 11
- Avg merge
- 4d 6h
- Merged PRs (30d)
- 8
Description
At least for the /locations endpoint, the json data from the fetch result can't be passed again into the appropriate store endpoint. CDA reports a serialization error.
Purpose:
I need to load all the locations from our production instance into cwbi-dev, so I wanted to use cwms-python to dump all the locations and insert them into cwbi.
Here's the script I used:
import os
import cwms
CWBI_API_KEY="apikey"
CWBI_API_ROOT="https://water.dev.cwbi.us/cwms-data"
cwms.api.init_session(api_root=os.environ.get("CDA_APT_ROOT", None), api_key=os.environ.get("CDA_APT_KEY", None))
locations = cwms.get_locations_catalog(office_id='SPK', unit_system='EN')
cwms.api.init_session(api_root=CWBI_API_ROOT, api_key=CWBI_API_KEY)
for location in locations.json['entries']:
print(location)
cwms.store_location(data=location, fail_if_exists=False)
The JSON has different key names than CDA returns, or expects in the POST.
For example:
office-id -> office
timezone-name -> time-zone
and many more.
Expectation:
I would expect the JSON data from the get_ method to match what CDA would return, especially since I have to supply JSON data to the relevant store_ methods in cwms-python.
Additional info:
Here's the JSON returned by CDA:
{
"office-id": "SPK",
"name": "Truckee R-Vista",
"latitude": 0,
"longitude": 0,
"active": true,
"public-name": "Truckee River At Vista, NV",
"timezone-name": "US/Pacific",
"location-kind": "SITE",
"nation": "US",
"state-initial": "NV",
"county-name": "Washoe",
"nearest-city": "Sparks",
"horizontal-datum": "NAD27",
"published-longitude": 119.7,
"published-latitude": 39.520555555556,
"vertical-datum": "NAVD88",
"elevation": 0,
"bounding-office-id": "UNK",
"elevation-units": "ft"
}
Here's the JSON returned by cwms-python:
{
"office": "SPK",
"name": "Truckee R-Vista",
"nearest-city": "Sparks",
"public-name": "Truckee River At Vista, NV",
"kind": "SITE",
"time-zone": "US/Pacific",
"latitude": 0.0,
"longitude": 0.0,
"published-latitude": 39.520555555556,
"published-longitude": 119.7,
"horizontal-datum": "NAD27",
"elevation": 0.0,
"unit": "ft",
"vertical-datum": "NAVD88",
"nation": "United States",
"state": "NV",
"county": "Washoe",
"bounding-office": "UNK",
"active": true,
"aliases": [
{"name": "Agency Aliases-USGS Station Name", "value": "Truckee River At Vista, NV"},
{"name": "Agency Aliases-USGS Station Number", "value": "10350000"}
]
}
The "aliases" key is acceptable for storing data, so I'm not sure why it doesn't show up on CDA when fetching via swagger.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing cwms.get_locations_catalog and cwms.store_location for the /locations endpoint, then compare their serialization and deserialization mappings with the CDA JSON shown in the issue. Verify the location object returned by the fetch path can be supplied to the store path, including aliases, and confirm the round trip no longer produces a serialization error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100